LINUX(centos)系統系統安裝EMC多路徑原系統內核未卸載,導致機器宕機處理過程
# last 看當前連接用戶，和之前連接用戶
# uptime
09:57:30 up 12:26, 4 users, load average: 0.70, 0.78, 0.93
# who -b 最近一次系統啓動時間
system boot 2020-04-20 21:38
查看當前系統版本的版本號
# cat /etc/system-release Oracle Linux Server release 7.6
查看當前emc版本
# powermt version Dell EMC powermt for PowerPath (c) Version 7.1 (build 75)
查看當前的emc存儲路徑
# powermt display VPLEX logical device count=25 ============================================================================== ----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------ ### HW Path Summary Total Dead IO/Sec Q-IOs Errors ============================================================================== 7 qla2xxx optimal 50 0 - 0 0 10 qla2xxx optimal 50 0 - 0 0
# powermt display dev=all |more #看到之前2條鏈路，分別跟4個啓動器劃zone,但這邊一條路徑只有2個啓動器 Pseudo name=emcpowera VPLEX ID=CKM00192900895 Logical device ID=6000144000000010E01ACEF0A76B9A33 [device_Symm0205_042E_1_vol] state=alive; policy=ADaptive; queued-IOs=0 ============================================================================== --------------- Host --------------- - Stor - -- I/O Path -- -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 10 qla2xxx sdcu CL1-00 active alive 0 0 10 qla2xxx sdbf CL1-03 active alive 0 0 7 qla2xxx sdah CL1-07 active alive 0 0 7 qla2xxx sdbt CL1-04 active alive 0 0 Pseudo name=emcpowerb VPLEX ID=CKM00192900895 Logical device ID=6000144000000010E01ACEF0A76B9A2E [device_Symm0205_042D_1_vol] state=alive; policy=ADaptive; queued-IOs=0 ============================================================================== --------------- Host --------------- - Stor - -- I/O Path -- -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 10 qla2xxx sdbd CL1-03 active alive 0 0 10 qla2xxx sdct CL1-00 active alive 0 0 7 qla2xxx sdag CL1-07 active alive 0 0 7 qla2xxx sdbs CL1-04 active alive 0 0 .......省略........
查看emc盤邏輯設備 # powermt display dev=all |grep -i logical Logical device ID=6000144000000010E01ACEF0A76B9A33 [device_Symm0205_042E_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A2E [device_Symm0205_042D_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A29 [device_Symm0205_042C_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A23 [device_Symm0205_042B_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A1E [device_Symm0205_042A_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A19 [device_Symm0205_0429_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A14 [device_Symm0205_0428_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A0F [device_Symm0205_0427_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A0A [device_Symm0205_0426_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A05 [device_Symm0205_0425_1_vol] Logical device ID=6000144000000010E01ACEF0A76B99FF [device_Symm0205_0424_1_vol] Logical device ID=6000144000000010E01ACEF0A76B99FA [device_Symm0205_0423_1_vol] Logical device ID=6000144000000010E01ACEF0A76B99F5 [device_Symm0205_0422_1_vol] Logical device ID=6000144000000010E01ACEF0A76B99F0 [device_Symm0205_0421_1_vol] Logical device ID=6000144000000010E01ACEF0A76B99EB [device_Symm0205_0420_1_vol] Logical device ID=6000144000000010E01ACEF0A76B99E6 [device_Symm0205_041F_1_vol] Logical device ID=6000144000000010E01ACEF0A76B99E1 [device_Symm0205_041E_1_vol] Logical device ID=6000144000000010E01ACEF0A76B99DC [device_Symm0205_041D_1_vol] Logical device ID=6000144000000010E01ACEF0A76B99D7 [device_Symm0205_041C_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A39 [device_Symm0205_042F_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A3F [device_Symm0205_0430_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A48 [device_Symm0205_0434_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A4D [device_Symm0205_0433_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A52 [device_Symm0205_0432_1_vol] Logical device ID=6000144000000010E01ACEF0A76B9A57 [device_Symm0205_0431_1_vol]
查看emc盤邏輯設備映射的盤
# powermt display dev=all |grep -i pseudo Pseudo name=emcpowera Pseudo name=emcpowerb Pseudo name=emcpowerc Pseudo name=emcpowerd Pseudo name=emcpowere Pseudo name=emcpowerf Pseudo name=emcpowerg Pseudo name=emcpowerh Pseudo name=emcpoweri Pseudo name=emcpowerj Pseudo name=emcpowerk Pseudo name=emcpowerl Pseudo name=emcpowerm Pseudo name=emcpowern Pseudo name=emcpowero Pseudo name=emcpowerp Pseudo name=emcpowerq Pseudo name=emcpowerr Pseudo name=emcpowers Pseudo name=emcpowert Pseudo name=emcpoweru Pseudo name=emcpowerv Pseudo name=emcpowerw Pseudo name=emcpowerx Pseudo name=emcpowery
# cd /etc/udev/ # ls hwdb.bin rules.d udev.conf # cd rules.d/ # ls 05-udev-pp.rules 70-persistent-ipoib.rules 80-udev-pp-xio.rules 99-oracle.rules 99-vmware-scsi-timeout.rules # ls -l total 20 -rw-r--r--. 1 root root 76 Apr 20 21:38 05-udev-pp.rules -rw-r--r--. 1 root root 628 Jun 12 2018 70-persistent-ipoib.rules -rw-r--r--. 1 root root 258 Apr 20 21:39 80-udev-pp-xio.rules -rw-r--r--. 1 root root 56 Mar 31 12:23 99-oracle.rules -rw-r--r--. 1 root root 148 Nov 1 2018 99-vmware-scsi-timeout.rules
# cat 70-persistent-ipoib.rules
# This is a sample udev rules file that demonstrates how to get udev to
# set the name of IPoIB interfaces to whatever you wish. There is a
# 16 character limit on network device names.
#
# Important items to note: ATTR{type}=="32" is IPoIB interfaces, and the
# ATTR{address} match must start with ?* and only reference the last 8
# bytes of the address or else the address might not match the variable QPN
# portion.
#
# Modern udev is case sensitive and all addresses need to be in lower case.
#
# ACTION=="add", SUBSYSTEM=="net", DRIVERS=="?*", ATTR{type}=="32", ATTR{address}=="?*00:02:c9:03:00:31:78:f2", NAME="mlx4_ib3"查看數據盤配置文件
# cat 99-oracle.rules KERNEL=="emcpower[a-z]*", OWNER="grid",GROUP="asmadmin"
# cat 05-udev-pp.rules ACTION=="add", KERNEL=="emcpower*[!0-9]", PROGRAM="/lib/udev/pp_udev.sh %k" [root@zrrsjkdb-01 rules.d]# ls -l /lib/udev/pp_udev.sh -rwxr-xr-x. 1 root root 300 Jan 23 16:13 /lib/udev/pp_udev.sh [root@zrrsjkdb-01 rules.d]# cat /lib/udev/pp_udev.sh #!/bin/sh # for each PowerPath device, turn off the idle slice for # CFQ. # for emcd in `/bin/ls -d /sys/block/$1 2> /dev/null`; do if [ -f $emcd/queue/iosched/slice_idle ]; then echo 0 > $emcd/queue/iosched/slice_idle echo 0 > $emcd/queue/iosched/group_idle fi done exit 0
在‘dmesg’命令的幫助下進行硬件的連接或斷開連接操作時，我們可以看到硬件的檢測或者斷開連接的信息。
‘dmesg’命令在多數基於Linux和Unix的操作系統中都可以使用。
# dmesg 根據數據反映盤讀寫有問題，篩選出盤的關鍵詞 [ 8.371237] scsi host8: qla2xxx [ 8.378555] qla2xxx [0000:21:00.0]-00fb:8: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 8.378565] qla2xxx [0000:21:00.0]-00fc:8: ISP2532: PCIe (5.0GT/s x8) @ 0000:21:00.0 hdma+ host#=8 fw=8.07.00 (90d5). [ 8.379734] qla2xxx [0000:21:00.1]-001a: : MSI-X vector count: 32. [ 8.379737] qla2xxx [0000:21:00.1]-001d: : Found an ISP2532 irq 454 iobase 0xffffaf42d8c01000. [ 8.419238] qla2xxx [0000:05:00.0]-500a:7: LOOP UP detected (8 Gbps). [ 8.801949] scsi host9: qla2xxx [ 8.808485] qla2xxx [0000:21:00.1]-00fb:9: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 8.808497] qla2xxx [0000:21:00.1]-00fc:9: ISP2532: PCIe (5.0GT/s x8) @ 0000:21:00.1 hdma+ host#=9 fw=8.07.00 (90d5). [ 8.809043] qla2xxx [0000:22:00.0]-001a: : MSI-X vector count: 32. [ 8.809058] qla2xxx [0000:22:00.0]-001d: : Found an ISP2532 irq 35 iobase 0xffffaf42d8c11000. [ 8.941119] scsi: waiting for bus probes to complete ... [ 9.225377] scsi host10: qla2xxx [ 9.231401] qla2xxx [0000:22:00.0]-00fb:10: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 9.231410] qla2xxx [0000:22:00.0]-00fc:10: ISP2532: PCIe (5.0GT/s x8) @ 0000:22:00.0 hdma+ host#=10 fw=8.07.00 (90d5). [ 9.231846] qla2xxx [0000:22:00.1]-001a: : MSI-X vector count: 32. [ 9.231850] qla2xxx [0000:22:00.1]-001d: : Found an ISP2532 irq 519 iobase 0xffffaf42d8c21000. [ 9.647372] scsi host11: qla2xxx [ 9.652475] qla2xxx [0000:22:00.1]-00fb:11: QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA. [ 9.652483] qla2xxx [0000:22:00.1]-00fc:11: ISP2532: PCIe (5.0GT/s x8) @ 0000:22:00.1 hdma+ host#=11 fw=8.07.00 (90d5). [ 9.703717] qla2xxx [0000:22:00.0]-500a:10: LOOP UP detected (8 Gbps). [ 13.812227] scsi: waiting for bus probes to complete ... [ 13.813167] scsi 7:0:0:0: Direct-Access EMC LUNZ 6010 PQ: 0 ANSI: 4 [ 13.814564] scsi: waiting for bus probes to complete ... [ 13.814915] sd 7:0:0:0: [sdb] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 13.814919] sd 7:0:0:0: [sdb] Sense Key : Illegal Request [current] [ 13.814923] sd 7:0:0:0: [sdb] Add. Sense: Logical unit not supported [ 13.815498] sd 7:0:0:0: [sdb] 0 512-byte logical blocks: (0 B/0 B) [ 13.815501] sd 7:0:0:0: [sdb] 0-byte physical blocks [ 13.815917] sd 7:0:0:0: [sdb] Test WP failed, assume Write Enabled [ 13.816181] sd 7:0:0:0: [sdb] Asking for cache data failed [ 13.816188] sd 7:0:0:0: [sdb] Assuming drive cache: write through [ 13.817603] sd 7:0:0:0: [sdb] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 13.817606] sd 7:0:0:0: [sdb] Sense Key : Illegal Request [current] [ 13.817608] sd 7:0:0:0: [sdb] Add. Sense: Logical unit not supported [ 13.819321] sd 7:0:0:0: [sdb] Attached SCSI disk [ 14.657189] scsi 7:0:1:0: Direct-Access EMC LUNZ 6010 PQ: 0 ANSI: 4 [ 14.657719] scsi 10:0:0:0: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.658204] sd 7:0:1:0: [sdc] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 14.658208] sd 7:0:1:0: [sdc] Sense Key : Illegal Request [current] [ 14.658210] sd 7:0:1:0: [sdc] Add. Sense: Logical unit not supported [ 14.658463] sd 7:0:1:0: [sdc] 0 512-byte logical blocks: (0 B/0 B) [ 14.658469] sd 7:0:1:0: [sdc] 0-byte physical blocks [ 14.658732] sd 7:0:1:0: [sdc] Test WP failed, assume Write Enabled [ 14.658815] sd 7:0:1:0: [sdc] Asking for cache data failed [ 14.658820] sd 7:0:1:0: [sdc] Assuming drive cache: write through [ 14.659403] scsi 7:0:2:0: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.660479] sd 7:0:1:0: [sdc] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 14.660483] sd 7:0:1:0: [sdc] Sense Key : Illegal Request [current] [ 14.660485] sd 7:0:1:0: [sdc] Add. Sense: Logical unit not supported [ 14.661238] sd 10:0:0:0: [sdd] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.661336] scsi 10:0:0:1: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.661735] sd 7:0:2:0: [sde] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.661846] scsi 7:0:2:1: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.662153] sd 7:0:1:0: [sdc] Attached SCSI disk [ 14.662533] sd 10:0:0:0: [sdd] Write Protect is off [ 14.662537] sd 10:0:0:0: [sdd] Mode Sense: 9f 00 00 00 [ 14.663081] scsi 7:0:2:2: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.663123] sd 7:0:2:1: [sdg] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.663619] sd 7:0:2:0: [sde] Write Protect is off [ 14.663622] sd 7:0:2:0: [sde] Mode Sense: 9f 00 00 00 [ 14.664161] sd 7:0:2:2: [sdh] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.664209] scsi 7:0:2:3: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.665908] scsi 7:0:2:4: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.666299] sd 7:0:2:3: [sdi] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.666521] scsi 10:0:0:2: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.667347] sd 10:0:0:1: [sdf] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.667384] sd 7:0:2:1: [sdg] Write Protect is off [ 14.667387] sd 7:0:2:1: [sdg] Mode Sense: 9f 00 00 00 [ 14.667416] sd 7:0:2:2: [sdh] Write Protect is off [ 14.667418] sd 7:0:2:2: [sdh] Mode Sense: 9f 00 00 00 [ 14.667640] sd 7:0:2:3: [sdi] Write Protect is off [ 14.667642] sd 7:0:2:3: [sdi] Mode Sense: 9f 00 00 00 [ 14.668538] scsi 7:0:2:5: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.668695] sd 10:0:0:1: [sdf] Write Protect is off [ 14.668698] sd 10:0:0:1: [sdf] Mode Sense: 9f 00 00 00 [ 14.669357] sd 10:0:0:0: [sdd] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.669415] sd 7:0:2:4: [sdj] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.669763] scsi 10:0:0:3: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.670280] sd 10:0:0:2: [sdk] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.670523] sd 7:0:2:2: [sdh] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.670540] sd 7:0:2:3: [sdi] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.670666] sd 7:0:2:0: [sde] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.670751] sd 7:0:2:4: [sdj] Write Protect is off [ 14.670754] sd 7:0:2:4: [sdj] Mode Sense: 9f 00 00 00 [ 14.671785] sd 10:0:0:2: [sdk] Write Protect is off [ 14.671788] sd 10:0:0:2: [sdk] Mode Sense: 9f 00 00 00 [ 14.671795] scsi 7:0:2:6: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.672359] sd 7:0:2:1: [sdg] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.672589] scsi 10:0:0:4: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.672614] sd 7:0:2:5: [sdl] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.673341] sd 10:0:0:3: [sdm] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.673486] sd 10:0:0:1: [sdf] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.673776] sd 7:0:2:4: [sdj] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.673824] sd 7:0:2:5: [sdl] Write Protect is off [ 14.673826] sd 7:0:2:5: [sdl] Mode Sense: 9f 00 00 00 [ 14.674257] sd 10:0:0:2: [sdk] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.674291] scsi 7:0:2:7: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.674525] scsi 10:0:0:5: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.674645] sd 10:0:0:4: [sdo] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.674678] sd 7:0:2:6: [sdn] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.675552] sd 10:0:0:3: [sdm] Write Protect is off [ 14.675556] sd 10:0:0:3: [sdm] Mode Sense: 9f 00 00 00 [ 14.675802] sd 10:0:0:4: [sdo] Write Protect is off [ 14.675805] sd 10:0:0:4: [sdo] Mode Sense: 9f 00 00 00 [ 14.675905] sd 10:0:0:5: [sdp] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.675906] scsi 10:0:0:6: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.675949] sd 7:0:2:6: [sdn] Write Protect is off [ 14.675952] sd 7:0:2:6: [sdn] Mode Sense: 9f 00 00 00 [ 14.676206] sd 7:0:2:5: [sdl] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.676387] scsi 7:0:2:8: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.677127] sd 10:0:0:6: [sdr] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.677301] sd 10:0:0:5: [sdp] Write Protect is off [ 14.677304] sd 10:0:0:5: [sdp] Mode Sense: 9f 00 00 00 [ 14.677338] sd 7:0:2:7: [sdq] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.677383] scsi 10:0:0:7: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.678344] sd 10:0:0:6: [sdr] Write Protect is off [ 14.678348] sd 10:0:0:6: [sdr] Mode Sense: 9f 00 00 00 [ 14.678564] sd 7:0:2:7: [sdq] Write Protect is off [ 14.678566] sd 7:0:2:7: [sdq] Mode Sense: 9f 00 00 00 [ 14.678685] scsi 10:0:0:8: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.678942] scsi 7:0:2:9: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.679001] sd 10:0:0:7: [sdt] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.679470] sd 7:0:2:8: [sds] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.679510] sd 7:0:2:6: [sdn] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.680125] sd 10:0:0:8: [sdu] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.680800] sd 10:0:0:3: [sdm] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.680814] scsi 10:0:0:9: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.680839] sd 10:0:0:4: [sdo] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.680854] sd 7:0:2:8: [sds] Write Protect is off [ 14.680858] sd 7:0:2:8: [sds] Mode Sense: 9f 00 00 00 [ 14.681127] sd 10:0:0:7: [sdt] Write Protect is off [ 14.681144] sd 10:0:0:7: [sdt] Mode Sense: 9f 00 00 00 [ 14.681425] sd 10:0:0:5: [sdp] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.681983] sd 10:0:0:8: [sdu] Write Protect is off [ 14.681999] sd 10:0:0:8: [sdu] Mode Sense: 9f 00 00 00 [ 14.682379] sd 10:0:0:6: [sdr] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.682396] sd 7:0:2:7: [sdq] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.682662] scsi 7:0:2:10: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.683358] sd 7:0:2:9: [sdv] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.684659] sd 7:0:2:9: [sdv] Write Protect is off [ 14.684663] sd 7:0:2:9: [sdv] Mode Sense: 9f 00 00 00 [ 14.685073] sd 10:0:0:7: [sdt] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.685801] sd 7:0:2:8: [sds] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.686711] sd 10:0:0:9: [sdw] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.686935] scsi 10:0:0:10: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.686986] sd 10:0:0:8: [sdu] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.687252] scsi 7:0:2:11: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.687447] sd 7:0:2:10: [sdx] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.687530] sd 7:0:2:9: [sdv] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.688729] sd 7:0:2:10: [sdx] Write Protect is off [ 14.688732] sd 7:0:2:10: [sdx] Mode Sense: 9f 00 00 00 [ 14.688736] sd 10:0:0:9: [sdw] Write Protect is off [ 14.688740] sd 10:0:0:9: [sdw] Mode Sense: 9f 00 00 00 [ 14.688773] sd 7:0:2:0: [sde] Attached SCSI disk [ 14.689928] sd 10:0:0:0: [sdd] Attached SCSI disk [ 14.690095] sd 7:0:2:11: [sdy] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.690207] scsi 7:0:2:12: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.690691] random: crng init done [ 14.690725] scsi 10:0:0:11: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.690750] sd 10:0:0:10: [sdz] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.690826] sd 7:0:2:3: [sdi] Attached SCSI disk [ 14.691168] sd 7:0:2:10: [sdx] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.691375] sd 7:0:2:11: [sdy] Write Protect is off [ 14.691384] sd 7:0:2:11: [sdy] Mode Sense: 9f 00 00 00 [ 14.691596] sd 7:0:2:2: [sdh] Attached SCSI disk [ 14.691603] sd 7:0:2:4: [sdj] Attached SCSI disk [ 14.692278] sd 10:0:0:10: [sdz] Write Protect is off [ 14.692288] sd 10:0:0:10: [sdz] Mode Sense: 9f 00 00 00 [ 14.692333] sd 10:0:0:9: [sdw] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.692924] sd 7:0:2:12: [sdaa] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.692925] scsi 7:0:2:13: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.693352] sd 7:0:2:5: [sdl] Attached SCSI disk [ 14.693801] sd 7:0:2:1: [sdg] Attached SCSI disk [ 14.694128] sd 7:0:2:12: [sdaa] Write Protect is off [ 14.694131] sd 7:0:2:12: [sdaa] Mode Sense: 9f 00 00 00 [ 14.694487] sd 10:0:0:1: [sdf] Attached SCSI disk [ 14.694497] sd 10:0:0:2: [sdk] Attached SCSI disk [ 14.695330] sd 7:0:2:13: [sdab] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.695331] scsi 7:0:2:14: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.695676] sd 7:0:2:11: [sdy] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.696711] sd 7:0:2:13: [sdab] Write Protect is off [ 14.696714] sd 7:0:2:13: [sdab] Mode Sense: 9f 00 00 00 [ 14.696901] sd 7:0:2:12: [sdaa] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.697413] sd 7:0:2:6: [sdn] Attached SCSI disk [ 14.697786] sd 7:0:2:14: [sdad] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.697949] scsi 7:0:2:15: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.698935] sd 7:0:2:14: [sdad] Write Protect is off [ 14.698938] sd 7:0:2:14: [sdad] Mode Sense: 9f 00 00 00 [ 14.698975] sd 7:0:2:13: [sdab] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.699158] sd 7:0:2:15: [sdae] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.699702] scsi 7:0:2:16: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.700305] sd 7:0:2:15: [sdae] Write Protect is off [ 14.700310] sd 7:0:2:15: [sdae] Mode Sense: 9f 00 00 00 [ 14.701241] sd 7:0:2:14: [sdad] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.701728] sd 7:0:2:16: [sdaf] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.702399] scsi 7:0:2:17: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.702407] sd 10:0:0:10: [sdz] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.702747] sd 7:0:2:15: [sdae] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.702871] sd 7:0:2:16: [sdaf] Write Protect is off [ 14.702874] sd 7:0:2:16: [sdaf] Mode Sense: 9f 00 00 00 [ 14.703543] sd 7:0:2:17: [sdag] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.704279] scsi 7:0:2:18: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.704691] sd 7:0:2:17: [sdag] Write Protect is off [ 14.704694] sd 7:0:2:17: [sdag] Mode Sense: 9f 00 00 00 [ 14.705199] sd 7:0:2:16: [sdaf] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.705768] sd 7:0:2:18: [sdah] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.706019] scsi 7:0:2:19: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.706915] sd 7:0:2:18: [sdah] Write Protect is off [ 14.706918] sd 7:0:2:18: [sdah] Mode Sense: 9f 00 00 00 [ 14.706966] sd 7:0:2:17: [sdag] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.707410] sd 7:0:2:19: [sdai] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.707504] scsi 7:0:2:20: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.708535] sd 7:0:2:19: [sdai] Write Protect is off [ 14.708538] sd 7:0:2:19: [sdai] Mode Sense: 9f 00 00 00 [ 14.708622] sd 7:0:2:20: [sdaj] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.708783] sd 7:0:2:13: [sdab] Attached SCSI disk [ 14.708832] scsi 7:0:2:21: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.709188] sd 7:0:2:18: [sdah] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.709754] sd 7:0:2:20: [sdaj] Write Protect is off [ 14.709757] sd 7:0:2:20: [sdaj] Mode Sense: 9f 00 00 00 [ 14.710640] sd 7:0:2:14: [sdad] Attached SCSI disk [ 14.710876] sd 7:0:2:19: [sdai] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.711096] sd 7:0:2:21: [sdak] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.711563] scsi 7:0:2:22: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.712060] sd 7:0:2:20: [sdaj] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.712238] sd 7:0:2:21: [sdak] Write Protect is off [ 14.712240] sd 7:0:2:21: [sdak] Mode Sense: 9f 00 00 00 [ 14.712655] sd 7:0:2:15: [sdae] Attached SCSI disk [ 14.713519] sd 7:0:2:22: [sdal] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.714008] scsi 7:0:2:23: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.714554] sd 7:0:2:21: [sdak] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.714740] sd 7:0:2:22: [sdal] Write Protect is off [ 14.714742] sd 7:0:2:22: [sdal] Mode Sense: 9f 00 00 00 [ 14.714793] sd 7:0:2:16: [sdaf] Attached SCSI disk [ 14.715492] scsi 10:0:0:12: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.715779] sd 7:0:2:23: [sdam] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.715939] scsi 7:0:2:24: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.716983] sd 7:0:2:23: [sdam] Write Protect is off [ 14.717036] sd 7:0:2:23: [sdam] Mode Sense: 9f 00 00 00 [ 14.717046] sd 7:0:2:22: [sdal] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.717202] sd 7:0:2:17: [sdag] Attached SCSI disk [ 14.717284] sd 7:0:2:24: [sdan] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.717577] scsi 7:0:3:0: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.718563] sd 7:0:2:24: [sdan] Write Protect is off [ 14.718565] sd 7:0:2:24: [sdan] Mode Sense: 9f 00 00 00 [ 14.719336] sd 7:0:2:7: [sdq] Attached SCSI disk [ 14.719444] sd 7:0:2:23: [sdam] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.719571] sd 7:0:2:18: [sdah] Attached SCSI disk [ 14.719739] sd 7:0:3:0: [sdao] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.719902] scsi 7:0:3:1: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.721167] sd 7:0:3:0: [sdao] Write Protect is off [ 14.721194] sd 7:0:3:0: [sdao] Mode Sense: 9f 00 00 00 [ 14.721238] sd 7:0:2:24: [sdan] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.721354] sd 7:0:2:19: [sdai] Attached SCSI disk [ 14.722737] scsi 7:0:3:2: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.723076] sd 7:0:2:20: [sdaj] Attached SCSI disk [ 14.723125] sd 7:0:3:1: [sdap] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.723833] sd 7:0:3:0: [sdao] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.724415] sd 7:0:3:1: [sdap] Write Protect is off [ 14.724417] sd 7:0:3:1: [sdap] Mode Sense: 9f 00 00 00 [ 14.724934] sd 10:0:0:11: [sdac] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.724949] sd 10:0:0:4: [sdo] Attached SCSI disk [ 14.725046] sd 7:0:2:21: [sdak] Attached SCSI disk [ 14.725901] scsi 7:0:3:3: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.726587] sd 7:0:2:8: [sds] Attached SCSI disk [ 14.726718] sd 7:0:3:2: [sdaq] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.727054] sd 7:0:3:1: [sdap] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.727187] sd 10:0:0:11: [sdac] Write Protect is off [ 14.727192] sd 10:0:0:11: [sdac] Mode Sense: 9f 00 00 00 [ 14.727663] sd 10:0:0:12: [sdar] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.727983] scsi 10:0:0:13: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.728035] sd 7:0:3:2: [sdaq] Write Protect is off [ 14.728037] sd 7:0:3:2: [sdaq] Mode Sense: 9f 00 00 00 [ 14.728817] sd 10:0:0:5: [sdp] Attached SCSI disk [ 14.728829] sd 10:0:0:12: [sdar] Write Protect is off [ 14.728831] sd 10:0:0:12: [sdar] Mode Sense: 9f 00 00 00 [ 14.728838] sd 10:0:0:3: [sdm] Attached SCSI disk [ 14.728953] sd 10:0:0:6: [sdr] Attached SCSI disk [ 14.729162] sd 10:0:0:7: [sdt] Attached SCSI disk [ 14.729295] sd 10:0:0:13: [sdat] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.729315] scsi 7:0:3:4: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.729387] scsi 10:0:0:14: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.729538] sd 7:0:2:22: [sdal] Attached SCSI disk [ 14.729809] sd 7:0:3:3: [sdas] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.729928] sd 10:0:0:11: [sdac] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.730348] sd 7:0:2:9: [sdv] Attached SCSI disk [ 14.730557] sd 10:0:0:13: [sdat] Write Protect is off [ 14.730560] sd 10:0:0:13: [sdat] Mode Sense: 9f 00 00 00 [ 14.730669] sd 7:0:3:2: [sdaq] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.730670] sd 10:0:0:14: [sdau] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.730849] scsi 10:0:0:15: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.730950] sd 10:0:0:8: [sdu] Attached SCSI disk [ 14.731034] sd 7:0:3:3: [sdas] Write Protect is off [ 14.731036] sd 7:0:3:3: [sdas] Mode Sense: 9f 00 00 00 [ 14.731891] sd 10:0:0:14: [sdau] Write Protect is off [ 14.731951] sd 10:0:0:14: [sdau] Mode Sense: 9f 00 00 00 [ 14.732127] sd 7:0:3:4: [sdav] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.732155] sd 10:0:0:12: [sdar] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.732231] sd 7:0:2:23: [sdam] Attached SCSI disk [ 14.732366] scsi 7:0:3:5: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.732874] sd 10:0:0:13: [sdat] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.733235] sd 10:0:0:15: [sdaw] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.733459] sd 7:0:3:4: [sdav] Write Protect is off [ 14.733492] sd 7:0:3:4: [sdav] Mode Sense: 9f 00 00 00 [ 14.733676] scsi 10:0:0:16: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.733732] sd 7:0:3:3: [sdas] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.733841] sd 7:0:2:10: [sdx] Attached SCSI disk [ 14.733910] sd 7:0:2:24: [sdan] Attached SCSI disk [ 14.734418] sd 10:0:0:15: [sdaw] Write Protect is off [ 14.734421] sd 10:0:0:15: [sdaw] Mode Sense: 9f 00 00 00 [ 14.734441] sd 10:0:0:14: [sdau] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.734504] sd 7:0:3:5: [sdax] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.734507] sd 7:0:2:12: [sdaa] Attached SCSI disk [ 14.734761] scsi 7:0:3:6: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.735250] sd 10:0:0:16: [sday] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.735590] sd 10:0:0:9: [sdw] Attached SCSI disk [ 14.735591] scsi 10:0:0:17: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.735718] sd 7:0:3:5: [sdax] Write Protect is off [ 14.735721] sd 7:0:3:5: [sdax] Mode Sense: 9f 00 00 00 [ 14.735798] sd 7:0:3:4: [sdav] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.736044] sd 7:0:2:11: [sdy] Attached SCSI disk [ 14.736398] sd 7:0:3:6: [sdaz] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.736545] sd 10:0:0:16: [sday] Write Protect is off [ 14.736548] sd 10:0:0:16: [sday] Mode Sense: 9f 00 00 00 [ 14.736913] sd 10:0:0:15: [sdaw] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.737055] scsi 7:0:3:7: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.737623] sd 7:0:3:6: [sdaz] Write Protect is off [ 14.737625] sd 7:0:3:6: [sdaz] Mode Sense: 9f 00 00 00 [ 14.737766] sd 7:0:3:0: [sdao] Attached SCSI disk [ 14.738034] sd 7:0:3:5: [sdax] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.738878] sd 7:0:3:1: [sdap] Attached SCSI disk [ 14.739847] sd 7:0:3:6: [sdaz] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.740562] sd 7:0:3:7: [sdba] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.741718] sd 7:0:3:7: [sdba] Write Protect is off [ 14.741722] sd 7:0:3:7: [sdba] Mode Sense: 9f 00 00 00 [ 14.741878] scsi 7:0:3:8: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.742073] sd 7:0:3:2: [sdaq] Attached SCSI disk [ 14.744096] sd 7:0:3:7: [sdba] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.744924] sd 7:0:3:8: [sdbb] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.745393] scsi 7:0:3:9: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.746054] sd 7:0:3:8: [sdbb] Write Protect is off [ 14.746057] sd 7:0:3:8: [sdbb] Mode Sense: 9f 00 00 00 [ 14.746131] sd 7:0:3:3: [sdas] Attached SCSI disk [ 14.747105] sd 7:0:3:9: [sdbc] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.747164] sd 10:0:0:10: [sdz] Attached SCSI disk [ 14.747267] sd 7:0:3:4: [sdav] Attached SCSI disk [ 14.747497] sd 10:0:0:16: [sday] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.747992] scsi 7:0:3:10: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.748246] sd 7:0:3:9: [sdbc] Write Protect is off [ 14.748249] sd 7:0:3:9: [sdbc] Mode Sense: 9f 00 00 00 [ 14.748324] sd 7:0:3:8: [sdbb] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.749831] sd 10:0:0:17: [sdbd] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.750179] sd 7:0:3:5: [sdax] Attached SCSI disk [ 14.750577] sd 7:0:3:9: [sdbc] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.751026] sd 10:0:0:17: [sdbd] Write Protect is off [ 14.751029] scsi 10:0:0:18: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.751032] sd 10:0:0:17: [sdbd] Mode Sense: 9f 00 00 00 [ 14.753318] sd 10:0:0:17: [sdbd] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.753483] sd 7:0:3:6: [sdaz] Attached SCSI disk [ 14.754796] sd 7:0:3:10: [sdbe] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.755664] sd 10:0:0:14: [sdau] Attached SCSI disk [ 14.756045] sd 7:0:3:10: [sdbe] Write Protect is off [ 14.756048] sd 7:0:3:10: [sdbe] Mode Sense: 9f 00 00 00 [ 14.756121] sd 10:0:0:12: [sdar] Attached SCSI disk [ 14.756450] sd 10:0:0:13: [sdat] Attached SCSI disk [ 14.756695] sd 10:0:0:11: [sdac] Attached SCSI disk [ 14.757058] scsi 7:0:3:11: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.757344] sd 10:0:0:18: [sdbf] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.758549] sd 7:0:3:10: [sdbe] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.758568] sd 10:0:0:18: [sdbf] Write Protect is off [ 14.758571] sd 10:0:0:18: [sdbf] Mode Sense: 9f 00 00 00 [ 14.758596] scsi 10:0:0:19: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.759715] scsi 7:0:3:12: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.759951] sd 7:0:3:11: [sdbg] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.759955] sd 7:0:3:7: [sdba] Attached SCSI disk [ 14.760046] scsi 10:0:0:20: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.760100] sd 10:0:0:19: [sdbh] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.760956] sd 10:0:0:18: [sdbf] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.761136] sd 10:0:0:15: [sdaw] Attached SCSI disk [ 14.761206] sd 10:0:0:19: [sdbh] Write Protect is off [ 14.761209] sd 10:0:0:19: [sdbh] Mode Sense: 9f 00 00 00 [ 14.761218] sd 7:0:3:11: [sdbg] Write Protect is off [ 14.761222] sd 7:0:3:11: [sdbg] Mode Sense: 9f 00 00 00 [ 14.761341] sd 10:0:0:16: [sday] Attached SCSI disk [ 14.762057] sd 10:0:0:20: [sdbi] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.763190] sd 7:0:3:12: [sdbj] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.763210] sd 7:0:3:8: [sdbb] Attached SCSI disk [ 14.763425] sd 10:0:0:20: [sdbi] Write Protect is off [ 14.763436] sd 10:0:0:20: [sdbi] Mode Sense: 9f 00 00 00 [ 14.763745] sd 10:0:0:19: [sdbh] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.763766] scsi 10:0:0:21: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.763796] sd 7:0:3:11: [sdbg] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.764056] scsi 7:0:3:13: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.764464] sd 7:0:3:12: [sdbj] Write Protect is off [ 14.764467] sd 7:0:3:12: [sdbj] Mode Sense: 9f 00 00 00 [ 14.765882] sd 10:0:0:20: [sdbi] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.766236] sd 10:0:0:17: [sdbd] Attached SCSI disk [ 14.767038] sd 7:0:3:12: [sdbj] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.767434] sd 7:0:3:9: [sdbc] Attached SCSI disk [ 14.769966] sd 10:0:0:21: [sdbk] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.771087] sd 7:0:3:13: [sdbl] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.771325] sd 10:0:0:21: [sdbk] Write Protect is off [ 14.771329] sd 10:0:0:21: [sdbk] Mode Sense: 9f 00 00 00 [ 14.771839] scsi 10:0:0:22: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.772212] sd 7:0:3:13: [sdbl] Write Protect is off [ 14.772215] sd 7:0:3:13: [sdbl] Mode Sense: 9f 00 00 00 [ 14.772795] scsi 7:0:3:14: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.773606] sd 10:0:0:21: [sdbk] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.774739] sd 7:0:3:10: [sdbe] Attached SCSI disk [ 14.774933] sd 7:0:3:13: [sdbl] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.775012] sd 10:0:0:22: [sdbm] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.775560] scsi 10:0:0:23: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.775610] sd 10:0:0:18: [sdbf] Attached SCSI disk [ 14.776193] sd 10:0:0:22: [sdbm] Write Protect is off [ 14.776196] sd 10:0:0:22: [sdbm] Mode Sense: 9f 00 00 00 [ 14.776680] scsi 7:0:3:15: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.776776] sd 7:0:3:14: [sdbn] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.778001] sd 7:0:3:14: [sdbn] Write Protect is off [ 14.778025] sd 7:0:3:14: [sdbn] Mode Sense: 9f 00 00 00 [ 14.779928] scsi 10:0:0:24: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.779952] sd 10:0:0:19: [sdbh] Attached SCSI disk [ 14.780045] sd 7:0:3:11: [sdbg] Attached SCSI disk [ 14.780079] sd 7:0:3:15: [sdbp] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.780360] sd 7:0:3:14: [sdbn] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.780649] sd 10:0:0:22: [sdbm] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.780927] scsi 7:0:3:16: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.781504] sd 7:0:3:15: [sdbp] Write Protect is off [ 14.781507] sd 7:0:3:15: [sdbp] Mode Sense: 9f 00 00 00 [ 14.782759] sd 7:0:3:12: [sdbj] Attached SCSI disk [ 14.783944] sd 10:0:0:20: [sdbi] Attached SCSI disk [ 14.784330] sd 7:0:3:15: [sdbp] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.787072] sd 10:0:0:23: [sdbo] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.788101] sd 7:0:3:16: [sdbq] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.789326] sd 10:0:0:23: [sdbo] Write Protect is off [ 14.789329] sd 7:0:3:16: [sdbq] Write Protect is off [ 14.789331] sd 10:0:0:23: [sdbo] Mode Sense: 9f 00 00 00 [ 14.789336] sd 7:0:3:16: [sdbq] Mode Sense: 9f 00 00 00 [ 14.789378] scsi 7:0:3:17: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.790967] sd 7:0:3:13: [sdbl] Attached SCSI disk [ 14.791724] sd 7:0:3:16: [sdbq] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.792027] sd 10:0:0:23: [sdbo] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.793962] sd 10:0:0:24: [sdbr] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.794011] scsi 10:0:1:0: Direct-Access EMC LUNZ 6010 PQ: 0 ANSI: 4 [ 14.794045] sd 7:0:3:17: [sdbs] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.794853] scsi 7:0:3:18: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.795082] sd 10:0:0:24: [sdbr] Write Protect is off [ 14.795084] sd 10:0:0:24: [sdbr] Mode Sense: 9f 00 00 00 [ 14.795190] sd 7:0:3:17: [sdbs] Write Protect is off [ 14.795193] sd 7:0:3:17: [sdbs] Mode Sense: 9f 00 00 00 [ 14.795681] sd 7:0:3:14: [sdbn] Attached SCSI disk [ 14.797599] sd 10:0:0:24: [sdbr] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.797616] sd 7:0:3:17: [sdbs] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.798114] sd 10:0:0:21: [sdbk] Attached SCSI disk [ 14.802061] sd 7:0:3:18: [sdbt] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.802719] sd 10:0:0:22: [sdbm] Attached SCSI disk [ 14.803052] scsi 7:0:3:19: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.803339] sd 7:0:3:18: [sdbt] Write Protect is off [ 14.803342] sd 7:0:3:18: [sdbt] Mode Sense: 9f 00 00 00 [ 14.803527] sd 10:0:1:0: [sdbu] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 14.803532] sd 10:0:1:0: [sdbu] Sense Key : Illegal Request [current] [ 14.803535] sd 10:0:1:0: [sdbu] Add. Sense: Logical unit not supported [ 14.804485] sd 10:0:1:0: [sdbu] 0 512-byte logical blocks: (0 B/0 B) [ 14.804488] sd 10:0:1:0: [sdbu] 0-byte physical blocks [ 14.804787] scsi 10:0:2:0: Direct-Access EMC LUNZ 6010 PQ: 0 ANSI: 4 [ 14.805085] sd 7:0:3:15: [sdbp] Attached SCSI disk [ 14.805122] sd 10:0:1:0: [sdbu] Test WP failed, assume Write Enabled [ 14.805409] sd 10:0:1:0: [sdbu] Asking for cache data failed [ 14.805424] sd 10:0:1:0: [sdbu] Assuming drive cache: write through [ 14.805881] sd 7:0:3:18: [sdbt] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.807711] sd 7:0:3:19: [sdbv] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.807734] scsi 7:0:3:20: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.808435] sd 10:0:1:0: [sdbu] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 14.808438] sd 10:0:1:0: [sdbu] Sense Key : Illegal Request [current] [ 14.808441] sd 10:0:1:0: [sdbu] Add. Sense: Logical unit not supported [ 14.808864] sd 10:0:0:23: [sdbo] Attached SCSI disk [ 14.808959] sd 7:0:3:19: [sdbv] Write Protect is off [ 14.808962] sd 7:0:3:16: [sdbq] Attached SCSI disk [ 14.808963] sd 7:0:3:19: [sdbv] Mode Sense: 9f 00 00 00 [ 14.808973] sd 10:0:2:0: [sdbw] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 14.808976] sd 10:0:2:0: [sdbw] Sense Key : Illegal Request [current] [ 14.808979] sd 10:0:2:0: [sdbw] Add. Sense: Logical unit not supported [ 14.810044] sd 10:0:2:0: [sdbw] 0 512-byte logical blocks: (0 B/0 B) [ 14.810046] sd 10:0:2:0: [sdbw] 0-byte physical blocks [ 14.810854] sd 10:0:1:0: [sdbu] Attached SCSI disk [ 14.811034] sd 10:0:2:0: [sdbw] Test WP failed, assume Write Enabled [ 14.811353] sd 10:0:2:0: [sdbw] Asking for cache data failed [ 14.811357] sd 10:0:2:0: [sdbw] Assuming drive cache: write through [ 14.811369] sd 7:0:3:19: [sdbv] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.811487] sd 7:0:3:20: [sdbx] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.812319] scsi 7:0:3:21: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.812599] sd 10:0:0:24: [sdbr] Attached SCSI disk [ 14.812631] sd 7:0:3:20: [sdbx] Write Protect is off [ 14.812634] sd 7:0:3:20: [sdbx] Mode Sense: 9f 00 00 00 [ 14.813298] sd 7:0:3:17: [sdbs] Attached SCSI disk [ 14.814946] sd 7:0:3:20: [sdbx] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.816357] scsi 10:0:3:0: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.816457] sd 10:0:2:0: [sdbw] Read Capacity(10) failed: Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 14.816460] sd 10:0:2:0: [sdbw] Sense Key : Illegal Request [current] [ 14.816462] sd 10:0:2:0: [sdbw] Add. Sense: Logical unit not supported [ 14.817053] sd 7:0:3:21: [sdby] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.817489] sd 10:0:2:0: [sdbw] Attached SCSI disk [ 14.818200] sd 7:0:3:21: [sdby] Write Protect is off [ 14.818203] sd 7:0:3:21: [sdby] Mode Sense: 9f 00 00 00 [ 14.819520] sd 7:0:3:18: [sdbt] Attached SCSI disk [ 14.819832] scsi 7:0:3:22: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.820787] sd 7:0:3:21: [sdby] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.822568] sd 10:0:3:0: [sdbz] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.823724] sd 10:0:3:0: [sdbz] Write Protect is off [ 14.823727] sd 10:0:3:0: [sdbz] Mode Sense: 9f 00 00 00 [ 14.823907] scsi 10:0:3:1: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.824940] sd 7:0:3:22: [sdca] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.825239] sd 7:0:3:19: [sdbv] Attached SCSI disk [ 14.825320] scsi 7:0:3:23: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.826139] sd 7:0:3:22: [sdca] Write Protect is off [ 14.826142] sd 7:0:3:22: [sdca] Mode Sense: 9f 00 00 00 [ 14.826438] sd 10:0:3:0: [sdbz] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.828038] sd 10:0:3:1: [sdcb] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.828779] sd 7:0:3:22: [sdca] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.829214] sd 10:0:3:1: [sdcb] Write Protect is off [ 14.829216] sd 10:0:3:1: [sdcb] Mode Sense: 9f 00 00 00 [ 14.829498] scsi 10:0:3:2: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.830737] sd 7:0:3:20: [sdbx] Attached SCSI disk [ 14.831538] sd 10:0:3:1: [sdcb] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.831948] sd 7:0:3:23: [sdcc] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.833187] sd 7:0:3:23: [sdcc] Write Protect is off [ 14.833191] sd 7:0:3:23: [sdcc] Mode Sense: 9f 00 00 00 [ 14.833482] scsi 7:0:3:24: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.834385] sd 10:0:3:2: [sdcd] 10487040 512-byte logical blocks: (5.37 GB/5.00 GiB) [ 14.835663] sd 10:0:3:2: [sdcd] Write Protect is off [ 14.835665] sd 10:0:3:2: [sdcd] Mode Sense: 9f 00 00 00 [ 14.835740] sd 7:0:3:23: [sdcc] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.835888] scsi 10:0:3:3: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.836793] sd 7:0:3:21: [sdby] Attached SCSI disk [ 14.837960] sd 10:0:3:2: [sdcd] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.839140] sd 7:0:3:24: [sdce] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.840345] sd 7:0:3:24: [sdce] Write Protect is off [ 14.840348] sd 7:0:3:24: [sdce] Mode Sense: 9f 00 00 00 [ 14.840544] sd 10:0:3:3: [sdcf] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.841800] sd 10:0:3:3: [sdcf] Write Protect is off [ 14.841803] sd 10:0:3:3: [sdcf] Mode Sense: 9f 00 00 00 [ 14.841980] sd 10:0:3:0: [sdbz] Attached SCSI disk [ 14.842048] scsi 10:0:3:4: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.842903] sd 7:0:3:24: [sdce] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.844196] sd 10:0:3:3: [sdcf] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.844300] sd 10:0:3:1: [sdcb] Attached SCSI disk [ 14.845602] sd 7:0:3:22: [sdca] Attached SCSI disk [ 14.847353] sd 10:0:3:4: [sdcg] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.847761] scsi 10:0:3:5: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.848622] sd 10:0:3:4: [sdcg] Write Protect is off [ 14.848625] sd 10:0:3:4: [sdcg] Mode Sense: 9f 00 00 00 [ 14.850893] sd 10:0:3:4: [sdcg] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.851059] sd 7:0:3:23: [sdcc] Attached SCSI disk [ 14.852443] sd 10:0:3:5: [sdch] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.852788] sd 10:0:3:2: [sdcd] Attached SCSI disk [ 14.853744] sd 10:0:3:5: [sdch] Write Protect is off [ 14.853746] sd 10:0:3:5: [sdch] Mode Sense: 9f 00 00 00 [ 14.854455] scsi 10:0:3:6: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.856215] sd 10:0:3:5: [sdch] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.861143] sd 10:0:3:6: [sdci] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.861709] sd 10:0:3:3: [sdcf] Attached SCSI disk [ 14.862291] sd 10:0:3:6: [sdci] Write Protect is off [ 14.862295] sd 10:0:3:6: [sdci] Mode Sense: 9f 00 00 00 [ 14.863050] scsi 10:0:3:7: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.863075] sd 7:0:3:24: [sdce] Attached SCSI disk [ 14.864560] sd 10:0:3:6: [sdci] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.867344] sd 10:0:3:4: [sdcg] Attached SCSI disk [ 14.868117] sd 10:0:3:7: [sdcj] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.869284] sd 10:0:3:7: [sdcj] Write Protect is off [ 14.869287] sd 10:0:3:7: [sdcj] Mode Sense: 9f 00 00 00 [ 14.870413] scsi 10:0:3:8: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.871599] sd 10:0:3:7: [sdcj] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.873653] sd 10:0:3:5: [sdch] Attached SCSI disk [ 14.876634] sd 10:0:3:8: [sdck] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.877234] scsi 10:0:3:9: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.877799] sd 10:0:3:8: [sdck] Write Protect is off [ 14.877801] sd 10:0:3:8: [sdck] Mode Sense: 9f 00 00 00 [ 14.880135] sd 10:0:3:8: [sdck] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.880430] sd 10:0:3:6: [sdci] Attached SCSI disk [ 14.881688] sd 10:0:3:9: [sdcl] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.882651] scsi 10:0:3:10: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.882845] sd 10:0:3:9: [sdcl] Write Protect is off [ 14.882848] sd 10:0:3:9: [sdcl] Mode Sense: 9f 00 00 00 [ 14.885291] sd 10:0:3:9: [sdcl] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.897350] sd 10:0:3:7: [sdcj] Attached SCSI disk [ 14.897494] scsi 10:0:3:11: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.897935] sd 10:0:3:10: [sdcm] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.899197] sd 10:0:3:10: [sdcm] Write Protect is off [ 14.899201] sd 10:0:3:10: [sdcm] Mode Sense: 9f 00 00 00 [ 14.901856] sd 10:0:3:10: [sdcm] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.904645] sd 10:0:3:11: [sdcn] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.904872] scsi 10:0:3:12: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.906055] sd 10:0:3:11: [sdcn] Write Protect is off [ 14.906057] sd 10:0:3:11: [sdcn] Mode Sense: 9f 00 00 00 [ 14.908374] sd 10:0:3:11: [sdcn] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.908750] sd 10:0:3:8: [sdck] Attached SCSI disk [ 14.910655] sd 10:0:3:9: [sdcl] Attached SCSI disk [ 14.910800] sd 10:0:3:12: [sdco] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.911972] sd 10:0:3:12: [sdco] Write Protect is off [ 14.911974] sd 10:0:3:12: [sdco] Mode Sense: 9f 00 00 00 [ 14.912138] scsi 10:0:3:13: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.914456] sd 10:0:3:12: [sdco] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.918659] sd 10:0:3:13: [sdcp] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.919887] sd 10:0:3:13: [sdcp] Write Protect is off [ 14.919890] sd 10:0:3:13: [sdcp] Mode Sense: 9f 00 00 00 [ 14.920499] sd 10:0:3:10: [sdcm] Attached SCSI disk [ 14.920718] scsi 10:0:3:14: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.924505] sd 10:0:3:13: [sdcp] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.930477] sd 10:0:3:11: [sdcn] Attached SCSI disk [ 14.936553] sd 10:0:3:14: [sdcq] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.936573] scsi 10:0:3:15: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.937810] sd 10:0:3:14: [sdcq] Write Protect is off [ 14.937813] sd 10:0:3:14: [sdcq] Mode Sense: 9f 00 00 00 [ 14.940335] sd 10:0:3:14: [sdcq] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.941059] sd 10:0:3:12: [sdco] Attached SCSI disk [ 14.945163] sd 10:0:3:15: [sdcr] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.946132] scsi 10:0:3:16: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.946503] sd 10:0:3:15: [sdcr] Write Protect is off [ 14.946505] sd 10:0:3:15: [sdcr] Mode Sense: 9f 00 00 00 [ 14.949327] sd 10:0:3:15: [sdcr] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.950871] sd 10:0:3:13: [sdcp] Attached SCSI disk [ 14.951863] sd 10:0:3:16: [sdcs] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.953365] sd 10:0:3:16: [sdcs] Write Protect is off [ 14.953386] sd 10:0:3:16: [sdcs] Mode Sense: 9f 00 00 00 [ 14.954637] scsi 10:0:3:17: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.956148] sd 10:0:3:16: [sdcs] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.973225] scsi 10:0:3:18: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.979100] sd 10:0:3:14: [sdcq] Attached SCSI disk [ 14.981677] sd 10:0:3:17: [sdct] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.983066] sd 10:0:3:17: [sdct] Write Protect is off [ 14.983069] sd 10:0:3:17: [sdct] Mode Sense: 9f 00 00 00 [ 14.985620] sd 10:0:3:17: [sdct] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.986060] sd 10:0:3:18: [sdcu] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.986176] sd 10:0:3:15: [sdcr] Attached SCSI disk [ 14.986881] scsi 10:0:3:19: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.987197] sd 10:0:3:18: [sdcu] Write Protect is off [ 14.987200] sd 10:0:3:18: [sdcu] Mode Sense: 9f 00 00 00 [ 14.989570] sd 10:0:3:18: [sdcu] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.990257] sd 10:0:3:19: [sdcv] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.990868] scsi 10:0:3:20: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.991451] sd 10:0:3:19: [sdcv] Write Protect is off [ 14.991460] sd 10:0:3:19: [sdcv] Mode Sense: 9f 00 00 00 [ 14.993552] sd 10:0:3:16: [sdcs] Attached SCSI disk [ 14.993975] sd 10:0:3:19: [sdcv] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 14.997341] sd 10:0:3:20: [sdcw] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 14.997678] scsi 10:0:3:21: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 14.998585] sd 10:0:3:20: [sdcw] Write Protect is off [ 14.998588] sd 10:0:3:20: [sdcw] Mode Sense: 9f 00 00 00 [ 15.001163] sd 10:0:3:20: [sdcw] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 15.001587] sd 10:0:3:17: [sdct] Attached SCSI disk [ 15.003116] sd 10:0:3:21: [sdcx] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 15.004241] sd 10:0:3:21: [sdcx] Write Protect is off [ 15.004244] sd 10:0:3:21: [sdcx] Mode Sense: 9f 00 00 00 [ 15.004641] scsi 10:0:3:22: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 15.005562] sd 10:0:3:18: [sdcu] Attached SCSI disk [ 15.006536] sd 10:0:3:21: [sdcx] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 15.012898] sd 10:0:3:19: [sdcv] Attached SCSI disk [ 15.013200] sd 10:0:3:22: [sdcy] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 15.014333] sd 10:0:3:22: [sdcy] Write Protect is off [ 15.014336] sd 10:0:3:22: [sdcy] Mode Sense: 9f 00 00 00 [ 15.014433] scsi 10:0:3:23: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 15.016628] sd 10:0:3:22: [sdcy] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA [ 15.017964] sd 10:0:3:20: [sdcw] Attached SCSI disk [ 15.018313] sd 10:0:3:23: [sdcz] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 15.019457] sd 10:0:3:23: [sdcz] Write Protect is off [ 15.019460] sd 10:0:3:23: [sdcz] Mode Sense: 9f 00 00 00 [ 15.020373] scsi 10:0:3:24: Direct-Access EMC Invista 6010 PQ: 0 ANSI: 4 [ 15.021708] sd 10:0:3:23: [sdcz] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA 寫緩存:禁用，讀緩存:禁用，不支持DPO或FUA [ 15.023875] sd 10:0:3:21: [sdcx] Attached SCSI disk [ 15.025404] sd 10:0:3:24: [sdda] 1048577280 512-byte logical blocks: (537 GB/500 GiB) [ 15.026537] sd 10:0:3:24: [sdda] Write Protect is off [ 15.026540] sd 10:0:3:24: [sdda] Mode Sense: 9f 00 00 00 [ 15.028876] sd 10:0:3:24: [sdda] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA 寫緩存:禁用，讀緩存:禁用，不支持DPO或FUA [ 15.032097] sd 10:0:3:22: [sdcy] Attached SCSI disk [ 15.038745] sd 10:0:3:23: [sdcz] Attached SCSI disk [ 15.049611] sd 10:0:3:24: [sdda] Attached SCSI disk ........
# su - grid Last login: Tue Apr 21 08:51:42 CST 2020 on pts/2 zrrsjkdb-01>asmcmd ASMCMD> lsdsk Path /dev/emcpowera /dev/emcpowerb /dev/emcpowerc /dev/emcpowerd /dev/emcpowere /dev/emcpowerf /dev/emcpowerg /dev/emcpowerh /dev/emcpoweri /dev/emcpowerj /dev/emcpowerk /dev/emcpowerl /dev/emcpowerm /dev/emcpowern /dev/emcpowero /dev/emcpowerp /dev/emcpowerq /dev/emcpowerr /dev/emcpowers /dev/emcpowert /dev/emcpoweru /dev/emcpowerv /dev/emcpowerw /dev/emcpowerx /dev/emcpowery ASMCMD> lsdg State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED EXTERN N 512 4096 1048576 11264000 1546468 0 1546468 0 N DATA/ MOUNTED NORMAL N 512 4096 1048576 15360 14434 5120 4657 0 Y OCR/ ASMCMD> lsdsk -p -G OCR Group_Num Disk_Num Incarn Mount_Stat Header_Stat Mode_Stat State Path 2 0 3911628624 CACHED MEMBER ONLINE NORMAL /dev/emcpowerq 2 1 3911628625 CACHED MEMBER ONLINE NORMAL /dev/emcpowerr 2 2 3911628626 CACHED MEMBER ONLINE NORMAL /dev/emcpowers ASMCMD> exit zrrsjkdb-01>exit logout
# cd /dev # ls -l emcpower* crw-r--r-- 1 root root 10, 57 Apr 20 21:38 emcpower brw-rw---- 1 grid asmadmin 120, 0 Apr 21 09:44 emcpowera brw-rw---- 1 grid asmadmin 120, 16 Apr 21 09:44 emcpowerb brw-rw---- 1 grid asmadmin 120, 32 Apr 21 09:43 emcpowerc brw-rw---- 1 grid asmadmin 120, 48 Apr 21 09:44 emcpowerd brw-rw---- 1 grid asmadmin 120, 64 Apr 21 09:44 emcpowere brw-rw---- 1 grid asmadmin 120, 80 Apr 21 09:44 emcpowerf brw-rw---- 1 grid asmadmin 120, 96 Apr 21 09:44 emcpowerg brw-rw---- 1 grid asmadmin 120, 112 Apr 21 09:44 emcpowerh brw-rw---- 1 grid asmadmin 120, 128 Apr 21 09:44 emcpoweri brw-rw---- 1 grid asmadmin 120, 144 Apr 21 09:44 emcpowerj brw-rw---- 1 grid asmadmin 120, 160 Apr 21 09:43 emcpowerk brw-rw---- 1 grid asmadmin 120, 176 Apr 21 09:42 emcpowerl brw-rw---- 1 grid asmadmin 120, 192 Apr 21 09:44 emcpowerm brw-rw---- 1 grid asmadmin 120, 208 Apr 21 09:43 emcpowern brw-rw---- 1 grid asmadmin 120, 224 Apr 21 09:44 emcpowero brw-rw---- 1 grid asmadmin 120, 240 Apr 21 09:44 emcpowerp brw-rw---- 1 grid asmadmin 120, 256 Apr 21 09:44 emcpowerq brw-rw---- 1 grid asmadmin 120, 272 Apr 21 09:44 emcpowerr brw-rw---- 1 grid asmadmin 120, 288 Apr 21 09:44 emcpowers brw-rw---- 1 grid asmadmin 120, 304 Apr 21 09:40 emcpowert brw-rw---- 1 grid asmadmin 120, 320 Apr 21 09:43 emcpoweru brw-rw---- 1 grid asmadmin 120, 336 Apr 21 09:44 emcpowerv brw-rw---- 1 grid asmadmin 120, 352 Apr 21 09:44 emcpowerw brw-rw---- 1 grid asmadmin 120, 368 Apr 21 09:44 emcpowerx brw-rw---- 1 grid asmadmin 120, 384 Apr 21 09:44 emcpowery
根據盤的讀寫報錯和路徑問題
安裝emc多路徑必須把系統內核多路徑刪除
這裏針對centos7系統爲例：(Disabling dm-multipath on a RHEL 7 or OL 7 host )
1) Blacklist all devices in /etc/multipath.conf
默認情況下，multipath會把所有設備都加入到黑名單（devnode "*"），也就是禁止使用。所以，我們首先需要取消該設置，把配置文件修改爲類似下面的內容：
# cat /etc/multipath.conf
blacklist {
devnode "*"
}2) Restart multipathd service for the multipath config file to get into effect:
# systemctl restart multipathd
3）Stop dm-multipath service and ensure dm-multipath does not start automatically on boot.
# systemctl stop multipathd
關閉開機自啓動
# systemctl disable multipathd Removed symlink /etc/systemd/system/sysinit.target.wants/multipathd.service.
4)重新創建ramdisk映像，以便從ramdisk映像中排除dm-multipath。
Recreate ramdisk image so that dm-multipath is excluded from ramdisk image.
# dracut /boot/initramfs-wo-DM-$(uname -r).img $(uname -r)
5）更新grub.cfg文件中的新條目，並將其設置爲默認值。（Update the new entry in the grub.cfg file and make it as default.）
系統使用新的ramdisk映像自動引導。(The system boots with the new ramdisk image automatically.)
# grub2-mkconfig -o /boot/grub2/grub.cfg Generating grub configuration file ... Found linux image: /boot/vmlinuz-4.14.35-1818.3.3.el7uek.x86_64 Found initrd image: /boot/initramfs-4.14.35-1818.3.3.el7uek.x86_64.img Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img Found linux image: /boot/vmlinuz-0-rescue-82f4d8c8188d4eadb4654dbbb78572bc Found initrd image: /boot/initramfs-0-rescue-82f4d8c8188d4eadb4654dbbb78572bc.img done
# grub2-set-default 'PowerPath - Red Hat Enterprise Linux Server'
6）使用新的ramdisk映像重新啓動主機，並確保dm-multipath沒有任何配置的設備。
(Restart the host with the new ramdisk image and ensure that the dm-multipath does not have any configured devices.)
# reboot
multipath原系統內核自帶的多路徑並未no loaded說明：成功
# multipath -ll Apr 21 18:56:05 | DM multipath kernel driver not loaded Apr 21 18:56:05 | DM multipath kernel driver not loaded
# powermt display dev=all Pseudo name=emcpowera VPLEX ID=CKM00192900895 Logical device ID=6000144000000010E01ACEF0A76B9A33 [device_Symm0205_042E_1_vol] state=alive; policy=ADaptive; queued-IOs=0 ============================================================================== --------------- Host --------------- - Stor - -- I/O Path -- -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 10 qla2xxx sdgm CL1-07 active alive 0 0 7 qla2xxx sdt CL1-03 active alive 0 0 10 qla2xxx sdff CL1-04 active alive 0 0 7 qla2xxx sddj CL1-02 active alive 0 0 10 qla2xxx sdbl CL1-05 active alive 0 0 10 qla2xxx sddi CL1-06 active alive 0 0 7 qla2xxx sdbk CL1-01 active alive 0 0 7 qla2xxx sdfi CL1-00 active alive 0 0 Pseudo name=emcpowerb VPLEX ID=CKM00192900895 Logical device ID=6000144000000010E01ACEF0A76B9A2E [device_Symm0205_042D_1_vol] state=alive; policy=ADaptive; queued-IOs=0 ============================================================================== --------------- Host --------------- - Stor - -- I/O Path -- -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 10 qla2xxx sdfd CL1-04 active alive 0 0 7 qla2xxx sddh CL1-02 active alive 0 0 10 qla2xxx sdbj CL1-05 active alive 0 0 10 qla2xxx sdgl CL1-07 active alive 0 0 7 qla2xxx sds CL1-03 active alive 0 0 10 qla2xxx sddg CL1-06 active alive 0 0 7 qla2xxx sdbi CL1-01 active alive 0 0 7 qla2xxx sdfg CL1-00 active alive 0 0 ......省略.......