遇到的問題：
系統重裝前有個LVM 做過擴容。重裝後後手動掛載是可以的。系統重啓後掛載失敗。
原ubuntu12.04 server 版本分區如下。
由於在升級glibc 導致系統重啓後無法啓動，做系統修復操作後還是錯。於是重裝系統。
重裝工程中對原 ext4分區進行格式化，LVM 未做改動。
系統安裝後安裝完。執行以下命令後可以正常掛載訪問。
e2label /dev/vgjenkins/lvjenkins /jenkins
e2label /dev/vgbuild/lvlocaldata /localdata
mkdir /mnt/jenkins/
mkdir /mnt/localdata
mount -L /jenkins /mnt/jenkins
mount -L /localdata /mnt/localdata
修改 /etc/fstab
vim /etc/fstab
#--add below
LABEL=/localdata /mnt/localdata ext4 defaults,relatime,usrquota,data=writeback 0 2
LABEL=/jenkins /mnt/jenkins ext4 defaults 0 2
重啓系統提示 /locadaldata 無法掛載。按S後跳過掛載開機。檢查LVM 配置如下
vgdisplay
Couldn't find device with uuid n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc.
--- Volume group ---
VG Name vgbuild
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 5
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 0
Max PV 0
Cur PV 2
Act PV 1
VG Size 6.94 TiB
PE Size 4.00 MiB
Total PE 1819968
Alloc PE / Size 1819592 / 6.94 TiB
Free PE / Size 376 / 1.47 GiB
VG UUID qM5sXD-YUFC-Ylwq-0VM1-qpI7-OvAc-CLdFJt
--- Volume group ---
VG Name vgjenkins
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 1
Act PV 1
VG Size 465.66 GiB
PE Size 4.00 MiB
Total PE 119209
Alloc PE / Size 119209 / 465.66 GiB
Free PE / Size 0 / 0
VG UUID dVGlj7-nF52-csCj-8oMg-tddl-1WtY-Zn7IeW
root@xmbuilder02:~# vgscan
Reading all physical volumes. This may take a while...
Couldn't find device with uuid n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc.
Found volume group "vgbuild" using metadata type lvm2
Found volume group "vgjenkins" using metadata type lvm2
root@xmbuilder02:~# lvscan
Couldn't find device with uuid n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc.
inactive '/dev/vgbuild/lvlocaldata' [6.94 TiB] inherit
ACTIVE '/dev/vgjenkins/lvjenkins' [465.66 GiB] inherit
查看磁盤的物理卷組
# pvdisplay
Couldn't find device with uuid n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc.
--- Physical volume ---
PV Name /dev/sda9
VG Name vgbuild
PV Size 4.22 TiB / not usable 1.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 1105249
Free PE 0
Allocated PE 1105249
PV UUID 0YRTc9-dBEp-Q6Oc-sqyX-PTzh-oqzj-AVUXHM
--- Physical volume ---
PV Name unknown device
VG Name vgbuild
PV Size 2.73 TiB / not usable 4.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 714719
Free PE 376
Allocated PE 714343
PV UUID n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc
--- Physical volume ---
PV Name /dev/sda8
VG Name vgjenkins
PV Size 465.66 GiB / not usable 1.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 119209
Free PE 0
Allocated PE 119209
PV UUID cTpMy1-ml54-jMFu-UN09-KR9E-4XT7-gStZ49
可以確定是 /dev/sdb 丟了
#lvscan
Couldn't find device with uuid n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc.
inactive '/dev/vgbuild/lvlocaldata' [6.94 TiB] inherit
ACTIVE '/dev/vgjenkins/lvjenkins' [465.66 GiB] inherit
# pvdisplay
Couldn't find device with uuid n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc.
--- Physical volume ---
PV Name unknown device
VG Name vgbuild
PV Size 2.73 TiB / not usable 4.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 714719
Free PE 376
Allocated PE 714343
PV UUID n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc
fdisk 命令查看磁盤
#fdisk -l
WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.
Disk /dev/sdb: 2997.8 GB, 2997752954880 bytes
255 heads, 63 sectors/track, 364456 cylinders, total 5854986240 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdb1 1 4294967295 2147483647+ ee GPT
WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted.
Disk /dev/sda: 5396.0 GB, 5395955318784 bytes
255 heads, 63 sectors/track, 656020 cylinders, total 10538975232 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sda1 1 4294967295 2147483647+ ee GPT
Disk /dev/mapper/vgjenkins-lvjenkins: 500.0 GB, 499998785536 bytes
255 heads, 63 sectors/track, 60788 cylinders, total 976560128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vgjenkins-lvjenkins doesn't contain a valid partition table
分析：
1. 找了不少信息，也在不少技術羣裏問了，都沒人回答，其實 想知道我要是用pvcreate 創建/dev/sdb1會導致之前數據丟失麼。要不我很想試試這個命令 來新建丟失的PV
pvcreate /dev/sdb1 --u n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc --norestorefile
2. 或者考慮 vgreduce或者 pvremove 來移除這個 PV Name unknown devicec，然後再添加上去。
但是vgreduce和命令不是太熟悉 pvremove
| vgreduce | pvremove | |
|---|---|---|
| 作用 | vgreduce命令通過刪除LVM卷組中的物理捲來減少卷組容量。不能刪除LVM卷組中剩餘的最後一個物理卷。 | pvremove命令用於刪除一個存在的物理卷。使用pvremove指令刪除物理卷時， 它將LVM分區上的物理卷信息刪除，使其不再被視爲一個物理卷。 |
| 語法 | vgreduce(選項)(參數) | pvremove(選項)(參數) |
| 選項 |
-a：如果命令行中沒有指定要刪除的物理卷，則刪除所有的空物理卷； --removemissing：刪除卷組中丟失的物理卷，使卷組恢復正常狀態。 |
-d：調試模式； -f：強制刪除； -y：對提問回答“yes”。 |
| 實例 |
使用vgreduce命令從卷組"vg2000"中移除物理卷/dev/sdb2。 在命令行中輸入下面的命令： [root@localhost ~]# vgreduce vg2000 /dev/sdb2 #將物理卷"/dev/sdb2"從卷組"vg2000"中刪除 輸出信息如下： Removed "/dev/sdb2" from volume group "vg2000" |
使用pvremove指令刪除物理卷/dev/sdb2。 在命令行中輸入下面的命令： pvremove /dev/sdb2 #刪除物理卷 Labels on physical volume "/dev/sdb2" successfully wiped |
3. 害怕數據丟失，還是來回顧一下之前添加擴容的過程吧
解決方法：
1 單純的移除 vgbuild 中丟失的部分是不行的因爲 VG vgbuild 中有一部分擴容，出問題的剛好是擴容的那部分。
root@xmbuilder02:/etc/lvm/backup# pvs
Couldn't find device with uuid n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc.
PV VG Fmt Attr PSize PFree
/dev/sda8 vgjenkins lvm2 a-- 465.66g 0
/dev/sda9 vgbuild lvm2 a-- 4.22t 0
unknown device vgbuild lvm2 a-m 2.73t 1.47g
root@xmbuilder02:/etc/lvm/backup# vgreduce --removemissing vgbuild
Couldn't find device with uuid n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc.
WARNING: Partial LV lvlocaldata needs to be repaired or removed.
There are still partial LVs in VG vgbuild.
To remove them unconditionally use: vgreduce --removemissing --force.
Proceeding to remove empty missing PVs.
打算重建PV了,用pvcreate 後sdb 是有了。
#pvcreate -u n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc --restorefile /etc/lvm/backup/vgbuild /dev/sdb
Couldn't find device with uuid n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc.
Physical volume "/dev/sdb" successfully created
root@xmbuilder02:~# pvs
PV VG Fmt Attr PSize PFree
/dev/sda8 vgjenkins lvm2 a-- 465.66g 0
/dev/sda9 vgbuild lvm2 a-- 4.22t 0
/dev/sdb vgbuild lvm2 a-m 2.73t 1.47g
於是在做個reteore試試
root@xmbuilder02:~#vgcfgrestore vgbuild
root@xmbuilder02:~# vgcfgrestore vgbuild
Restored volume group vgbuild
root@xmbuilder02:~# pvs
PV VG Fmt Attr PSize PFree
/dev/sda8 vgjenkins lvm2 a-- 465.66g 0
/dev/sda9 vgbuild lvm2 a-- 4.22t 0
/dev/sdb vgbuild lvm2 a-- 2.73t 1.47g
最終的記錄
root@xmbuilder02:~# pvs
PV VG Fmt Attr PSize PFree
/dev/sda8 vgjenkins lvm2 a-- 465.66g 0
/dev/sda9 vgbuild lvm2 a-- 4.22t 0
/dev/sdb vgbuild lvm2 a-- 2.73t 1.47g
root@xmbuilder02:~# vgscan
Reading all physical volumes. This may take a while...
Found volume group "vgbuild" using metadata type lvm2
Found volume group "vgjenkins" using metadata type lvm2
root@xmbuilder02:~# lvscan
ACTIVE '/dev/vgbuild/lvlocaldata' [6.94 TiB] inherit
ACTIVE '/dev/vgjenkins/lvjenkins' [465.66 GiB] inherit
root@xmbuilder02:~# vgdisplay
--- Volume group ---
VG Name vgbuild
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 10
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 0
Max PV 0
Cur PV 2
Act PV 2
VG Size 6.94 TiB
PE Size 4.00 MiB
Total PE 1819968
Alloc PE / Size 1819592 / 6.94 TiB
Free PE / Size 376 / 1.47 GiB
VG UUID qM5sXD-YUFC-Ylwq-0VM1-qpI7-OvAc-CLdFJt
--- Volume group ---
VG Name vgjenkins
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 1
Act PV 1
VG Size 465.66 GiB
PE Size 4.00 MiB
Total PE 119209
Alloc PE / Size 119209 / 465.66 GiB
Free PE / Size 0 / 0
VG UUID dVGlj7-nF52-csCj-8oMg-tddl-1WtY-Zn7IeW
root@xmbuilder02:~# pvdisplay
--- Physical volume ---
PV Name /dev/sda9
VG Name vgbuild
PV Size 4.22 TiB / not usable 1.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 1105249
Free PE 0
Allocated PE 1105249
PV UUID 0YRTc9-dBEp-Q6Oc-sqyX-PTzh-oqzj-AVUXHM
--- Physical volume ---
PV Name /dev/sdb
VG Name vgbuild
PV Size 2.73 TiB / not usable 4.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 714719
Free PE 376
Allocated PE 714343
PV UUID n3lryH-nt0A-8H1g-6Dz3-gR43-8DKE-BvRXcc
--- Physical volume ---
PV Name /dev/sda8
VG Name vgjenkins
PV Size 465.66 GiB / not usable 1.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 119209
Free PE 0
Allocated PE 119209
PV UUID cTpMy1-ml54-jMFu-UN09-KR9E-4XT7-gStZ49
最終的結果，東西沒丟，舒心一笑。
root@xmbuilder02:~# e2label /dev/vgbuild/lvlocaldata /localdata
root@xmbuilder02:~# mount -L /localdata /mnt/localdata
root@xmbuilder02:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 63G 12K 63G 1% /dev
tmpfs 13G 1.9M 13G 1% /run
/dev/sda2 184G 400M 174G 1% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 63G 0 63G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sda5 9.1G 21M 8.6G 1% /tmp
/dev/sda3 181M 38M 131M 23% /boot
/dev/sda6 9.1G 557M 8.1G 7% /usr
/dev/sda7 9.1G 350M 8.3G 4% /var
/dev/mapper/vgjenkins-lvjenkins 459G 324G 136G 71% /mnt/jenkins
/dev/mapper/vgbuild-lvlocaldata 6.9T 6.8T 156G 98% /mnt/localdata