In a new rocky9 vm, LVM2 does NOT see the newly attached PVs under LUKS.
===
The end goal is to migrate a simple "repo" server (that holds iso files) from centos8 to rocky9.
I have a freshly installed rocky9 linux vm. It boots fine and has LUKS and LVM.
While the old and new VMs were stopped, I've taken the
<disk>...</disk> sections for the data disks and added them into the new-ish /etc/libvirt/qemu/repo9.xml , and I did a /usr/bin/systemctl reload libvirtd . It sees the disks.
I've also taken the entries from /etc/crypttab and migrated them to the new repo9 host.
After a fresh reboot, it asks for the luks PW and I provide it once (same pw for all devices).
# /bin/cat /etc/redhat-release
Rocky Linux release 9.0 (Blue Onyx)
Yes, blkid sees them.
# /usr/sbin/blkid | grep -i luks
/dev/mapper/luks-2f5518cd-8453-4660-abe0-b2550b30ba10: UUID="52oE6g-azSo-95kU-3NGu-Hyfq-tcN6-XKcQe5" TYPE="LVM2_member"
/dev/sda2: UUID="2f5518cd-8453-4660-abe0-b2550b30ba10" TYPE="crypto_LUKS" PARTUUID="42a79f19-02"
/dev/vda1: UUID="5c44226e-3bc5-4b04-b369-beea69f6ca8c" TYPE="crypto_LUKS" PARTUUID="82535424-01"
/dev/vdb1: UUID="a50cd287-d4d2-4301-bd44-d3937e4d9df3" TYPE="crypto_LUKS" PARTUUID="9f2b9d8e-01"
/dev/vdc1: UUID="b893a690-a1a0-4021-b078-1d971fe156d2" TYPE="crypto_LUKS" PARTUUID="abbfb472-01"
/dev/vdd1: UUID="7e42260f-9739-4d8a-8e07-5cc19590d460" TYPE="crypto_LUKS" PARTUUID="40333c29-01"
/dev/mapper/luks-b893a690-a1a0-4021-b078-1d971fe156d2: UUID="tqgPNb-Jbos-IlCG-nWD9-8yMP-5ZII-jyvZb0" TYPE="LVM2_member"
/dev/mapper/luks-5c44226e-3bc5-4b04-b369-beea69f6ca8c: UUID="mnraHG-RX5m-eEmV-uYhr-UEng-x8JT-5YKZ3P" TYPE="LVM2_member"
/dev/mapper/luks-7e42260f-9739-4d8a-8e07-5cc19590d460: UUID="pSuS4s-e5vc-r3wV-PhBq-d7Nt-Myjf-RxZ5t0" TYPE="LVM2_member"
/dev/mapper/luks-a50cd287-d4d2-4301-bd44-d3937e4d9df3: UUID="odlXGf-Hzzd-C30f-cNML-wEFb-UVEA-IZYdzf" TYPE="LVM2_member"
And lsblk sees them.
# /usr/bin/lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 14G 0 disk
├─sda1 8:1 0 900M 0 part /boot
└─sda2 8:2 0 11.7G 0 part
└─luks-2f5518cd-8453-4660-abe0-b2550b30ba10 253:0 0 11.7G 0 crypt
├─vg00-slash 253:1 0 9000M 0 lvm /
vda 252:0 0 8G 0 disk
└─vda1 252:1 0 8G 0 part
└─luks-5c44226e-3bc5-4b04-b369-beea69f6ca8c 253:4 0 8G 0 crypt
vdb 252:16 0 7G 0 disk
└─vdb1 252:17 0 7G 0 part
└─luks-a50cd287-d4d2-4301-bd44-d3937e4d9df3 253:5 0 7G 0 crypt
vdc 252:32 0 15G 0 disk
└─vdc1 252:33 0 15G 0 part
└─luks-b893a690-a1a0-4021-b078-1d971fe156d2 253:6 0 15G 0 crypt
vdd 252:48 0 90G 0 disk
└─vdd1 252:49 0 90G 0 part
└─luks-7e42260f-9739-4d8a-8e07-5cc19590d460 253:9 0 90G 0 crypt
The output above is bad in that it only goes as far as the luks entry (for non-vg00). A proper looking output should have the LV under luks, like:
vdd 252:48 0 90G 0 disk
└─vdd1 252:49 0 90G 0 part
└─luks-7e42260f-9739-4d8a-8e07-5cc19590d460 253:9 0 90G 0 crypt
├─vg01-opt 253:7 0 500M 0 lvm /opt
...
Here is the output of the file commands, which says the luks-* points to dm-, and the dm- are LUKS type.
# /usr/bin/file -s /dev/mapper/luks-*
/dev/mapper/luks-2f5518cd-8453-4660-abe0-b2550b30ba10: symbolic link to ../dm-0
/dev/mapper/luks-5c44226e-3bc5-4b04-b369-beea69f6ca8c: symbolic link to ../dm-4
/dev/mapper/luks-7e42260f-9739-4d8a-8e07-5cc19590d460: symbolic link to ../dm-9
/dev/mapper/luks-a50cd287-d4d2-4301-bd44-d3937e4d9df3: symbolic link to ../dm-5
/dev/mapper/luks-b893a690-a1a0-4021-b078-1d971fe156d2: symbolic link to ../dm-6
# /usr/bin/file -s /dev/dm-[04956]
/dev/dm-0: LVM2 PV (Linux Logical Volume Manager), UUID: 52oE6g-azSo-95kU-3NGu-Hyfq-tcN6-XKcQe5, size: 12566134784
/dev/dm-4: LVM2 PV (Linux Logical Volume Manager), UUID: mnraHG-RX5m-eEmV-uYhr-UEng-x8JT-5YKZ3P, size: 8586788864
/dev/dm-5: LVM2 PV (Linux Logical Volume Manager), UUID: odlXGf-Hzzd-C30f-cNML-wEFb-UVEA-IZYdzf, size: 7513047040
/dev/dm-6: LVM2 PV (Linux Logical Volume Manager), UUID: tqgPNb-Jbos-IlCG-nWD9-8yMP-5ZII-jyvZb0, size: 16102981632
/dev/dm-9: LVM2 PV (Linux Logical Volume Manager), UUID: pSuS4s-e5vc-r3wV-PhBq-d7Nt-Myjf-RxZ5t0, size: 96618938368
So I try manual steps to see the PVs.
# /usr/sbin/pvscan #(only finds the pv for vg00)
PV /dev/mapper/luks-2f5518cd-8453-4660-abe0-b2550b30ba10 VG vg00 lvm2 [<11.70 GiB / 980.00 MiB free]
Total: 1 [<11.70 GiB] / in use: 1 [<11.70 GiB] / in no VG: 0 [0 ]
# /usr/sbin/pvs #(only finds the pv for vg00)
PV VG Fmt Attr PSize PFree
/dev/mapper/luks-2f5518cd-8453-4660-abe0-b2550b30ba10 vg00 lvm2 a-- <11.70g 980.00m
Here's the lvm2 version:
# /usr/bin/rpm -q lvm2
lvm2-2.03.14-4.el9.x86_64
And the configs are all defaults, no bad rejecting filters, accepts all.
# /usr/bin/rpm -qV lvm2
(no output)
# /usr/sbin/lvm dumpconfig
config {
}
local {
}
dmeventd {
}
activation {
}
global {
}
shell {
}
backup {
}
log {
}
allocation {
}
devices {
}
# /bin/cat /etc/crypttab
luks-2f5518cd-8453-4660-abe0-b2550b30ba10 UUID=2f5518cd-8453-4660-abe0-b2550b30ba10 none discard
luks-5c44226e-3bc5-4b04-b369-beea69f6ca8c UUID=5c44226e-3bc5-4b04-b369-beea69f6ca8c none discard
luks-a50cd287-d4d2-4301-bd44-d3937e4d9df3 UUID=a50cd287-d4d2-4301-bd44-d3937e4d9df3 none discard
luks-b893a690-a1a0-4021-b078-1d971fe156d2 UUID=b893a690-a1a0-4021-b078-1d971fe156d2 none discard
luks-7e42260f-9739-4d8a-8e07-5cc19590d460 UUID=7e42260f-9739-4d8a-8e07-5cc19590d460 none discard
I try to explicitly mention an inline config to filter->accept relevant items, but no luck:
# /usr/sbin/pvscan --config 'devices { filter = [ "a|.*|" ] } '
/usr/sbin/pvscan --config 'devices { filter = [ "a|.*|" , "a|/dev/mapper/luks-*|" , "a|/dev/dm-*|" ] } '
PV /dev/mapper/luks-2f5518cd-8453-4660-abe0-b2550b30ba10 VG vg00 lvm2 [<11.70 GiB / 980.00 MiB free] Total: 1 [<11.70 GiB] / in use: 1 [<11.70 GiB] / in no VG: 0 [0 ]
But here's a glimmer of hope.
The ONLY trick I have that makes the LVM PVs visible is if I specify them after --devices :
# /usr/bin/pvs --devices $(echo /dev/mapper/luks-* | tr ' ' ',' )
PV VG Fmt Attr PSize PFree
/dev/mapper/luks-2f5518cd-8453-4660-abe0-b2550b30ba10 vg00 lvm2 a-- <11.70g 980.00m
/dev/mapper/luks-5c44226e-3bc5-4b04-b369-beea69f6ca8c vg01 lvm2 a-- <8.00g 0
/dev/mapper/luks-7e42260f-9739-4d8a-8e07-5cc19590d460 vg01 lvm2 a-- 89.98g 0
/dev/mapper/luks-a50cd287-d4d2-4301-bd44-d3937e4d9df3 vg01 lvm2 a-- <7.00g 0
/dev/mapper/luks-b893a690-a1a0-4021-b078-1d971fe156d2 vg01 lvm2 a-- <15.00g 5.13g
But if i run it without --devices, again it reverts to not seeing the PVs.
# /usr/sbin/pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/luks-2f5518cd-8453-4660-abe0-b2550b30ba10 vg00 lvm2 a-- <11.70g 980.00m
I tried a yum update, not many patches yet, just kernel* and binutils, but those didn't help.
Why is my LVM2 not seeing most PVs under LUKS?
Any ideas would be greatly appreciated!!
Thank you.
