After a power outage I was greeted with this delightful error…
TASK ERROR: activating LV 'pve/data' failed: Check of pool pve/data failed (status:1). Manual repair required!
A little diagnostics first to make sure I didn’t lose any critical metadata:
root# pvs
PV VG Fmt Attr PSize PFree
/dev/sda3 pve lvm2 a-- 1.36t <16.00g
root# vgs
VG #PV #LV #SN Attr VSize VFree
pve 1 13 0 wz--n- 1.36t <16.00g
root# lvs -a
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
data pve twi---tz-- 1.22t
[data_tdata] pve Twi------- 1.22t
[data_tmeta] pve ewi------- 12.75g
[lvol0_pmspare] pve ewi------- 12.75g
root pve -wi-ao---- 96.00g
snap_vm-100-disk-0_vzdump pve Vri---tz-k 256.00g data vm-100-disk-0
swap pve -wi-ao---- 8.00g
vm-100-disk-0 pve Vwi---tz-- 256.00g data
vm-101-disk-0 pve Vwi---tz-- 64.00g data
vm-102-disk-0 pve Vwi---tz-- 20.00g data
vm-103-disk-0 pve Vwi---tz-- 24.00g data
vm-105-disk-0 pve Vwi---tz-- 16.00g data
vm-108-disk-0 pve Vwi---tz-- 4.00m data
vm-108-disk-1 pve Vwi---tz-- 32.00g data
vm-109-disk-0 pve Vwi---tz-- 96.00g data
vm-111-disk-0 pve Vwi---tz-- 8.00g data
And then finally repairing so I can mount and start them again:
WARNING: I lost vm-109-disk-0, it kept returning this generic error Failed to run lxc.hook.pre-start for container 109. I attempted to mount with pct mount 109 and saw that a better error message about the container disk. I’m not sure if the following repair did anything positive or negative to it. I ended up restoring that one from a local backup.
root# lvconvert --repair pve/data
WARNING: LV pve/data_meta0 holds a backup of the unrepaired metadata. Use lvremove when no longer required.