I have a weird problem with my NAS, a Zyxel 540 with 4 × 2TB drives in RAID 5 configuration. Everything was working smoothly until I decided to replace one drive with a new WD Red, same size. As everyone expected, NAS would find out a new disk has inserted and starts to rebuild the RAID, meanwhile data are still safe... I already did this operation and worked, so... no problem!
In my dreams...
After I replaced the drive, NAS said the volume was inaccessible.
I panicked, so I mounted back the old drive...nothing happened, still problems...but data were accessible in the NAS manager, but not via LAN, and impossible to copy if not via terminal.
I tried (just to be sure) a partial recover on one drive with PhotoRec. The data are still there, so the problem must be on headers or whatever.
I tried some commands by ssh to check status of the RAID, like:
mdadm --create etc
mdadm --examine /dev/md2
and moreover, find out drives order is gone, so I started all combinations like:
mdadm --create --assume-clean --level=5 --raid-devices=4 --metadata=1.2 --chunk=64K --layout=left-symmetric /dev/md2 /dev/sdd3 /dev/sdb3 /dev/sdc3 /dev/sda3
I expected to make NAS work again with the correct combination, until I submitted this combination: C / A / D / B
Now I can't change combination, I'm stuck with this message:
mdadm --stop /dev/md2
it responds with this:
mdadm: Cannot get exclusive access to /dev/md2:Perhaps a running process, mounted filesystem or active volume group?
I also tried cat /proc/mounts
, volume is not mounted :(
lsof | grep /dev/md2
, nothing appears
AND
# e2fsck -n /dev/md2
e2fsck 1.42.12 (29-Aug-2014)
Warning! /dev/md2 is in use.
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open /dev/md2
The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem. If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>
LVM configuration:
output from pvs
:
PV VG Fmt Attr PSize PFree
/dev/md2 vg_ca74d470 lvm2 a-- 5.45t 0
Output from vgs
:
VG #PV #LV #SN Attr VSize VFree
vg_ca74d470 1 2 0 wz--n- 5.45t 0
Output from lvs
:
LV VG Attr LSize [snip]
lv_be37a89a vg_ca74d470 -wi-a----- 5.45t
vg_info_area vg_ca74d470 -wi-a----- 100.00m
Software RAID configuration:
# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md2 : active raid5 sda3[1] sdd3[3] sdb3[2]
5848150464 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/3] [_UUU]
md1 : active raid1 sdb2[4] sdd2[6] sdc2[5] sda2[7]
1998784 blocks super 1.2 [4/4] [UUUU]
md0 : active raid1 sdb1[4] sdd1[6] sdc1[5] sda1[7]
1997760 blocks super 1.2 [4/4] [UUUU]
unused devices: <none>
I'm out of options, guys... What should I do?