MDADM RAID5 recovery, refused to add replacement disk

Question

I dealing with a faulty raid5 setup (3x2TB disks):

Number  Start   End     Size    File system     Name     Flags
 1      1049kB  1026MB  1024MB  ext3            primary
 2      1026MB  6146MB  5120MB  ext3            primary
 3      6146MB  6147MB  1049kB                  primary
 4      6147MB  6148MB  1049kB                  primary
 5      6148MB  7172MB  1024MB  linux-swap(v1)  primary
 6      7172MB  1992GB  1985GB                  primary

I created the same partition table on a new disk, reassembling the first 2 linux partition was no problem with:

mdadm --assemble --run /dev/md1 /dev/sdb2 /dev/sdd2 Then adding my new disk: mdadm --add /dev/md1 /dev/sda2

But when I got to the important one (the 6th data storage partition) first it started crying that the size is not big enough on the 3rd disk, well oke, I deleted the swap out on my 3rd disk and created a bigger /dev/sda5.

After this it refused adding it with: mdadm using add cannot work and might destroy data on /dev/sda5 -> which I don't care about since there is nothing on it.

I tried everything like: mdadm --assemble --force --run /dev/md6 /dev/sdb6 /dev/sdd6 /dev/sda5

It refused to readd the 3rd disk. So I read that I have to recreate the array what I did:

mdadm --create /dev/md6 -v -l 5 -n 3 /dev/sdb6 /dev/sdd6 /dev/sda5

It allowed it, but I have a strong feeling that instead of doing any kind of recovery right now, it just wiped out all the raid data! What /proc/mdstat shows is:

md6 : active raid5 sda5[3] sdd6[1] sdb6[0]
      3876362240 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
      [>....................]  recovery =  4.0% (78723276/1938181120) finish=337.9min speed=91711K/sec

I have no idea what's happening now and still have to wait 337 minutes until it completes. I tried file -s /dev/md6 and instead of saying ext3 it just say sticky data. Will my data be accessible and recovered at the end of this?

Please help!

Thanks

Answer 1

"It looks like the superblocks are screwed up on some of your disks, and no amount of assembling is going to put the array back together. The only way to fix this is to recreate the superblocks for the array. I know that sounds scary, but mdadm is smart enough to not overwrite your data. Here's an example...."

Nope it's not, I just lost all my data, as I suspected it created a fresh raid array. I know it in the second I did file -s at the beginning and it said sticky data :( that I lost my data.

49bbb20  23 4d 33 4d 33 4d 33 4d  33 4d 33 4d 33 4d 33 4d  |#M3M3M3M3M3M3M3M|
049bbb30  33 4d 33 4d 33 4d 33 4d  33 4d 33 4d 31 4d 23 4d  |3M3M3M3M3M3M1M#M|
049bbb40  33 4d 33 4d 33 4d 33 4d  33 4d 33 4d 33 4d 33 4d  |3M3M3M3M3M3M3M3M|

The whole new md6 array is now full of this M3M3M3.