Here's my situation, I have a NAS that I use as a backup server, as well as another Linux machine that had a full backup as well. The NAS has three 2TB drives, so I decided to try BTRFS with RAID5 to see how well that was working. I mirrored both and everything was well. Later I decided to upgrade the OS on the second machine, so I deleted the folder with the second backup, (the data is still potentially there). Thinking I had plenty of time with the NAS backup in place, I didn't get around to making the second backup, (this happened within the last 2 weeks). Now my NAS has a bad hard drive, which I replaced already, and I can't get the BTRFS system to mount up. I started reading this website and ran the -repair option, which seemed like a good idea at the time, but later in the thread it's pointed out that it is not.
So now I'm at a full stop so I don't damage anything further. dmesg states that BTRFS can't mount the device because a replace operation is ongoing, but btrfs replace status /mnt says that it was never started.
My questions:
1) Is it still possible to recover and backup my BTRFS drive?
2) If so, what information do I need to provide to get this underway?
3) If not, are there any tools available right now that can restore my other backup? I think I formatted the drive from ext4 to xfs.
Thanks in advance.
Edit: I forgot to mention that I did start the replace initially by using the information here. And that it had to be restarted due to a power failure.
Edit 2: Would it be possible to have the RAID5 downgrade from 3 drives to just 2 without the other drive in there?
Edit 3: Getting a kernel bug when I let the system just sit there:
[ 685.809945] ------------[ cut here ]------------ [ 685.810386] kernel BUG at /build/linux-a2WvEb/linux-4.4.0/fs/btrfs/raid56.c:1917! [ 685.810833] invalid opcode: 0000 [#1] SMP [ 685.811279] Modules linked in: crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper input_leds cryptd snd_hda_codec_realtek serio_raw snd_hda_codec_generic snd_hda_codec_hdmi shpchp snd_hda_intel k10temp snd_hda_codec snd_hda_core snd_hwdep snd_pcm i2c_piix4 snd_seq_midi snd_seq_midi_event snd_rawmidi nuvoton_cir rc_core snd_seq snd_seq_device snd_timer snd mac_hid soundcore kvm_amd kvm irqbypass nfsd auth_rpcgss parport_pc nfs_acl lockd ppdev grace lp sunrpc parport autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear pata_acpi hid_generic usbhid hid amdkfd amd_iommu_v2 radeon i2c_algo_bit ttm drm_kms_helper syscopyarea sysfillrect sysimgblt r8169 fb_sys_fops mii drm pata_atiixp [ 685.813359] ahci libahci fjes [ 685.813876] CPU: 1 PID: 2553 Comm: btrfs-transacti Not tainted 4.4.0-36-generic #55-Ubuntu [ 685.814400] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./FM2A85X-ITX, BIOS P1.50 05/04/2013 [ 685.814931] task: ffff8800db2cb700 ti: ffff8800d3a54000 task.ti: ffff8800d3a54000 [ 685.815462] RIP: 0010:[] [] __raid_recover_end_io+0x475/0x520 [btrfs] [ 685.816028] RSP: 0018:ffff8800d3a57720 EFLAGS: 00010213 [ 685.816568] RAX: ffff8800d3ce5700 RBX: ffff8800d0c66000 RCX: 0000000000000000 [ 685.817112] RDX: 0000000080000000 RSI: 0000000000000000 RDI: 0000000000000003 [ 685.817658] RBP: ffff8800d3a57760 R08: ffffea00035596c0 R09: 0000000000000018 [ 685.818207] R10: ffff8800d0c1d9a0 R11: ffffffff81ccf5ea R12: ffff8800d0c1d9b0 [ 685.818759] R13: 0000000000000003 R14: ffff8800d0c1d9a0 R15: ffff8800d2f87000 [ 685.819316] FS: 00007f108f26b700(0000) GS:ffff88011ec80000(0000) knlGS:0000000000000000 [ 685.819876] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 685.820436] CR2: 0000000000ef5000 CR3: 0000000001e0a000 CR4: 00000000000406e0 [ 685.820999] Stack: [ 685.821557] 0000000000000000 0000000000000010 0000000100000001 0000000000000000 [ 685.822128] 0000000000000010 ffff8800d2f87000 0000000000000003 ffff8800d2f87000 [ 685.822695] ffff8800d3a577b0 ffffffffc04169d5 ffff8800d3a57770 0000000000000000 [ 685.823258] Call Trace: [ 685.823832] [] __raid56_parity_recover+0x1e5/0x200 [btrfs] [ 685.824410] [] raid56_parity_recover+0xf6/0x160 [btrfs] [ 685.824990] [] btrfs_map_bio+0xe3/0x320 [btrfs] [ 685.825557] [] ? kmem_cache_alloc+0x187/0x1f0 [ 685.826137] [] ? btrfs_bio_wq_end_io+0x28/0x70 [btrfs] [ 685.826720] [] btree_submit_bio_hook+0xff/0x110 [btrfs] [ 685.827305] [] submit_one_bio+0x6d/0xa0 [btrfs] [ 685.827886] [] read_extent_buffer_pages+0x193/0x2f0 [btrfs] [ 685.828464] [] ? free_root_pointers+0x60/0x60 [btrfs] [ 685.829040] [] btree_read_extent_buffer_pages.constprop.51+0xac/0x110 [btrfs] [ 685.829621] [] btrfs_read_buffer+0x2c/0x30 [btrfs] [ 685.830199] [] read_block_for_search.isra.34+0xe4/0x380 [btrfs] [ 685.830781] [] ? unlock_up+0x172/0x1a0 [btrfs] [ 685.831363] [] btrfs_search_slot+0x381/0xa00 [btrfs] [ 685.831946] [] lookup_inline_extent_backref+0xed/0x5a0 [btrfs] [ 685.832532] [] ? set_extent_dirty+0x32/0x40 [btrfs] [ 685.833114] [] ? btrfs_alloc_path+0x1a/0x20 [btrfs] [ 685.833696] [] __btrfs_free_extent.isra.70+0x113/0xd30 [btrfs] [ 685.834285] [] ? btrfs_merge_delayed_refs+0x66/0x650 [btrfs] [ 685.834873] [] __btrfs_run_delayed_refs+0xaab/0x11f0 [btrfs] [ 685.835452] [] ? lock_timer_base.isra.22+0x54/0x70 [ 685.836038] [] btrfs_run_delayed_refs+0x7d/0x2a0 [btrfs] [ 685.836627] [] ? btrfs_wait_pending_ordered+0x22/0x90 [btrfs] [ 685.837214] [] btrfs_commit_transaction+0x4d2/0xa90 [btrfs] [ 685.837801] [] transaction_kthread+0x229/0x240 [btrfs] [ 685.838390] [] ? btrfs_cleanup_transaction+0x570/0x570 [btrfs] [ 685.838970] [] kthread+0xd8/0xf0 [ 685.839548] [] ? kthread_create_on_node+0x1e0/0x1e0 [ 685.840132] [] ret_from_fork+0x3f/0x70 [ 685.840718] [] ? kthread_create_on_node+0x1e0/0x1e0 [ 685.841304] Code: 75 d4 89 45 d0 e9 94 fe ff ff 8b 5d d0 4c 89 f1 be 00 10 00 00 89 da ff 15 c9 dc eb ff 8b 45 d4 89 5d d4 89 45 d0 e9 72 fd ff ff 0b 4d 8d a7 98 00 00 00 4c 89 e7 e8 ea 75 41 c1 f0 41 80 8f [ 685.842653] RIP [] __raid_recover_end_io+0x475/0x520 [btrfs] [ 685.843314] RSP [ 685.843980] ---[ end trace d312a2207e01141e ]---
