1

I recently performed a rsync from one aws instance to another. This resulted in the root file system going into read-only mode.

I can remount the file system with read-write permission but after rebooting the system it will revert back to read only:

mount
...
/dev/xvda1 on / type xfs (ro,relatime,attr2,inode64,noquota)
...

sudo mount -o remount,rw /dev/xvda1

mount
...
/dev/xvda1 on / type xfs (rw,relatime,attr2,inode64,noquota)
...

reboot

mount
...
/dev/xvda1 on / type xfs (ro,relatime,attr2,inode64,noquota)
...

This is a CentOS instance.

Couldn't find a similar post but please redirect me if I have missed one. Any help is appreciated.

Update

journalctl
...
Jul 23 11:48:36 ip-xxx.compute.internal systemd-remount-fs[1773]: mount: can't find LABEL=root
Jul 23 11:48:36 ip-xxx.compute.internal systemd-remount-fs[1773]: /bin/mount for / exited with exit status 1.
Jul 23 11:48:36 ip-xxx.compute.internal systemd[1]: systemd-remount-fs.service: main process exited, code=exited, status=1/FAILURE
Jul 23 11:48:36 ip-xxx.compute.internal systemd[1]: Failed to start Remount Root and Kernel File Systems.
Jul 23 11:48:36 ip-xxx.compute.internal systemd[1]: Unit systemd-remount-fs.service entered failed state.
Jul 23 11:48:36 ip-xxx.compute.internal systemd[1]: systemd-remount-fs.service failed.
...
cat /etc/fstab | head -n 1
LABEL=root /         xfs    defaults,relatime  1 1

Solution Please see the accepted answer from nKn. However in my case I needed to take a couple of extra steps:

  • As this was root/boot filesystem, I needed to attach the volume to another instance and then relabel the filesystem before re-attaching to the original instance. This was in AWS and can be done by stopping two instances and going to Volumes, Actions > Attach volume > Select second instance.

  • As my system was xfs I need to use: xfs_admin -L "root" /dev/sdb (https://docs.oracle.com/cd/E37670_01/E37355/html/ol_admin_xfs.html) once attached to the second instance.

Improve this question
2
  • Usually system log provides additional information about why is the filesystem mounted as ro. Type journalctl and try to find any related information.
    – nKn
    Commented Jul 23, 2019 at 11:40
  • Thanks, I have updated the question with some logs of interest. I will do some googling around these.
    – Connor Carolan
    Commented Jul 23, 2019 at 11:55

1 Answer 1

Reset to default
0

The problem seems to be here, as you may have noted:

Jul 23 11:48:36 ip-xxx.compute.internal systemd-remount-fs[1773]: mount: can't find LABEL=root

In your /etc/fstab file you're trying to mount a device labelled root but for some reason it's not available anymore. Maybe an update/upgrade might have dropped the label (although this is not common). To fix the issue, simply re-label the device.

You'll have to find the device in your filesystem under the /dev directory, and then run:

e2label /dev/your-device root

After that and rebooting, it should mount the filesystem in rw mode.

Improve this answer
2
  • Thank you, this was the correct answer however I did need to take a couple of extra steps: - As my system was xfs I need to use: xfs_admin -L "root" /dev/sdb (docs.oracle.com/cd/E37670_01/E37355/html/ol_admin_xfs.html) - This was root/boot filesystem, I needed to attach the volume to another instance and then relabel the filesystem before re-attaching. Might be worth updating your answer but thank you for your help. I will update the question will this resolution for others if you don't update it here.
    – Connor Carolan
    Commented Jul 23, 2019 at 12:45
  • Maybe it will be better if you update the question because you know exactly the details of the steps you have followed. Glad you managed to solve it.
    – nKn
    Commented Jul 23, 2019 at 12:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .