Today my server is a ubuntu 22.04 setup where I am starting 2 VMs namely mgmt-vm and storage-vm using LXD. Both the VMs are using lvm based storage pool which is on top of logical software RAID5 drive containing 3 physical drives.
- Inside storage-vm I am setting up a ceph instance with 3 loop devices as disks.
- Inside mgmt-vm, I am starting 4 containers whose storage is mapped with the CEPH in storage-vm.
As I am trying to copy around TB worth of data from the management vm to one of the containers, I am seeing that it is quite slow. As shown in below picture, nmon is showing that dm-5/dm-2/dm-3 are 100% which are mapping to my storage-vm. I can see a bottleneck here and not sure how can i resolve the same
I am curious which of the below cases will be my ideal solution
- Removing software based RAID5 and just using the physical disks (well in that case i will loose RAID capability)
- Use a Hardware based RAID controller instead of Software based RAID
Any other suggestions would be of great help.
