WSL 2 should automatically release disk space back to the host OS

[dsmaher]

Is your feature request related to a problem? Please describe.
I occasionally have workloads that consume large amounts of disk space for temporary use in /tmp. For example, transcoding large video files or processing large datasets exported from a database. Since WSL2 stores its filesystem on a .vhdx, that file grows when the dataset is processed and never releases that space. Also, it doesn't appear to re-use released disk space in the .vhdx when files are deleted... it seems to prefer to grow the file rather than re-use existing empty space. (I haven't explicitly tested that theory, but my workload deletes temporary files as they are used and I never had enough of them existing simultaneously to reach 250GB, but the size of the .vhdx eventually expanded to that size.

When I'm done with that workload, I have to:

wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full

This is annoying since it basically means that my system backups include a huge .vhdx file that is mostly empty. Also, until about 2 weeks ago, it was consuming the entire remainder of my system disk. I had only 50GB free, which was enough to handle the workload, but since it continued to grow even after files were removed, that didn't matter. (I figured it was time to upgrade the disk size anyway)

Describe the solution you'd like
Automatic compaction of the .vhdx, or a way to do so while WSL2 is still running so that I can schedule it for frequent cleanups.

Describe alternatives you've considered

• Going back to WSL1 since it uses the host filesystem and is faster accessing that than WSL2. However, this alternative makes docker less useful. Docker on WSL2 is actually almost perfect; not so much on WSL1.
• Exporting/Importing WSL 2 to move it to another disk. Not even sure if this is possible, but if it is,
  it would probably be fine for my desktop, but not really an option for my laptop. Not really worth the time either since that entire process would probably take about 30 minutes, but compacting is usually under 3.

[adrianciacob]

+1 for being able to move the .vhdx to a different partition, either via config file or the "Move" feature in the Control Panel of Windows (currently getting a message that it's disabled)

[dvescovi1]

+1 here too

[wizhippo]

+1 also add the ability to specify the initial size limit.

[dvescovi1]

I also would like to see a setting for initial partition size limit. It looks like all distros are installed with a 250GB partition size. I see no (easy) way to extend this. My yocto builds require slightly more space than this and they have to be on an ext3/4 partition so I can not use the /mnt/c or other NTFS partitions.

[davidwin]

In Home you don't even have the Optimize-VHD cmdlet, but thankfully (the slightly clumsier) diskpart command which can also be used to compact the .vhdx file. When I did this, it seemed like I first had to run sudo fstrim / in WSL to get a significant saving (14GB reclaimed with 8GB in use). A periodic background task that did this is really needed (and even better if it could be done online, of course).

The default partition size should have some connection to the size of the Windows host partition. What is the use of a limit that exceeds that of the host (unless actual magic is involved)? Also, artificially limiting the maximum size to a fraction of that of the host seems a bit unnecessary.

I think you really want an experience that is as close to that of WSL1 as possible, but with the incredible perf and compat gains, of course. That is, the WSL2 file system shouldn't feel like it was a static volume bound to a file on the host (although it is). Ideally, even the WSL partition/volume size should shrink and grow with available space on the host in a somewhat timely fashion. Now there is a stretch goal for you!

[taylankasap]

FWIW, since these commands looked benign I ran them to reclaim some disk space:

wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full

This broke my docker. I had to reset to factory defaults & reinstall docker and restart the pc a few times along the way to fix. All docker data is gone (doesn't matter in my case because it's just dev environment). YMMV.

Microsoft Windows 19041.1 with WSL2
Docker Desktop Edge 2.1.7.0 (41536) | Engine: 19.03.5

[StefanScherer]

I've tried the commands as well. I could find an issue that docker engine (docker.sock) is not available after starting. But after restarting Docker Desktop and changing the settings / resources / wsl integration to off + apply, then to on again + apply did the trick. Looks like the sock

I ran the test a second time, but now my whole Ubuntu 18.04 distro is broken, the terminal closes either immediately or after some seconds without any text in it.

[taylankasap]

Maybe disabling wsl integration first, then quitting docker and then using the optimize-vhd command can make it work. But I'm not going to take that risk now.

[StefanScherer]

It's back alive, got the Ubuntu distro working again. Seem like I just have to wait a little bit. Terminal opens and I have the bash prompt. The WSL integration was disabled, after enabling again docker also works again in Ubuntu.

[MatyRi]

If you are looking for a way how to reclaim your disk space from Docker using WSL2, then there is a button for this if you are using Docker for Desktop (Edge 2.3.0.0). In the Dashboard / Troubleshoot - there is a Clean / Purge Data option which will let you select what do you want to remove. It safely shrank the Docker vhdx disk file to a few kilobytes. Just a warning that this obviously removes everything and not just unused space.

It does not solve the original issue but it's a quick way to a fresh start.

[merkuriy]

For Windows 10 Home (alternative Optimize-VHD cmdlet):

wsl --shutdown
diskpart
# open window Diskpart
select vdisk file="C:\WSL-Distros\…\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
exit

Thanks to @davidwin for the tip #4699 (comment).

[sorenwacker]

I just got no disc space, I have not even installed much. I added a symbolic link that was no problem on my old system.

[nickjj]

I'm on Windows 10 Pro build 2004 here (stable 2020 spring edition) and I just noticed I lost 60GB on my primary SSD from this because I started moving my source code into WSL 2's file system and forgot one of my podcast sites has a bunch of wave files in a private directory.

When I run optimize-vhd -Path .\ext4.vhdx -Mode full as admin from PowerShell it throws an error with:

optimize-vhd : '.\ext4.vhdx' is not an existing virtual hard disk file. 
At line:1 char:1                                                                                                        
+ optimize-vhd -Path .\ext4.vhdx -Mode full
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (:) [Optimize-VHD], VirtualizationException
+ FullyQualifiedErrorId : InvalidParameter,Microsoft.Vhd.PowerShell.Cmdlets.OptimizeVhd                                                                    

How can I get around this error? This put my SSD at a dangerously low level of space and I'd rather not have to reinstall the entire WSL 2 instance from scratch and reconfigure Docker every time I accidentally copy an unused file into WSL 2's file system.

[onomatopellan]

@nickjj You need to be in the directory where the distro's vhdx resides. In my case with Ubuntu20.04 I had to

wsl.exe --shutdown
cd C:\Users\onoma\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\
optimize-vhd -Path .\ext4.vhdx -Mode full

[nickjj]

Thanks, I was able to find it there too. That and what @StefanScherer mentioned about restarting Docker, along with disabling / enabling the distro in Docker Desktop again got things back to normal without any system level restart.

[rainabba]

My contribution, bringing together knowledge from above. It still requires a little manual work to get the DistroFolder variable set for the distro you want to target (you could have more than one, find them with wsl --list)

Cheers!

### Optimize (shrink) WSL 2 .vhdx
## Must be run in PowerShell as Administrator user
# DistroFolder found at: $env:LOCALAPPDATA\Packages\
# Examples:
#   CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc
#   CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc

cd $env:LOCALAPPDATA\Packages\REPLACE_ME_WITH_TARGET_DISTRO_FOLDERNAME\LocalState\
wsl --shutdown
optimize-vhd -Path .\ext4.vhdx -Mode full
#Run `wsl` or your favorite terminal to resume use

BELOW UNTESTED!!
More info at https://docs.microsoft.com/en-us/powershell/module/hyper-v/resize-vhd?view=win10-ps

### Resize WSL 2 .vhdx
## WARNING!! UNTESTED!! Read and make note of the MaxSizeGB variable and functionality before running
## Must be run in PowerShell as Administrator user
# DistroFolder found at: $env:LOCALAPPDATA\Packages\
# Examples:
#   CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc
#   CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc

Set-Variable -Name "MaxSizeGB" -Value "32GB"
cd $env:LOCALAPPDATA\Packages\REPLACE_ME_WITH_TARGET_DISTRO_FOLDERNAME\LocalState\
wsl --shutdown
resize-vhd -Path .\ext4.vhdx -SizeBytes $MaxSize
#Run `wsl` or your favorite terminal to resume use

[eromoe]

Both cmd and powershell can't find optimize-vhd command even in admin preveliege

optimize-vhd : 无法将“optimize-vhd”项识别为 cmdlet、函数、脚本文件或可运行程序的名称。请检查名称的拼写，如果包括路径
，请确保路径正确，然后再试一次。
所在位置 行:1 字符: 1
+ optimize-vhd -Path .\ext4.vhdx -Mode full
+ ~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (optimize-vhd:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

[onomatopellan]

@eromoe optimize-vhd is only available in Windows 10 Pro with Hyper-v feature installed. Otherwise you will need to use the compact option in Diskpart.

[mieslep]

The WSL2 disk management is definitely a pig. Developing some (legacy app) Docker containers, after a week my "working" WSL2 .vhdx is 110G (with only 14 GB of actual data) and my docker-desktop-data .vhdx is 40G (with only 3 GB of actual data).

[nickjj]

Has anyone ever seen this when trying to run Optimize-VHD to compact things?

Optimize-VHD : Failed to compact the virtual disk.
The system failed to compact 'C:\Users\Nick\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\ext4.vhdx'.   Failed to compact the virtual disk.
The system failed to compact
'C:\Users\Nick\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc\LocalState\ext4.vhdx':   The process cannot access the file because it is being used by another process. (0x80070020).
At line:1 char:1
+ Optimize-VHD -Path .\ext4.vhdx -Mode full
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ResourceBusy: (:) [Optimize-VHD], VirtualizationException
+ FullyQualifiedErrorId : ObjectInUse,Microsoft.Vhd.PowerShell.Cmdlets.OptimizeVhd  

At this point my drive has almost no space and I'm not finding anything on Google on how to compact it, short of destroying the entire WSL 2 instance and remaking it from scratch.

I've stopped Docker Desktop and everything. There's no hint on what's using it.

[onomatopellan]

@nickjj With wsl.exe --shutdown should be enough to free all the mounted vhdx files.

[datocrats-org]

The second alternative mentioned is proving the fastest and most reliable for me, Exporting/Importing WSL 2 to move it to another disk. The other thing I have tried is converting between Fixed and Dynamic volumes in combination with this. My use is for docker's WSL2 integration, and the need to export and restore large data volumes (>250GB). This is similar to what most of the respondents are referring to, yet they do not always say whether they are using WSL1 vs. WSL2. WSL1 sets a fixed disk size as specified in your Docker settings. WSL2 uses windows default setting of 256GB max volume size. I could not bash into or ssh into the docker-data distro shown or access its ext4.vhdx volume through various methods. What would be really nice is if WSL would adopt a docker overlay2 driver or some other more friendly HyperV linux filesystem, faster than vhdx on ntfs, with an API to control allocation. Even better would be some type of cloud filesystem integration that could rival local dev speeds.

Here are my notes in gist

[jcoutch]

I just symlinked ~\AppData\Local\Docker\wsl\data to a folder on a different partition/drive, and moved the VHDX. While the problem with the VHDX growing is still there, at least it won't eat all the space on my primary drive. I wrote up a blog post about it, and here's the script I used (use at your own risk):

$ErrorActionPreference = "Stop"

$newLocation = "E:\VMs\WSL2\"

cd "~\AppData\Local\Docker\wsl\data"
wsl --shutdown
Optimize-VHD .\ext4.vhdx -Mode Full
mkdir $newLocation -Force
mv ext4.vhdx $newLocation
cd ..
rm "data"
New-Item -ItemType SymbolicLink -Path "data" -Target $newLocation

[onomatopellan]

@mikemaccana That's where Docker Desktop with WSL2 backend saves the docker images. Use the Troubleshoot option in the Dashboard to clean them.

[mikemaccana]

OK, ran:

To get rid of the docker images that had been accumulating. I have my hard drive back, and Docker is now pulling down images (good) but it's failing with:

ERROR: Service 'postgres' failed to build: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.65.1:53: read udp 192.168.65.3:39176->192.168.65.1:53: i/o timeout

Which I'm now troubleshooting.

Edit: since the IP addresses are local, poking around in Docker networking config and restarting docker fixed it.

[onomatopellan]

For further Docker Desktop issues is better to ask them in https://github.com/docker/for-win/issues

[AmineI]

@mikemaccana 150GB taken in Docker wsl/data vdhx

This issue is also logged here indeed docker/for-win#244 , go there for Docker-specific elements; as mentioned by onomatopellan above.

[hoaaah]

I have also tried every method proposed on various stackoverflow posts and GitHub issue posts. Is this bug specific to certain Windows versions? I have run the Optimize-VHD commandlet, diskpart, and wsl --set-sparse. Tried upgrading to later WSL 2 versions and that did not improve things either. Optimize-VHD reduces space by a couple gigabytes which is just enough to keep my machine from committing sudoku. Getting pretty sick of at this point.

I've never use Optimize-VHD or diskpart anymore, just use this #4699 (comment) solution and it worked tiredlessly. If you have good internet connection to pull image after performed this, you will be ok.

[nullwiz]

This is a terrible problem. Terrible. Optimize-VHD does nothing, resizing is not possible. Running "fstrim /" does not change things. It still wont reduce the disk size. Interestingly, the difference is almost of 200GB in the distro I had installed. Also tried wslcompact.

I did work with big files recently -- and there is no way to get that distro to a working state again. There is only 40gb in it.

This makes WSL unusable if you are going to be using it for almost anything nowadays. My image is from Ubuntu-20.04, not that long ago.

UPDATE: do

docker system prune -a 

and WAIT. Then re-do all of the zeroing. It will free disk space.

[NirmalManoj]

I have been wondering why my WSL is taking so much space despite deleting a lot of data. Just realized that such a significant issue is left unfixed for more than 4 years. Please fix this soon, @microsoft WSL Team.

[qwertynik]

Is there some dirty politics that is preventing this issue from being resolved? I remember @craigloewen-msft posting an update regarding changes done to solve this. However, from what I recollect I couldn't use those solutions - either because it required Windows Pro or Windows Upgrade. Was anyone else able to use the recent solution introduced by Microsoft to resolve this issue?

[igortas]

@qwertynik Nothing works. I think wsl2 automatic solution for removing space works only when u set it on fresh instance.

[qwertynik]

@igortas this seems extremely peculiar. WSL2 was a game changing (supposedly) feature in an attempt to make Windows more developer friendly. But the way WSL2 runs, it is not usable (in a safe way), for so many.

Which is why I posted this comment earlier.

For many this will soon be true

Time spent on this ticket > Time required to make money for buying a Mac + Getting accustomed to using Mac.

If not Mac, atleast Linux.

There is something very fundamental that we don't understand, or something that Microsoft cannot share about this issue.

[igortas]

@qwertynik the easiest way is to reinstall the wsl2 instances once in a while...

[qwertynik]

Yeah. That's a good idea @igortas. Easy to forget about such simple solutions when remaining deep into problems. But this has its own demands. One needs to remember the steps that were taken to configure the distro powered by WSL2. Or, rather one should completely rely on a docker like service alone when working with different distros - this atleast will make reinstallations easier. Because, when using docker it is easier to build the habit of having all config in code. While these options are great, the best would have been to run a command and free up the space that is not being utilised. There is something about virtualization technology that should be making this difficult to solve.

[prasanthcakewalk]

With the optimize-vhd cmdlet approach, first mounting the vhd in read-only mode before performing the optimization can apparently lead to a better compression. I noticed a small, but not insignificant, improvement by doing this (of course, YMMV). The code would be (in powershell, as admin, from the directory containing the vhd):

wsl --shutdown

mount-vhd -Path .\ext4.vhdx -ReadOnly
optimize-vhd -Path .\ext4.vhdx -Mode full
dismount-vhd -Path .\ext4.vhdx

The mount-vhd and dismount-vhd cmdlets also come with Hyper-V.

Sources:

1. Explanation of the different optimization modes: https://learn.microsoft.com/en-us/powershell/module/hyper-v/optimize-vhd?view=windowsserver2022-ps#-mode
2. The text under Example 1 here says that not pre-mounting the disk in read-only mode will change the optimization mode from full to prezeroed: https://learn.microsoft.com/en-us/powershell/module/hyper-v/optimize-vhd?view=windowsserver2022-ps#example-1
3. This article has all the information in this comment, plus a powershell function to run do the mounting, optimizing, and dismounting: https://www.altaro.com/hyper-v/compact-hyper-v-virtual-disks-vhdx/

[RandomDSdevel]

     I didn't think that Hyper-V and the (Hyper-V–based, but distinct,) Windows Virtual Machine Platform that WSL2 uses could be installed at the same time, though? 'Mount-Vhd' complains that Hyper-V isn't installed if you only have the Hyper-V PowerShell command module installed and try to run it.

[prasanthcakewalk]

I didn't think that Hyper-V and the (Hyper-V–based, but distinct,) Windows Virtual Machine Platform that WSL2 uses could be installed at the same time, though? 'Mount-Vhd' complains that Hyper-V isn't installed if you only have the Hyper-V PowerShell command module installed and try to run it.

@RandomDSdevel I had to turn on Hyper-V Services and Hyper-V Module for Windows PowerShell features to run those commands. Not sure if they could possibly conflict with Virtual Machine Platform or not, but I didn't experience a conflict on my machine.

[christinakotsa]

I have Windows 10 (Home) with WSL and I have tried both options suggested for Home here but either nothing is happening or I keep losing memory on my hard disk. After I have shut down the WSL and tried to run:
select vdisk file="C:\Users\chrik\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu_79rhkp1fndgsc\LocalState\ext4.vhdx"
compact vdisk

the message I got was:

DISKPART> detach vdisk

DiskPart has encountered an error: The requested operation could not be completed due to a limitation of the virtual disk system. Virtual hard disk files must not be compressed or encrypted and must not be fragmented.
See the System Event Log for more information. (It is translated from Greek).

The files I work with are mostly gzip files containing large biological data. Of course, I have either transferred them in USB sticks or deleted them, and yet no space is freed on my hard disk. Can I unzip them somehow or do something so that these commands can work? Any help could be extremely useful!

Thank you!

[RandomDSdevel]

@prasanthcakewalk:

@RandomDSdevel I had to turn on Hyper-V Services and Hyper-V Module for Windows PowerShell features to run those commands. Not sure if they could possibly conflict with Virtual Machine Platform or not, but I didn't experience a conflict on my machine.

     I can also confirm that having the Virtual Machine Runtime installed, then installing they Hyper-V Module for Windows PowerShell and Hyper-V Services, but not the Hyper-V Hypervisor, works.

---

     The sequence of commands you gave still doesn't work directly with VHDX image files that have NTFS compression applied to them, but it still helps streamline things as compared to having to use diskpart's shell and manually issuing commands there.

[tveyben]

Just like @prasanthcakewalk I was not able to use optimize-vhd until after installing Hyper-V.
@craigloewen-msft IMHO this tool should also have been installed when installing WSL2...

Anyways - I prefer a non GUI method og getting the tool (well Hyper-V) installed, so I use this PS cmd (in an elevated prompt):

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V-All

And yes - you DO have to perform the reboot to get the tool...!

My VHD then got reduced from 126 to 86 GB

Using diskpart->compact seems to perform the same action as performing that made no further reduction (that makes sense I think)

I am only using 48 GB

PS C:\WINDOWS\system32> wsl.exe --system -d Ubuntu-20.04 df -h /mnt/wslg/distro
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdc        251G   48G  191G  20% /mnt/wslg/distro

So I guess that now it's time to look into defragging the disk to reclaim space (and getting a bigger disk)...

Hmm
e4defrag /dev/sdc

Made no change - still 48 GB usage
so likely the files are not fragmentet, but data is not stored "contiguously" (space between the files)?...

[RandomDSdevel]

     Before support for sparse disk images got added and maybe still for non-sparse disk images, WSL would pin deleted files' 'inode's by creating folders starting with, IIRC, '.wsl,' then continuing with a (pseudo-)random hash in the internally-mounted disk image filesystem root. That's something else to check for. Converting a WSL (2) distribution's backing virtual hard disk image to sparse doesn't remove these folders. The Virtual Machine Platform won't create any more of these folders inside sparse disks, however.

[mateuszfl]

@christinakotsa

I have Windows 10 (Home) with WSL and I have tried both options suggested for Home here but either nothing is happening or I keep losing memory on my hard disk. After I have shut down the WSL and tried to run: select vdisk file="C:\Users\chrik\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu_79rhkp1fndgsc\LocalState\ext4.vhdx" compact vdisk

the message I got was:

DISKPART> detach vdisk

DiskPart has encountered an error: The requested operation could not be completed due to a limitation of the virtual disk system. Virtual hard disk files must not be compressed or encrypted and must not be fragmented. See the System Event Log for more information. (It is translated from Greek).

The files I work with are mostly gzip files containing large biological data. Of course, I have either transferred them in USB sticks or deleted them, and yet no space is freed on my hard disk. Can I unzip them somehow or do something so that these commands can work? Any help could be extremely useful!

Thank you!

I have the same problem, did you find a solution?

[astroboylrx]

I've done the optimize-vhd -Path .\ext4.vhdx -Mode full before and everything was fine.
But today when I did it, WSL2 failed to start. All I got is

Failed to attach disk 'C:\Users\xxxxx\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu22.04LTS_79rhkp1fndgsc\LocalState\ext4.vhdx' to WSL2: The file or directory is corrupted and unreadable.
Error code: Wsl/Service/CreateInstance/MountVhd/HCS/0x80070570

Googling doesn't help too much. Would it be possible to repair it?

[karolineWss]

Olá, tenho um Windows 11 pro e WSL2 e consegui resolver meu problema da seguinte forma.

1° passo: Se você não está com o Hyper-V ativado, ative-o da seguinte forma:

Painel de controle > Programas > Ativar ou desativar recursos do Windows > Selecione a caixa do “Hyper-V” > Click em “Ok” > Reinicie o computador.

2° passo: Se seu VHD já está com espaço em disco alto e você programou para espaçar automaticamente com wsl --manage <distro> -s true antes de compactar, programe para wsl --manage <distro> -s false pois esse comando não faz com que seu armazenamento livre retorne para o disco (C:) e ao tentar compactar com ele ativado em “true” gera erro no optimize-VHD.

Obs: Caso você ainda não tenha programado para para espaçar automaticamente, pule o passo 2.

3° passo:
3.1 - Desligue o wsl pelo powershell como administrador com o comando: wsl --shutdown

3.2 - Identifique o arquivo de disco virtual: O arquivo de disco virtual geralmente está localizado em: C:\Users\<seu_nome_de-usuário>\AppData\Local\Packages<distro_package_name>\LocalState\ext4.vhdx

Obs 3.2: Substitua <your_username> e <distro_package_name> pelos valores apropriados.

Obs 3.2: Certifique-se de NÃO clicar no arquivo “ext4.vhdx” para não gerar erro que o arquivo esteja sendo usado em outra instância, certifique-se de que nada relacionado ao wsl esteja aberto.

3.3 Para Windows Pro ou superior: • Use o Optimize-VHD: > Abra o PowerShell como administrador > Navegue até o diretório que contém o arquivo “ext4.vhdx” com: cd C:\Users\<seu_nome_de-usuário>\AppData\Local\Packages\‹distro_package_name>\LocalState\

3.4 Execute o comando: optimize-vhd -Path . \ext4.vhdx -Mode full

4° passo: ative o espaçamento automático com wsl --manage <distro> -s true

Pronto, seu VHD foi compactado para o tanto de espaço de fato utilizado, e o armazenamento espaçado voltou para o seu disco rígido C:

Funcionou para mim, eu estava com 105G no VHD sendo que de fato eu só estava usando 19G no meu \home. Isso levou ele para 20G e meu SSD (C:) que estava só com 68GB livre, voltou para 152GB livre.

[christinakotsa]

@christinakotsa

I have Windows 10 (Home) with WSL and I have tried both options suggested for Home here but either nothing is happening or I keep losing memory on my hard disk. After I have shut down the WSL and tried to run: select vdisk file="C:\Users\chrik\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu_79rhkp1fndgsc\LocalState\ext4.vhdx" compact vdisk
the message I got was:
DISKPART> detach vdisk
DiskPart has encountered an error: The requested operation could not be completed due to a limitation of the virtual disk system. Virtual hard disk files must not be compressed or encrypted and must not be fragmented. See the System Event Log for more information. (It is translated from Greek).
The files I work with are mostly gzip files containing large biological data. Of course, I have either transferred them in USB sticks or deleted them, and yet no space is freed on my hard disk. Can I unzip them somehow or do something so that these commands can work? Any help could be extremely useful!
Thank you!

I have the same problem, did you find a solution?

I've transitioned from utilizing a virtual machine to implementing a dual-boot setup on my PC, i.e. I now have both Windows and Linux operating systems installed separately.

[LucasHang]

I have Windows 10 (Home) with WSL and I have tried both options suggested for Home here but either nothing is happening or I keep losing memory on my hard disk. After I have shut down the WSL and tried to run: select vdisk file="C:\Users\chrik\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu_79rhkp1fndgsc\LocalState\ext4.vhdx" compact vdisk

the message I got was:

DISKPART> detach vdisk

DiskPart has encountered an error: The requested operation could not be completed due to a limitation of the virtual disk system. Virtual hard disk files must not be compressed or encrypted and must not be fragmented. See the System Event Log for more information. (It is translated from Greek).

The files I work with are mostly gzip files containing large biological data. Of course, I have either transferred them in USB sticks or deleted them, and yet no space is freed on my hard disk. Can I unzip them somehow or do something so that these commands can work? Any help could be extremely useful!

Thank you!

For me, this error was caused because of the "sparse" configuration of wsl set to true. Then I ran wsl --manage ubuntu --set-sparse false and then the process with diskpart worked out.

[Twenkid]

For me, this error was caused because of the "sparse" configuration of wsl set to true. Then I ran wsl --manage ubuntu --set-sparse false and then the process with diskpart worked out.

Thanks, that resolved it for me too!

[twilwa]

So, i'm successfully able to perform the compacting operation -- but it seems the solution is only temporary, as the disk re-expands shortly after I begin interacting in WSL again. There a way to set a limit on the size of the virtual disk that anyone's figured out?

[berlincount]

So, i'm successfully able to perform the compacting operation -- but it seems the solution is only temporary, as the disk re-expands shortly after I begin interacting in WSL again. There a way to set a limit on the size of the virtual disk that anyone's figured out?

well, yes and no.

three tips beforehand: calling fstrim -v / before doing the compaction sometimes makes a notable difference; and if you're using something like that doing docker system prune -a even before that even more. re-setting sparse mode by doing wsl --manage ubuntu --set-sparse true (or equivalent) after compacting is also highly recommended.

so, the long answer:

someone made the maybe questionable decision of both giving the virtual harddisk inside the VHD the size of a full terabyte as well as giving the filesystem partition of that virtual harddisk the full size. so, the limitation is basically a terabyte, and the virtual harddisk file will never grow beyond that - much. some metadata is required, and some space is used for (thin provisioning) housekeeping, but essentially, when all space is used, its size is 1TB plus some headers/wrappers, same as with thick provisioning.

if you use diskpart/Resize-VHD, you can grow that volume easily - afterwards, you'll have to grow the partition with parted or similar, and resize the filesystem to fit with resize2fs, and you'll have a bigger disk.

getting to the 'no' for reducing/limiting:

getting to a smaller disk is similar, but unreasonably harder. you'll have to resize2fs -M to reduce the size of the filesystem, then parted to reduce the size of the partition, and then diskpart/Resize-VHD the reduce the size of the virtual harddisk.

the tricky bit is getting the reduced size right, otherwise you'll lose data. making the resized partition somewhat bigger than the resized filesystem should give you some marging, same as making the resized VHD somewhat bigger than the resized partition. you can grow partition and filesystem after to fit the space after the VHD resize to reclaim whatever safety marking you did.

the hard bit is that resize2fs -M is an offline operation, and can not be done with the virtual machine running WSL (unlike growing, which can be online). you'll have to attach the VHD to another virtual machine and do the operation there.

it'll work, but .. the fact that so far I couldn't be bothered myself and instead relocated the VHD to a volume where I have more diskspace (and can afford the 1TB used) should tell you how annoying it is.

I hope this explanation provides some additional information for a few people, maybe you'll get to further ideas on how to improve the situation :)

[robinloxley1]

Based on @karolineWss with a bit translation, and previous people's contributions, summarize what worked for my machine (Windows 11 Pro). Reduced from 300+GB to 78 GB in my case.

WSL version: 2.2.4.0
Kernel version: 5.15.153.1-2
WSLg version: 1.0.61
MSRDC version: 1.2.5326
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26091.1-240325-1447.ge-release
Windows version: 10.0.22631.3593

---

Solution with optimize-vhd

Step 0: Backup wsl vhdx if you are not confident

wsl.exe --export Ubuntu D:\exported.tar

Step 1: enable hyper-v features in Windows Professional

1. Go to Control Panel | Programs and features | Turn windows features on or off
2. Tick Hyper-V > Hyper-V Management tools > Hyper-V Module for Windows PowerShell
3. Tick Hyper-V Platform > Hyper-V Services
4. When installed, reboot since asked

Step 2: Run following as admin

wsl.exe --shutdown  # ensure wsl is turned off
wsl.exe -l -v  # confirm its status as stopped
wsl --manage ubuntu --set-sparse false  # ensure false before operation
cd 'C:\Users\<UserName>\AppData\Local\Packages<distro_package_name>\LocalState\ext4.vhdx'  # where the huge ext4.vhdx is.

optimize-vhd -Path .\ext4.vhdx -Mode Full [-Confirm -Whatif]  # remove [...] if you are sure.

# if everything goes well, you shall see the original ext4.vhdx is reduced.
wsl --manage Ubuntu --set-sparse true  # turn on sparse mode from now on

wsl  # login wsl to verify, hoping no corruption

[BigSamu]

Do you have any updates for resolving this issue? I was using the WSL Ubuntu-22.04 Distribution and tried everything you said. At last, I was able to recover just 10 GB from the 380 GB used. However, I was unable to recover more memory. This is so frustrating.

Ultimately, I have to run the command wsl.exe --unregister Ubuntu-22.04, but WARNING!! This deletes everything. I recovered all the space with that, but this was not the correct solution. Now I need to install from scratch everything in my WSL2 (I was just using it for programming some projects I have backed up in Github)

[polarathene]

Do you have any updates for resolving this issue? I was using the WSL Ubuntu-22.04 Distribution and tried everything you said.

I have had no issues, although I updated WSL2 to the beta version with the sparse disk support (caution, downgrade/rollback doesn't seem to be supported well), which I think is now out as a stable release or will be soon?

I used wslcompact to minimize my disks, as that worked well for me.

• If you run into a memory issue, you'd need to make an adjustment.
• Although for Docker I typically go into the Docker Desktop GUI settings and purge the whole disk (faster than wslcompact and provides a clean slate).

Here are the powershell terminal commands I use:

wslcompact -c docker-desktop-data
wslcompact -c Ubuntu

Then I set the disks to be sparse:

wsl.exe --manage docker-desktop-data --set-sparse true
wsl.exe --manage Ubuntu --set-sparse true

Now when I delete stuff from either vdisk (eg: for Docker it's docker image rm image-name or docker system df prune) the storage is reclaimed 👍 (NOTE: Docker still accumulates some storage somewhere, thus you'd probably need to use the GUI method to reset that storage once in a while, requiring you to make the disk sparse again)

---

References of my earlier comments that are now buried (they are all very verbose but probably contain useful insights for most):

• WSL 2 should automatically release disk space back to the host OS #4699 (comment)
• WSL 2 should automatically release disk space back to the host OS #4699 (comment)
• WSL 2 should automatically release disk space back to the host OS #4699 (comment)
• WSL 2 should automatically release disk space back to the host OS #4699 (comment)
• WSL 2 should automatically release disk space back to the host OS #4699 (comment)
• WSL 2 should automatically release disk space back to the host OS #4699 (comment)

---

However, I was unable to recover more memory. This is so frustrating.

I also documented this above in those reference links, but this is an issue with memory reclaim.

• WSL2 writes to disk based on memory usage too, even if it's not something that'd use swap or page file, like reading a 4GB file, or a container that builds a project and needs several GB RAM to do that.
• That memory won't appear to be allocated in the WSL2 instance itself (free -m), except as buffer/cache which is free to be used for actual memory allocation. However on the host it will be reported as allocated via Task Manager and you'll find additional disk space used.

Eventually over time WSL2 would release that memory on the host side, otherwise you need to force it. In a a WSL2 terminal like for Ubuntu, run sudo sysctl vm.drop_caches=3, provide your password to authorize and it'll clear that buffer/cache from free -m. You'll then notice via Task Manager that the memory usage starts dropping. This is much slower without that command.

If that still doesn't do it then you may need to shutdown WSL with wsl shutdown in powershell terminal. If you ran out of disk space that may not work, and you'd have to restart Windows AFAIK.

Good news is there is settings to opt-in for better memory reclaim, although I think it had some caveats especially with Docker so I didn't bother with it. Not sure if that got fixed since, and if the update would have that enabled by default or not.

---

Now I need to install from scratch everything in my WSL2

Well now you know how to do it properly 👍

If it helps this is what I am running:

wsl --version

WSL version: 2.0.0.0
Kernel version: 5.15.123.1-1
WSLg version: 1.0.57

That is the beta/pre-release WSL2 2.0 release IIRC, if there is a new stable release out perhaps give that a try too.

[berlincount]

So, i'm successfully able to perform the compacting operation -- but it seems the solution is only temporary, as the disk re-expands shortly after I begin interacting in WSL again. There a way to set a limit on the size of the virtual disk that anyone's figured out?

well, yes and no.

I ended up trying it, and was successful in resizing & size limiting my Ubuntu:

• used New-VHD to create a new dynamic vhdx file with a desired maximum size
• spun up a Debian as toolbox system (you might need to install some packages for some tools)
• used lsblk and wsl bare mounts to add existing and new vhdx files as block devices to Debian
• used resize2fs -M /dev/sdc (in my case) to resize the filesystem to the smallest possible size
• used dd bs=4k if=/dev/sdc of=/dev/sdd conv=sparse to copy the filesystem from one block device to other (this copies past the end of the input filesystem, but not past the end of the target device, and leaves unused blocks unallocated. you could set count=XXX to only read the blocks the filesystem actually used, but .. I just let it run over night)
• used resize2fs /dev/sdd to resize filesystem to volume size
• shut down (and deleted) Debian system
• exchanged the old Ubuntu ext4.vhdx with the new vhdx file
• reenabled the sparse flag for the Ubuntu root fs
• tested, and the root filesystem of my Ubuntu now had all my file, and the smaller (max) volume size instead of 1TB

so now the rootfs container on the host system can't grow much (metadata!) beyond the size given, and my Ubuntu can't run into a stall due to host file system full.

[polarathene]

my Ubuntu can't run into a stall due to host file system full.

Have you verified that? Even if your Ubuntu has free space, if the host disk is full (0 bytes left), then it would be the same problem with no extra space to allocate. Reducing the vdisk max size only limits the disk growing too large and wasting disk space on the host, but sparse vdisk makes that mostly unnecessary.

If I am low on disk space and use WSL2 to build a project that uses a few GB of RAM, this memory usage would also use similar amount of disk space on the host. That risks running out of host disk space and WSL2 becomes unresponsive, not even wsl --shutdown would work from the host.

As reclaiming disk space on the host does not recover from that issue, you have to reboot the host AFAIK, it's a major drawback vs VMware guest.

[BigSamu]

Do you have any updates for resolving this issue? I was using the WSL Ubuntu-22.04 Distribution and tried everything you said.

I have had no issues, although I updated WSL2 to the beta version with the sparse disk support (caution, downgrade/rollback doesn't seem to be supported well), which I think is now out as a stable release or will be soon?

I used wslcompact to minimize my disks, as that worked well for me.

• If you run into a memory issue, you'd need to make an adjustment.
• Although for Docker I typically go into the Docker Desktop GUI settings and purge the whole disk (faster than wslcompact and provides a clean slate).

Here are the powershell terminal commands I use:

wslcompact -c docker-desktop-data
wslcompact -c Ubuntu

Then I set the disks to be sparse:

wsl.exe --manage docker-desktop-data --set-sparse true
wsl.exe --manage Ubuntu --set-sparse true

Now when I delete stuff from either vdisk (eg: for Docker it's docker image rm image-name or docker system df prune) the storage is reclaimed 👍 (NOTE: Docker still accumulates some storage somewhere, thus you'd probably need to use the GUI method to reset that storage once in a while, requiring you to make the disk sparse again)

References of my earlier comments that are now buried (they are all very verbose but probably contain useful insights for most):

• WSL 2 should automatically release disk space back to the host OS #4699 (comment)
• WSL 2 should automatically release disk space back to the host OS #4699 (comment)
• WSL 2 should automatically release disk space back to the host OS #4699 (comment)
• WSL 2 should automatically release disk space back to the host OS #4699 (comment)
• WSL 2 should automatically release disk space back to the host OS #4699 (comment)
• WSL 2 should automatically release disk space back to the host OS #4699 (comment)

However, I was unable to recover more memory. This is so frustrating.

I also documented this above in those reference links, but this is an issue with memory reclaim.

• WSL2 writes to disk based on memory usage too, even if it's not something that'd use swap or page file, like reading a 4GB file, or a container that builds a project and needs several GB RAM to do that.
• That memory won't appear to be allocated in the WSL2 instance itself (free -m), except as buffer/cache which is free to be used for actual memory allocation. However on the host it will be reported as allocated via Task Manager and you'll find additional disk space used.

Eventually over time WSL2 would release that memory on the host side, otherwise you need to force it. In a a WSL2 terminal like for Ubuntu, run sudo sysctl vm.drop_caches=3, provide your password to authorize and it'll clear that buffer/cache from free -m. You'll then notice via Task Manager that the memory usage starts dropping. This is much slower without that command.

If that still doesn't do it then you may need to shutdown WSL with wsl shutdown in powershell terminal. If you ran out of disk space that may not work, and you'd have to restart Windows AFAIK.

Good news is there is settings to opt-in for better memory reclaim, although I think it had some caveats especially with Docker so I didn't bother with it. Not sure if that got fixed since, and if the update would have that enabled by default or not.

Now I need to install from scratch everything in my WSL2

Well now you know how to do it properly 👍

If it helps this is what I am running:

wsl --version

WSL version: 2.0.0.0
Kernel version: 5.15.123.1-1
WSLg version: 1.0.57

That is the beta/pre-release WSL2 2.0 release IIRC, if there is a new stable release out perhaps give that a try too.

@polarathene, many thanks for your clarification. Pretty clear!!! Appreciate that!