Powershell Invoke-WebRequest with Method Head allocate too much RAM

Question

Problem found on server with Powershell v 4.0 (server with Powershell 5.1 they do not seem affected by the same problem)

This a strange behavior of Invoke-WebRequest command.

Take this example:

For some reason, I need to know the file size before download it on my server.

Then I use the following commands:

$WebClient = Invoke-WebRequest -Uri $element -Method Head -Credential $Cred
$filesize = $webClient.Headers.'Content-Length'

The problem is before send $WebClient request Powershell use (for Commit RAM) about 120MB, but after the request powershell enlarge your RAM a the file $filesize value (example remote file is 800MB, new Commit RAM is 920MB).

When you work with Powershell WinRM session limited at 1GB of RAM this can be a problem. (OOM)

Answer 1

You can use Invoke-WebRequest with the HEAD method to get just the headers and not download anything. If the resource you're requesting has a known length, then you'll get a Content-Length header which you can use:

(Invoke-WebRequest $url -Method Head).Headers.'Content-Length'

Just note that not all servers return the Content-Length header for all requests. In that case you will need to use your above method which reads the entire file, although it's slower and wasteful of memory.

You can also interface directly to the Windows DLLs that do Internet requests, either WinINet (not available on Windows Server), or using Windows HTTP Services.