Multipart/form-data Support for Invoke-WebRequest and Invoke-RestMethod in PowerShell Core

Pictured: A packet capture of a current build of PowerShell Core submitting a multipart/form-data POST request from Invoke-WebRequest.

Intro

Over the past few months I have been donating a generous portion of my spare time to help improve the Web Cmdlets (Invoke-WebRequest and Invoke-RestMethod) in PowerShell Core. This is partly because I want and need certain functionality for both personal and work related projects. It is also because I have had some minor gripes about these Cmdlets for some time.

One common ask I have seen repeated in just about every PowerShell forum is multipart/form-data support. It seems like a reasonable thing to ask when there are many endpoints that will only work with a multipart/form-data submission. There is an open issue (#2112) on the PowerShell GitHub echoing the same request. It was brought to my attention and I decided to give it a serious look.

The result is that PowerShell Core now has partial multipart/form-data support in both Web Cmdlets. This change didn't make the cut for 6.0.0-beta.7 but it will be available starting in 6.0.0-beta.8 and is available now if you build it manually or grab the latest nightly build.

This blog will cover some of the challenges involved in supporting multipart/form-data, how to make use of this new feature, and about future plans for additional support.

Because typing multipart/form-data is annoying, I will be shortening it to just multipart. Please don't let this be mistaken for other multipart submission methods.

Also, I will be referring collectively to Invoke-WebRequest and Invoke-RestMethod as Web Cmdlets. In this case, there is no need to call out each command as they offer the same base functionality for multipart support.

What Took You So Long?

I had the same question when this first came up. It seems like a simple thing to implement. .NET (in both FullCLR and CoreCLR flavors) has multipart support. It wasn't until I started looking at the various requests for this that I noticed people were asking for seriously different things that all fall under the banner of multipart support. It turns out that multipart is very flexible and there is so much you can do with it. If you focus on a single multipart task, then it is easy to see it as silly that it was not included. When you pull back and look at the full scope of multipart it becomes clearer as to why it was likely not included.

I can't speak for the Microsoft PowerShell Team, but my suspicion is that they wanted to avoid implementing a complex feature that has a small subset of users who desire it for very different reasons. It probably just didn't make economical sense to focus time and resources on it. This is the beauty of Open Source and partly why I'm very glad they decided to bring PowerShell to the Open Source community: hobbyists, like myself, who want a feature bad enough will code it.

Great, Now How Do I Use it?

I know, you all don't like my blabbing on. You came here for code, and I respect that. But before we dive in, please indulge me for just a moment.

Keep in mind that this is the starting point. There is still a bunch of manual work involved. But, for users who will have complex needs for multipart support, this will likely be the only way. As stated, multipart is very flexible and it's not possible to extend the current Web Cmdlets without overcomplicating them for mainstream uses.

Partial multipart support has been implemented by extending the -Body parameter to accept and parse a MultipartFormDataContent object. This means that you need to make a bunch of .NET object calls to create the MultipartFormDataContent object and then submit that object to -Body.

String Content

Here is an example adopted from the pester tests for the Web Cmdlets:

$multipartContent = [System.Net.Http.MultipartFormDataContent]::new()

$stringHeader = [System.Net.Http.Headers.ContentDispositionHeaderValue]::new("form-data")

$stringHeader.Name = "TestString"

$StringContent = [System.Net.Http.StringContent]::new("TestValue")

$StringContent.Headers.ContentDisposition = $stringHeader

$multipartContent.Add($stringContent)

Invoke-WebRequest -Uri $uri -Body $multipartContent -Method 'POST'

That will create a multipart request with single field named TestString with the value of TestValue.

File Content

Here is another example adapted from the pester tests:

$multipartContent = [System.Net.Http.MultipartFormDataContent]::new()

$multipartFile = 'C:\path\to\file.txt'

$FileStream = [System.IO.FileStream]::new($multipartFile, [System.IO.FileMode]::Open)

$fileHeader = [System.Net.Http.Headers.ContentDispositionHeaderValue]::new("form-data")

$fileHeader.Name = "TestFile"

$fileHeader.FileName = 'file.txt'

$fileContent = [System.Net.Http.StreamContent]::new($FileStream)

$fileContent.Headers.ContentDisposition = $fileHeader

$fileContent.Headers.ContentType = [System.Net.Http.Headers.MediaTypeHeaderValue]::Parse("text/plain")

$multipartContent.Add($fileContent)

Invoke-WebRequest -Uri $uri -Body $multipartContent -Method 'POST'

This example submits the C:\path\to\file.txt file as the form field name TestFile as text/plain and with the supplied filename of file.txt.

Mixed Content

This example is for submitting a profile update that includes a first name, last name, profile picture, and auto play midi (for that geocities nostalgia).

$multipartContent = [System.Net.Http.MultipartFormDataContent]::new()

$stringHeader = [System.Net.Http.Headers.ContentDispositionHeaderValue]::new("form-data")

$stringHeader.Name = "GivenName"

$StringContent = [System.Net.Http.StringContent]::new("Mark")

$StringContent.Headers.ContentDisposition = $stringHeader

$multipartContent.Add($stringContent)

$stringHeader = [System.Net.Http.Headers.ContentDispositionHeaderValue]::new("form-data")

$stringHeader.Name = "Surname"

$StringContent = [System.Net.Http.StringContent]::new("Kraus")

$StringContent.Headers.ContentDisposition = $stringHeader

$multipartContent.Add($stringContent)

$multipartFile = 'C:\pics\profile.png'

$FileStream = [System.IO.FileStream]::new($multipartFile, [System.IO.FileMode]::Open)

$fileHeader = [System.Net.Http.Headers.ContentDispositionHeaderValue]::new("form-data")

$fileHeader.Name = "ProfilePic"

$fileHeader.FileName = 'profile.png'

$fileContent = [System.Net.Http.StreamContent]::new($FileStream)

$fileContent.Headers.ContentDisposition = $fileHeader

$fileContent.Headers.ContentType = [System.Net.Http.Headers.MediaTypeHeaderValue]::Parse("image/png")

$multipartContent.Add($fileContent)

$multipartFile = 'C:\music\LinkinPark_CrawlingInMySkin.midi'

$FileStream = [System.IO.FileStream]::new($multipartFile, [System.IO.FileMode]::Open)

$fileHeader = [System.Net.Http.Headers.ContentDispositionHeaderValue]::new("form-data")

$fileHeader.Name = "BackGroundMusic"

$fileHeader.FileName = 'LinkinPark_CrawlingInMySkin.midi'

$fileContent = [System.Net.Http.StreamContent]::new($FileStream)

$fileContent.Headers.ContentDisposition = $fileHeader

$fileContent.Headers.ContentType = [System.Net.Http.Headers.MediaTypeHeaderValue]::Parse("audio/midi")

$multipartContent.Add($fileContent)

Invoke-WebRequest -Uri $uri -Body $multipartContent -Method 'POST'

I think this illustrates some of the complexity of implementation. Notice that each file requires its own media type header.

There Has To Be A Better Way!

I hope there will be. In other words, I am still working on this but I can't promise anything solid. Also, any work I do will be up to the PowerShell maintainers to pull or not. So take what I say here as my own personal desires that could possibly end up becoming part of PowerShell Core and nothing more.

What I want to do is to extend the -InFile and -Body parameters to support multipart. This will probably work with an -AsMultipart switch or something similar. The idea is to supply a dictionary to -Body (similar to how a body dictionary can be supplied today for from submissions), a single file for -InFle, and the file's media type supplied to -ContentType. I call this a limited simplified support implementation. It will allow a single file and string fields to be submitted simultaneously, but wont support multiple files. There are some minor logic issues I have to overcome first.

Also, I am working on a DSL for creating MultipartFormDataContent objects. It's not anywhere near ready to share, unfortunately. This would be a separate module available on the PowerShell Gallery. The goal would be to make it a bit easier for those who will need to use more complex mutlipart scenarios and as a fallback incase I can't get the changes I want implemented.

Word of Caution

One thing to be cautious of: when you submit a MultipartFormDataContent object to -Body anything supplied -ContentType will be ignored. multipart/form-data is its own content type and it includes a boundary. All of this is controlled by the MultipartFormDataContent object itself and would clash with any other Content-Type headers.

Conclusion

If you are interested in following my PowerShell Core work you can watch my GitHub repo. if you are interested in seeing what else I have contributed to PowerShell core, you can look at my merge archive repo. I have a few other things in motion for the Web Cmdlets.One of which is an ongoing project to move the pester tests away from their reliance on httpbing.org which continually results in false test fails.

Join the conversation on Reddit!