Common HttpClient tasks you're googling or asking ChatGPT

Using HttpClient has become my default choice when making HTTP requests from .NET. While there are many good alternatives out there (like Refit, Flurl, and RestSharp), having a built-in client that does most tasks in a pretty easy and straightforward way, makes this an obvious choice for many. While the API is simple, I often find myself Googling, asking ChatGPT, or looking through my existing code for common tasks. In this post, I have collected a list of tasks that at least I need from time to time. But I hope that it will help everyone else too.

Before digging down into the list, I want to put a few words around the lifecycle of HttpClient. Most people know by now that you should not "new up" HttpClients for every HTTP request you make. Hundreds of posts have been written about the subject, so I don't want to dig further into why. Just remember that my examples create a new HttpClient every time, but in reality you shouldn't do that. Either share HttpClient as a singleton or use HttpClientFactory.

Set a `User-Agent` header

It is good practice to always set a User-Agent header when making HTTP requests. Some APIs even demand that to successfully process the request. While the obvious approach would be to add a custom header, there's a built-in way to parse and add a User-Agent:

var client = new HttpClient();

var request = new HttpRequestMessage(HttpMethod.Get, "https://example.com");
request.Headers.UserAgent.ParseAdd("MyApp/1.0");

var response = await client.SendAsync(request);

This will include the User-Agent MyApp/1.0 to the request. You can also set the User-Agent as a header on all requests made with the HttpClient, instance:

var client = new HttpClient();
client.DefaultRequestHeaders.UserAgent.ParseAdd("MyApp/1.0");

// All requests made with this instance will now include the User-Agent
var response = await client.GetAsync("https://example.com");

Automatically decompress GZip and Deflate responses

In case the API or endpoint you need to call is doing response compression, you will need to enable decompression. It can come as a small surprise, so here's the code:

var handler = new HttpClientHandler
{
    AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};

var client = new HttpClient(handler);

Having to specify this manually is only required if you are creating your HttpClient manually. When using HttpClientFactory in ASP.NET Core, AutomaticDecompressionand is set by default, starting from .NET Core 2.1.

Disable SSL certificate validation (not recommended for production)

I get it. You shouldn't disable SSL certificate validation when requesting remote endpoints. The web runs on HTTPS now, so an SSL error should normally result in an error. You may run into a situation where you need to request an old API that never moved to HTTPS, and disabling certificate validation can be a lifesaver:

var handler = new HttpClientHandler
{
    ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
};

var client = new HttpClient(handler);

Send and receive strongly typed payloads

I don't know why I keep forgetting these methods. Maybe because they were added in a later version of .NET. But HttpClient supports strongly typed objects when both writing and reading data:

var client = new HttpClient();

var user = new User
{
    Firstname = "Thomas",
    Lastname = "Ardal"
}

// Send strongly typed object
var response = await client.PostAsJsonAsync("https://example.com", user);

// Receive strongly typed object
user = await client.GetFromJsonAsync<User>("https://example.com");

Using the PostAsJsonAsync and GetFromJsonAsync methods, HttpClient will automatically do the JSON serialization and deserialization. You can use the normal methods that accepts strings if you want to manually convert an object to JSON and vice versa.

Use Polly to add retry logic

We often think in successful scenarios only when developing code. The same goes for making HTTP requests. But, as with everything else, an API may be temporarily down or in other ways in a degraded state. Adding a bit of retry magic is easy using the excellent Polly package:

var policy = Policy
    .Handle<HttpRequestException>()
    .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(retryAttempt));

var client = new HttpClient();

await policy.ExecuteAsync(() => client.GetAsync("https://example.com"));

In the example above, I define a policy with Polly, retrying the action requested with this policy up to three times with a pause in between retries.

When using the AddHttpClient method in ASP.NET Core projects and similar, you can even use the built-in support for Polly by installing the following package:

dotnet add package Microsoft.Extensions.Http.Polly

Then, setting a custom policy when creating the HttpClient in Program.cs:

builder.Services.AddHttpClient("MyClient")
    .AddPolicyHandler(Policy
        .Handle<HttpRequestException>()
        .WaitAndRetryAsync(3, retryAttempt =>
            TimeSpan.FromSeconds(retryAttempt)));

Cancel a request with `CancellationToken`

Like most async methods, the ones available on HttpClient also accept a CancellationToken. This can be useful in situations where you manually want to cancel the request or hook the request on a CancellationToken provided for your code (like in Azure Functions):

using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));

try
{
    var response = await client.GetAsync("https://example.com", cts.Token);
}
catch (TaskCanceledException)
{
    Console.WriteLine("Request was canceled.");
}

This will cancel the request after 5 seconds. This is like setting a timeout, but it works a bit differently. You can re-use an existing CancellationToken. You have control of the scope of the cancellation and you can call cts.Cancel() manually if you need to cancel the request rather than waiting for it to cancel after the configured timeout.

Add Bearer Token Authentication

Another task that I forget the syntax for is setting a bearer token. This is useful in situations where a tool generates a developer token or similar which needs to be on the request. This may be obvious to everyone else, but if you are as old as me and have worked with the previous HTTP clients in .NET for years, here's a quick demo of how simple it is with HttpClient:

client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);

That's it. These are some of the many use cases of HttpClient that I forget in between uses. I'll try to extend this blog post next time my brain is garbage-collected.

Set a User-Agent header