Scott Hanselman

Parallel.ForEachAsync in .NET 6

October 21, 2021 Comment on this post [6] Posted in DotNetCore | Open Source
Sponsored By

Great tweet from Oleg Kyrylchuk (follow him!) showing how cool Parallel.ForEachAsync is in .NET 6. It's new! Let's look at this clean bit of code in .NET 6 that calls the public GitHub API and retrieves n number of names and bios, given a list of GitHub users:

using System.Net.Http.Headers;
using System.Net.Http.Json;

var userHandlers = new []
{
"users/okyrylchuk",
"users/shanselman",
"users/jaredpar",
"users/davidfowl"
};

using HttpClient client = new()
{
BaseAddress = new Uri("https://api.github.com"),
};
client.DefaultRequestHeaders.UserAgent.Add(new ProductInfoHeaderValue("DotNet", "6"));

ParallelOptions parallelOptions = new()
{
MaxDegreeOfParallelism = 3
};

await Parallel.ForEachAsync(userHandlers, parallelOptions, async (uri, token) =>
{
var user = await client.GetFromJsonAsync<GitHubUser>(uri, token);

Console.WriteLine($"Name: {user.Name}\nBio: {user.Bio}\n");
});

public class GitHubUser
{
public string Name { get; set; }
public string Bio { get; set; }
}

Let's note a few things in this sample Oleg shared. First, there's no Main() as that's not required (but you can have it if you want).

We also see just two usings, bringing other namespaces into scope. Here's what it would look like with explicit namespaces:

using System;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Threading.Tasks;

We've got an array of users to look up in userHandlers. We prep an HttpClient and setup some ParallelOptions, giving our future ForEach the OK to "fan out" to up to three degrees of parallelism - that's the max number of concurrent tasks we will enable in one call. If it's -1 there is no limit to the number of concurrently running operations.

The really good stuff is here. Tight and clean:

await Parallel.ForEachAsync(userHandlers, parallelOptions, async (uri, token) =>
{
var user = await client.GetFromJsonAsync<GitHubUser>(uri, token);

Console.WriteLine($"Name: {user.Name}\nBio: {user.Bio}");
});

"Take this array and naively fan out into parallel tasks and make a bunch of HTTP calls. You'll be getting JSON back that is shaped like the GitHubUser."

We could make it even syntactically shorter if we used a record vs a class with this syntax:

public record GitHubUser (string Name, string Bio);

This makes "naïve" parallelism really easy. By naïve we mean "without inter-dependencies." If you want to do something and you need to "fan out" this is super easy and clean.


Sponsor: Make login Auth0’s problem. Not yours. Provide the convenient login features your customers want, like social login, multi-factor authentication, single sign-on, passwordless, and more. Get started for free.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
October 26, 2021 19:59
What's the current idiomatic approach for "using" with "HttpClient"? My last encounter (.NET 4.8) it was a bit of a no-no.
October 27, 2021 2:03
The advice I’ve always read in the past has been to only use Parallel.ForEach for cpu intensive operations, and use async with Task.WaitAll for performing non-cpu intensive operations in parallel (like I/O bound operations as you show in this example). So now that we have this hybrid Parallel.ForEachAsync, when should we use it? Always? Any guidance on this would be appreciated. Thanks.
October 27, 2021 10:47
Well, thanks for explaining. But I'm very, very stupid at this sort of thing. Whatever I do, I can't get it done. I'm already happy that I can start up the Internet and even email. Can you please send me the download link for the Video "Bye" in my secondary Email? That would be great! https://mail.google.com/mail/u/1/#inbox
This is the Video I mean: https://www.vpro.nl/speel~VPRO_1131430~bye~.html
October 27, 2021 12:34
The TPL (Task Parallel Library) was great invention back in the 2012. I still remember how great progress it was comparing to manually dealing with threads.

And later async / await came and changed rules of the game forever, which were widely adopted in other languages.

I had used Parallel.ForEach() a couple of time for CPU time sensitive operations. But for I/O, especially network, you needed to use Tasks to achieve parallelism (before Async APIs). Until now :)

Looking forward to see Parallel.ForEachAsync() back ported to the .NET Framework/Mono ecosystem.
October 28, 2021 6:19
In previous version of .NET if we want the same thing, we need to use the below code.

await Task.Run(() => Parallel.ForEach(userHandlers, parallelOptions, async (uri, token) =>
{
var user = await client.GetFromJsonAsync<GitHubUser>(uri, token);

Console.WriteLine($"Name: {user.Name}\nBio: {user.Bio}\n");
}));

Now it is much more clean.
October 30, 2021 14:50
Re: Fix Website Errors

Hello

My name is Sofia I am a Google Certified Digital Marketer and also an SEO expert.

I found few errors which correspond with a drop of website traffic over the last 1-2 months which I thought I would bring to your attention.

I would be happy to send you errors and the solutions that would help to improve your website performance and traffic?

Sincerely,
Sofia Jones
sofiaseowebmaster@gmail.com

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.