blog post image

Andrew Lock | .NET Escapades Andrew Lock

  • .NET Core 6
  • Source Code Dive

A deep-dive into the new Task.WaitAsync() API in .NET 6

In this post I look at how the new Task.WaitAsync() API is implemented in .NET 6, looking at the internal types used to implement it.

Adding a timeout or cancellation support to await Task

In my previous post , I showed how you could "cancel" an await Task call for a Task that didn't directly support cancellation by using the new WaitAsync() API in .NET 6.

I used WaitAsync() in that post to improve the code that waits for the IHostApplicationLifetime.ApplicationStarted event to fire. The final code I settled on is shown below:

In this post, I look at how the .NET 6 API Task.WaitAsync() is actually implemented.

Diving into the Task.WaitAsync implementation

For the rest of the post I'm going to walk through the implementation behind the API. There's not anything very surprising there, but I haven't looked much at the code behind Task and its kin, so it was interesting to see some of the details.

Task.WaitAsync() was introduced in this PR by Stephen Toub .

We'll start with the Task.WaitAsync methods :

These three methods all ultimately delegate to a different, private , WaitAsync overload (shown shortly) that takes a timeout in milliseconds. This timeout is calculated and validated in the ValidateTimeout method , shown below, which asserts that the timeout is in the allowed range, and converts it to a uint of milliseconds.

Now we come to the WaitAsync method that all the public APIs delegate too . I've annotated the method below:

Most of this method is checking whether we can take a fast-path and avoid the extra work involved in creating a CancellationPromise<T> , but if not, then we need to dive into it. Before we do, it's worth addressing the VoidTaskResult generic parameter used with the returned CancellationPromise<T> .

VoidTaskResult is an internal nested type of Task , which is used a little like the unit type in functional programming ; it indicates that you can ignore the T .

Using VoidTaskResult means more of the implementation of Task and Task<T> can be shared. In this case, the CancellationPromise<T> implementation is the same in both the Task.WaitAsync() implementation (shown above), and the generic versions of those methods exposed by Task<TR> . .

So with that out the way, let's look at the implementation of CancellationPromise<T> to see how the magic happens.

Under the hood of CancellationPromise<T>

There's quite a few types involved in CancellationPromise that you probably won't be familiar with unless you regularly browse the .NET source code, so we'll take this one slowly.

First of all, we have the type signature for the nested type CancellationPromise<T> :

There's a few things to note in the signature alone:

  • private protected —this modifier means that the CancellationPromise<T> type can only be accessed from classes that derive from Task , and are in the same assembly . Which means you can't use it directly in your user code.
  • Task<TResult> —the CancellationPromise<T> derives from Task<TResult> . For the most part it's a "normal" task, that can be cancelled, completed, or faulted just like any other Task .
  • ITaskCompletionAction —this is an internal interface that essentially allows you to register a lightweight action to take when a Task completes. This is similar to a standard continuation created with ContinueWith , except it is lower overhead . Again, this is internal , so you can't use it in your types. We'll look in more depth at this shortly.

We've looked at the signature, now let's look at it's private fields. The descriptions for these in the source cover it pretty well I think:

So we have 3 fields:

  • The original Task on which we called WaitAsync()
  • The cancellation token registration received when we registered with the CancellationToken . If the default cancellation token was used, this will be a "dummy" default instance.
  • The timer used to implement the timeout behaviour (if required).

Note that the _timer field is of type TimerQueueTimer . This is another internal implementation, this time it is part of the overall Timer implementation . We're going deep enough as it is in this post, so I'll only touch on how this is used briefly below. For now it's enough to know that it behaves similarly to a regular System.Threading.Timer .

So, the CancellationPromise<T> is a class that derives from Task<T> , maintains a reference to the original Task , a CancellationTokenRegistration , and a TimerQueueTimer .

The CancellationPromise constructor

Lets look at the constructor now. We'll take this in 4 bite-size chunks. First off, the arguments passed in from Task.WaitAsync() have some debug assertions applied, and then the original Task is stored in _task . Finally, the CancellationPromise<T> instance is registered as a completion action for the source Task (we'll come back to what this means shortly).

Next we have the timeout configuration. This creates a TimerQueueTimer and passes in a callback to be executed after millisecondsDelay (and does not execute periodically). A static lambda is used to avoid capturing state, which instead is passed as the second argument to the TimerQueueTimer . The callback tries to mark the CancellationPromise<T> as faulted by setting a TimeoutException() (remember that CancellationPromise<T> itself is a Task ), and then does some cleanup we'll see later.

Note also that flowExecutionContext is false , which avoids capturing and restoring the execution context for performance reasons. For more about execution context, see this post by Stephen Toub .

After configuring the timeout, the constructor configures the CancellationToken support. This similarly registers a callback to fire when the provided CancellationToken is cancelled. Note that again this uses UnsafeRegister() (instead of the normal Register() ) to avoid flowing the execution context into the callback.

Finally, the constructor does some house keeping. This accounts for the situation where the source Task completes while the constructor is executing , before the timeout and cancellation have been registered. Or if the timeout fires before the cancellation is registered. Without the following block, you could end up with leaking resources not being cleaned up

That's all the code in the constructor. Once constructed, the CancellationPromise<T> is returned from the WaitAsync() method as a Task (or a Task<T> ), and can be awaited just as any other Task . In the next section we'll see what happens when the source Task completes.

Implementing ITaskCompletionAction

In the constructor of CancellationPromise<T> we registered a completion action with the source Task (the one we called WaitAsync() on):

The object passed to AddCompletionAction() must implement ITaskCompletionAction (as CancellationPromise<T> does) ITaskCompletionAction interface is simple, consisting of a single method (which is invoked when the source Task completes) and a single property:

CancellationPromise<T> implements this method as shown below. It sets InvokeMayRunArbitraryCode to true (as all non-specialised scenarios do) and implements the Invoke() method, receiving the completed source Task as an argument.

The implementation essentially "copies" the status of the completed source Task into the CancellationPromise<T> task:

  • If the source Task was cancelled, it calls TrySetCancelled , re-using the exception dispatch information to "hide" the details of CancellationPromise<T>
  • If the source task was faulted, it calls TrySetException()
  • If the task completed, it calls TrySetResult

Note that whatever the status of the source Task , the TrySet* method may fail, if cancellation was requested or the timeout expired in the mean time. In those cases the bool variable is set to false , and we can skip calling Cleanup() (as the successful path will call it instead).

Now you've seen all three callbacks for the 3 possible outcomes of WaitAsync() . In each case, whether the task, timeout, or cancellation completes first, we have some cleanup to do.

Cleaning up

One of the things you can forget when working with CancellationToken s and timers, is to make sure you clean up after yourself. CancellationPromise<T> makes sure to do this by always calling Cleanup() . This does three things:

  • Dispose the CancellationTokenRegistration returned from CancellationToken.UnsafeRegister()
  • Close the ThreadQueueTimer (if it exists), which cleans up the underlying resources
  • Removes the callback from the source Task , so the ITaskCompletionAction.Invoke() method on CancellationPromise<T> won't be called.

Each of these methods is idempotent and thread-safe, so it's safe to call the Cleanup() method from multiple callbacks, which might happen if something fires when we're still running the CancellationPromise<T> constructor, for example.

One point to bear in mind is that even if a timeout occurs, or the cancellation token fires and the CancellationPromise<T> completes, the source Task will continue to execute in the background. The caller who executed source.WaitAsync() won't ever see the output of result of the Task , but if that Task has side effects, they will still occur.

And that's it! It took a while to go through it, but there's not actually much code involved in the implementation of WaitAsync() , and it's somewhat comparable to the "naive" approach you might have used in previous versions of .NET, but using some of .NET's internal types for performance reasons. I hope it was interesting!

In this post I took an in-depth look at the new Task.WaitAsync() method in .NET 6, exploring how it is implemented using internal types of the BCL. I showed that the Task returned from WaitAsync() is actually a CancellationPromise<T> instance, which derives from Task<T> , but which supports cancellation and timeouts directly. Finally, I walked through the implementation of CancellationPromise<T> , showing how it wraps the source Task .

Popular Tags

task in net core

Stay up to the date with the latest posts!

Dot Net Tutorials

Back to: C#.NET Tutorials For Beginners and Professionals

Task in C# with Examples

In this article, I am going to discuss Task in C# with Examples. Please read our previous article where we discussed how to implement Asynchronous Programming using Async and Await Operators in C# with Examples.

In C#, when we have an asynchronous method, in general, we want to return one of the following data types.

  • Task and Task<T>
  • ValueTask and ValueTask<T>

We will talk about ValueTask later, Now let us keep the focus on Task. The Task data type represents an asynchronous operation. A task is basically a “promise” that the operation to be performed will not necessarily be completed immediately, but that it will be completed in the future.

What is the difference between Task and Task<T> in C#?

Although we use both of them i.e. Task and Task<T> in C# for the return data type of an asynchronous method, the difference is that the Task is for methods that do not return a value while the Task<T> is for methods that do return a value of type T where T can be of any data type, such as a string, an integer, and a class, etc. We know from basic C# that a method that does not return a value is marked with a void. This is something to avoid in asynchronous methods. So, don’t use async void except for event handlers.

Example to Understand Task in C#:

In our previous example, we have written the following SomeMethod.

Now, what we will do is we will move the Task.Dealy to separate method and call that method inside the SomeMethod. So, let’s create a method with the name Wait as follows. Here, we mark the method as async so it is an asynchronous method that will not block the currently executing thread. And when calling this method it will wait for 10 seconds. And more importantly, here we use the return type as Task as this method is not going to return anything.

In asynchronous programming when your method does not return anything, then instead of using void you can use Task. Now, from the SomeMethod we need to call the Wait method. If we call the Wait method like the below then we will get a warning.

Here, you can see green lines under the Wait method as shown in the below image.

Task in C# with Examples

Why is that?

This is because the Wait method returns a Task and because it does return a Task, then it means that this will be a promise. So, this warning of the Wait method informed us that, if we don’t use the await operator while calling the Wait method, the Wait method is not going to wait for this operation to finish, which means that once we call the Wait method, the next line of code inside the SomeMethod is going to be executed immediately. Let us see that practically. The following is the complete example code.

Output: Once you execute the above code, then you will observe that without any delay we are getting the output as shown in the below image. This is because we are not using the await operator while calling the Wait method and hence it will not wait for the Wait method to complete. After 10 seconds the print statement within the Wait method is printed.

What is the difference between Task and Task<T> in C#?

In the above example, we use await Task.Delay inside the Wait method. This will suspend the thread for this Wait method execution only. It will not suspend the thread for SomeMethod execution. Now, let’s see what happens when we use the await operator as shown in the below example.

First thing, once you put the await operator, as shown above, the green warning will be gone. With await operator we are saying, please wait for this Wait method execution to finish before executing the next line of code inside the SomeMethod. That means it will not execute the last print statement inside the SomeMethod until the Wait method completes its execution.

await operator in C#

So, let us see that. The complete example code is given below

Example to Understand Task in C#

Now, you can observe in the above output that once it calls the Wait method, then the SomeMethod will wait for the Wait method to complete its execution. You can see that before printing the last print statement of the SomeMethod, it prints the printing statement of the Wait method. Hence, it proves that when we use await operator then the current method execution waits until the called async method completes its execution. Once the async method, in our example Wait method, complete its example, then the calling method, in our example SomeMethod, will continue its execution i.e. it will execute the statement which is present after the async method call.

What if you don’t want to wait for an asynchronous method in C#?

If you don’t want your method execution to wait for the asynchronous method to complete its execution, then, in that case, you need to use the return type of the asynchronous method to void. For a better understanding, please have a look at the below example. In the below example, we have used void as the return type of the asynchronous Wait method and while calling the asynchronous Wait method inside the SomeMethod we are not using await operator. This time please observe we are not getting any warning.

What if you don’t want to wait for an asynchronous method in C#?

Now you can observe the first four statements are printed immediately without waiting for the Wait method. After 10 seconds only the last statement is printed on the console window. Now, I hope you understand when to use Task and when to use void as the return type of an asynchronous method. I hope you also understand the importance of await operator.

Here, we have seen the examples of the async method without returning any value and hence we can use either void or Task as per our requirement. But what if the async method returns a value? If the async method returns a value then we need to use Task<T> and which we will discuss in our next article.

In the next article, I am going to discuss How to Return a Value from a Task in C# with Examples. Here, in this article, I try to explain Task in C# with Examples. I hope you enjoy this Task C# with Examples article.

dotnettutorials 1280x720

About the Author: Pranaya Rout

Pranaya Rout has published more than 3,000 articles in his 11-year career. Pranaya Rout has very good experience with Microsoft Technologies, Including C#, VB, ASP.NET MVC, ASP.NET Web API, EF, EF Core, ADO.NET, LINQ, SQL Server, MYSQL, Oracle, ASP.NET Core, Cloud Computing, Microservices, Design Patterns and still learning new technologies.

7 thoughts on “Task in C#”

task in net core

Guys, Please give your valuable feedback. And also, give your suggestions about this Task in C# concept. If you have any better examples, you can also put them in the comment section. If you have any key points related to Task in C#, you can also share the same.

task in net core

what is the difference of async and Task

task in net core

Simply Awesome. Very nice explanation…I am looking for more content in this site. Keep up doing good work.

task in net core

Don’t show videos ads . It’s ruining the site. And it’s very irritating as well.

Thanks we take into this consideration and disable the video ads.

task in net core

hi, thanks for this very good tutorial, truly a step by step and clear run through, i went over a good few articles and left more puzzled after each one, until I read yours. Exactly how code should be explained.

task in net core

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Code2night

Updated On Jul 20,2022

Welcome, fellow developers, to Code2Night! In this article, we will see a task scheduler in asp.net core. In the realm of building robust web applications with ASP.NET Core, there comes a time when we need to automate certain processes or tasks to run at regular intervals. Whether it's sending out email reminders, updating database records, or performing background computations, a reliable task scheduler becomes an indispensable tool in our arsenal.

In this blog post, we will delve into the exciting world of task scheduling in asp.net core . we'll explore how to design and implement a task to execute recurring jobs efficiently, ensuring that critical processes are automated with precision., throughout this journey, we'll harness the power of asp dot net core, a versatile and powerful framework that has revolutionized web development. we'll leverage its flexibility, performance, and extensive ecosystem to create a task that fits seamlessly into our application architecture., by the end of this blog post, you'll have a comprehensive understanding of how to build a robust and reliable task for recurring jobs in your asp dot net core applications. you'll be equipped with the knowledge and tools to automate your application's processes effectively, allowing you to focus on what truly matters—delivering exceptional user experiences., so, let's embark on this coding adventure together and discover the art of task scheduling in asp dot net core. are you ready let's dive in.

task scheduler

Task schedulers are basically used to run some piece of code at regular intervals so they can be called recurring jobs. So for creating a task scheduler in asp.net core follow these steps

So, first of all, we have to take a new asp.net core project application and add one new class SchedulerService. 

Now run the application and the RunJob method will run after the given interval. This is how you can create a task scheduler in asp.net core.

task in net core

Free Download for Youtube Subscribers!

First click on subscribe now and then subscribe the channel and come back here. then click on "verify and download" button for download link.

Support Us....!

Please subscribe to support us, thank you for downloading...., be a member.

MarketSplash

Implementing ASP.NET Core Background Tasks In Your Project

Explore the integration of ASP.NET Core background tasks to streamline processes and enhance your applications' efficiency.

💡 KEY INSIGHTS

  • Asynchronous programming patterns are crucial for ensuring that long-running background tasks do not block the main thread, maintaining application responsiveness.
  • Implementing error handling within background tasks is essential, as unhandled exceptions can disrupt the smooth operation of the application.
  • Resource management and monitoring in background tasks help prevent performance degradation and ensure system stability over time.
  • Graceful shutdown strategies, including the use of cancellation tokens, are vital for safely stopping background tasks without data loss or corruption.

Leveraging ASP.NET Core to manage background tasks effectively can enhance the responsiveness and scalability of your applications. This article walks through the essentials of integrating and optimizing background services, ensuring your server-side logic runs smoothly.

task in net core

Understanding ASP.NET Core Background Tasks

Setting up a background task in asp.net core, implementing ihostedservice, using backgroundservice class, handling long-running tasks in asp.net core, task queues in asp.net core, cancellation tokens and shutdown triggers, logging and error handling, best practices for background tasks, frequently asked questions, creating a background task, registering the task, handling task execution, monitoring and reliability.

ASP.NET Core Background Tasks allow developers to run background operations in a web application.

IHostedService and BackgroundService are the primary interfaces for implementing background tasks. These are used to create services that run in the background and can be executed on application start or stopped gracefully.

To create a background task, you start by implementing the IHostedService interface , which requires defining two methods: StartAsync and StopAsync . Here's a simple code example:

Once your background task is defined, you must register it in the service container within the Startup.cs :

You manage the actual task execution within the StartAsync method . It's critical to handle exceptions and ensure the task does not crash the application.

Cancellation Tokens are used to stop the task gracefully. They can signal the task to terminate when the application is shutting down.

Maintaining Reliability and Monitoring is crucial. Implement logging within your tasks to track their progress and errors.

Background tasks should be designed to be Idempotent and Resilient , meaning they can handle restarts and errors without losing data or causing corruption.

Service Registration

Dependency injection.

Setting up a Background Task in ASP.NET Core involves a few straightforward steps. The process integrates the task into the application's lifecycle, ensuring it starts and stops with the application.

The BackgroundService class is a base class for implementing a long-running IHostedService . Here's a basic implementation:

Register your background service in the ConfigureServices method of the Startup.cs file. This tells the framework to manage the task's lifecycle.

Inside the ExecuteAsync method , implement the Logic of your task. This method runs when the service starts.

Use Cancellation Tokens to listen for stop signals to terminate the task safely, which is essential for graceful shutdowns.

Dependency Injection can be used within your background service. Simply add required services to the constructor of your background task class.

Ensuring proper Error Handling and Logging within your background task is vital for maintaining application stability and diagnosing issues.

Initialization And Cleanup

Registration in the service container.

Implementing IHostedService is about defining tasks that the ASP.NET Core application controls directly. This interface allows precise control over the background task's lifetime.

When you implement IHostedService , you need to define two methods: StartAsync and StopAsync . The StartAsync method is called when the application starts, and StopAsync is called when the application is shutting down.

In StartAsync , initialize Resources needed for your task, such as database connections or timers. Conversely, StopAsync should free those resources.

Cancellation Tokens provided to these methods should be observed for any ongoing tasks. They signal the need for a task to terminate, for example, when the application is closing.

To activate the IHostedService implementation , it must be registered in the application's service container. This is done in the Startup.cs file:

Singleton Lifecycle is usually preferred for IHostedService implementations because they are typically started and stopped exactly once during the application's lifetime.

Overriding ExecuteAsync

Service lifecycle, dependency injection and logging.

The BackgroundService class is an abstract class that simplifies the implementation of a long-running IHostedService . This class provides a framework for writing background tasks.

By extending BackgroundService , you only need to override the ExecuteAsync method . This method is called when the Service Starts .

The ExecuteAsync method should contain the Logic of your background task. This is where you would place the task's operational code.

The Lifecycle of the service is managed by the host. The host calls StartAsync at the beginning and StopAsync at the end, which calls ExecuteAsync and then waits for it to complete when the application stops, respectively.

Cancellation Tokens are used within ExecuteAsync to handle graceful stops. Always observe the stoppingToken to stop the task when the application is shutting down.

With BackgroundService , you can easily inject dependencies such as Logging or other services through the constructor.

Remember to register your BackgroundService in the Startup.cs to have it managed by the ASP.NET Core hosting environment.

Ensuring Responsiveness

Managing resources, task cancellation, error handling.

Handling Long-Running Tasks in ASP.NET Core is a common requirement for many applications.

One approach is to use Hosted Services . These are background tasks that run in parallel to the application's main execution flow.

To ensure that the application remains responsive, tasks should be non-blocking. This is typically achieved using Asynchronous Programming patterns.

For tasks that are truly long-running, it's important to manage Memory and other resources carefully to prevent leaks that can degrade performance over time.

Long-running tasks should honor Cancellation Requests to allow for graceful shutdowns. This is managed through the use of cancellation tokens provided by the hosting environment.

Error Handling in long-running tasks is critical. Unhandled exceptions can cause a background task to stop working unexpectedly. Always include try-catch blocks to handle exceptions appropriately.

Processing The Queue

Synchronization.

Task Queues in ASP.NET Core provide a way to manage and process background tasks sequentially.

A task queue is implemented as a Collection that is thread-safe, such as ConcurrentQueue . Tasks are enqueued for processing by background services.

A BackgroundService can be used to process tasks from the queue. It waits for tasks to be enqueued and processes them one by one.

When multiple instances of a service might dequeue tasks concurrently, synchronization is necessary to ensure that only one instance processes a task at a time.

Dependency Injection can be used to register the task queue and the service that processes it within the application's services.

Error Handling within the queue processing is crucial to prevent one failing task from blocking the queue. Each task should be wrapped in a try-catch block.

Integrating Cancellation Tokens

Triggering shutdown, handling task cleanup.

Cancellation Tokens in ASP.NET Core signal a request to cancel an operation. They are essential for the graceful shutdown of background tasks and services.

Shutdown Triggers , such as application stop events, use cancellation tokens to notify running tasks that they should terminate.

Cancellation tokens are threaded through method calls to be checked periodically during the execution of a task. This allows for cooperative Cancellation of operations.

ASP.NET Core applications can trigger shutdown using IApplicationLifetime or the newer IHostApplicationLifetime .

When implementing long-running services , always ensure you pass and observe the cancellation token provided by the framework to respect the application's lifetime events.

During shutdown, ensure that resources are released and any final data is persisted or rolled back properly to maintain Data Integrity .

In summary, cancellation tokens are a vital part of writing resilient and responsive services and tasks in ASP.NET Core.

Structured Logging

Exception handling best practices.

Logging is a critical component of any application, providing insights into the system's behavior and helping diagnose issues when they occur.

To implement logging, you should inject an ILogger instance into your service or task class. This allows you to log messages at various levels of severity.

Structured logging allows you to include context in your log messages, making them more informative and easier to query.

Error Handling is just as important as logging. In background tasks, exceptions should be caught and logged to avoid unhandled exceptions causing the application to crash.

It's best practice to catch specific exceptions and handle them accordingly. This provides a more granular control over the error handling flow.

In tasks that involve looping or continuous processing, ensure that an error in one iteration does not prevent the next iteration from executing.

Good Logging And Error Handling practices are essential for the maintainability and reliability of background services in ASP.NET Core.

Avoiding Resource Starvation

Monitoring and health checks, graceful shutdown.

Mastering ASP.NET Core background tasks is like tuning an engine; it's all about ensuring seamless performance under the hood, so the drive is smooth and uninterrupted on the surface.

Yehia A.Salam

ASP.NET Developer

Source: Stack Overflow

Best Practices For Background Tasks are essential to ensure that they are reliable, maintainable, and scalable.

Ensure tasks are Idempotent , meaning they can be run repeatedly without causing unintended effects. This is important for tasks that may be retried due to failures.

Resource Management is critical. Background tasks should use resources efficiently, releasing them as soon as they are no longer needed.

It's important to prevent Resource Starvation , where a background task uses so many resources that it impacts the main application. Implement throttling or limits on resource usage.

Implement Monitoring and Health Checks to track the status and performance of background tasks. This can alert you to issues before they become critical.

Graceful Shutdown is ensuring that tasks can be stopped without data loss or corruption. Listen for cancellation tokens and save progress if needed.

Lastly, keep your tasks Decoupled and Modular . This makes them easier to manage, test, and maintain over the application's lifecycle.

What is a background task in ASP.NET Core?

A Background Task in ASP.NET Core is a non-blocking process that runs in the background of an application, handling long-running operations independently of user interactions.

How do you create a background task?

To create a background task, implement the IHostedService interface or extend the BackgroundService abstract class, then register it with the service container.

Can background tasks run after a request is completed?

Yes, background tasks can continue to run after a request is completed if they are designed to be independent of the request's lifecycle.

How do you stop a background task gracefully?

To stop a background task gracefully, use Cancellation Tokens to signal the task to stop, and implement your stopping logic accordingly.

What happens to background tasks when the application shuts down?

When the application shuts down, it triggers the StopAsync method of each registered IHostedService . You can implement custom cleanup logic here.

Let's see what you learned!

What is the primary interface used for creating background tasks in ASP.NET Core?

Continue learning with these asp.net guides.

  • Integrating ASP.NET Core Hangfire For Task Scheduling
  • Effective ASP.NET Core Caching Strategies For Your Projects
  • Implementing ASP.NET Core Distributed Caching
  • Boosting ASP.NET Core Performance Optimization
  • ASP.NET Core Monitoring Essentials

Subscribe to our newsletter

Subscribe to be notified of new content on marketsplash..

Code Maze

  • Blazor WASM 🔥
  • ASP.NET Core Series
  • GraphQL ASP.NET Core
  • ASP.NET Core MVC Series
  • Testing ASP.NET Core Applications
  • EF Core Series
  • HttpClient with ASP.NET Core
  • Azure with ASP.NET Core
  • ASP.NET Core Identity Series
  • IdentityServer4, OAuth, OIDC Series
  • Angular with ASP.NET Core Identity
  • Blazor WebAssembly
  • .NET Collections
  • SOLID Principles in C#
  • ASP.NET Core Web API Best Practices
  • Top REST API Best Practices
  • Angular Development Best Practices
  • 10 Things You Should Avoid in Your ASP.NET Core Controllers
  • C# Back to Basics
  • C# Intermediate
  • Design Patterns in C#
  • Sorting Algorithms in C#
  • Docker Series
  • Angular Series
  • Angular Material Series
  • HTTP Series
  • .NET/C# Author
  • .NET/C# Editor
  • Our Editors
  • Leave Us a Review
  • Code Maze Reviews

Select Page

Parallel.ForEachAsync() and Task.Run() With When.All in C#

Posted by Aneta Muslic | Feb 20, 2024 | 0

Parallel.ForEachAsync() and Task.Run() With When.All in C#

Parallel programming is a common and broad concept in the .NET world. In this article, we compare two well-known methods we use to achieve it when running a repetitive asynchronous task. We take a look at how they behave under the hood and compare the pros and cons of each one.

Let’s make a start.

Parallel Programming

In general, parallel programming involves using multiple threads or processors to execute tasks concurrently. It aims to improve performance and responsiveness by dividing tasks into smaller parts that can be processed simultaneously. 

Apart from improving performance and responsiveness, there are additional advantages when using parallel programming. Firstly, by breaking tasks into concurrent subtasks, we can effectively reduce overall execution time. One additional benefit is throughput enhancement as a result of handling multiple tasks simultaneously. Also, running tasks in parallel helps us ensure scalability since it efficiently distributes tasks across processors. This allows performance to scale seamlessly when adding resources.

One more thing we should take into consideration when working with parallel programming is which kind of processes we are trying to parallelize. In this article, we will mention I/O-bound and CPU-bound ones.  

Become a patron at Patreon!

I/O bound processes are processes where the computational duration is determined by the time spent awaiting input/output operations, an example of this is a database call. On the other hand, we have CPU-bound processes. In this case, the performance of the CPU determines the task duration, an example is a method that does some heavy numerical calculations.

Now that we have a quick primer about parallel programming and different types of processes, let’s quickly set everything up and see it in action.

Setting up Async Methods

Since we already have a great article going more in-depth on How to Execute Multiple Tasks Asynchronously , here we will only create a baseline for the Task.WhenAll() method which we will modify when comparing the two approaches.

We start with the default web-API project and expand the WeatherForecastController method with an asynchronous method that runs multiple times:

In the context of types of processes, AsyncMethod() emulates the I/O-bound process and the task delay represents the waiting time of a sub-system response.

After we set everything up, let’s see how to execute these tasks in parallel.

Use Task.WhenAll

First, we need to refactor the GetWeatherForecastWhenAll() method to use the Task.WhenAll() method. It takes an enumerable of tasks and returns a new completed task once all the individual tasks in the collection finish running:

We define an empty list of tasks. Next, we call AsyncMethod() three times without the await keyword. This starts executing these tasks one after another without waiting for them to complete . This is exactly what we want since we add those tasks to our tasks list and use Task.WhenAll() to wait for all of them to complete.

Lastly, when all the tasks are completed, we flatten the combinedResults variable that holds the results and return the result to the user.

We need to keep thread usage in mind when we use parallel execution of tasks. Starting too many threads at once increases context-switching overhead and may impact overall application efficiency. Also, we don’t want to block the main thread. So let’s see how we can get a better understanding of how this method works under the hood regarding threads.

Thread Processing

We start by adding logging to the threads:

Here, we add a Console.WriteLine() statement at the beginning and end of each method. There, we print on which thread methods start and end by using Environment.CurrentManagedThreadId .

Now, if we execute our request, in the output window we can see how threads behave:

Let’s break this down to understand what happens.

When we send an HTTP request, a thread from the thread pool gets assigned to handle it. In our case, it is thread number 16. Then, when we invoke our async methods and we don’t use the await keyword, tasks will usually start executing on the same thread, i.e., 16.

However, when an asynchronous operation encounters the await keyword, in our case await on Task.WhenAll() , it releases the current thread to the thread pool during the waiting period for the task to be completed. When the awaiting operation completes and we want to return the result, the continuation might not necessarily resume on the original thread. That is why we see some of the tasks finish on different threads than they start on.

Besides creating a task by not using the await keyword we can also use Task.Run() method, so let’s take a look at it.

Use Task.Run With Task.WhenAll

By using the Task.Run()  method to execute tasks, we make sure that each new task executes on a separate thread :

Here, we use the Task.Run() method to execute AsyncMethod() three times in a row. Again, by skipping the await keyword we are not awaiting any method to complete, but we run them in parallel and on Task.WhenAll() await their results.

Now, let’s retake a look at the output logs when executing the request:

This time, we see that each new task starts its execution on a new thread. We expect this behavior when using Task.Run() since its purpose is to offload work from the current thread. Same as in the previous example due to the async/await nature and thread pool assigning threads, tasks finish on different threads than they originally start on.

Using Task.Run() requires caution as it might have some drawbacks. Since it offloads work to a new thread, any time it deals with a large number of tasks it can create a large number of threads, each consuming resources and possibly causing thread pool starvation.

Now that we have seen how we can explicitly offload each task to a new thread, let’s look at how we can use another method to perform these tasks in parallel.

Using Parallel.ForEachAsync

Another way we parallelize this work is to use the Parallel.ForEachAsync() method:

First, we set the MaxDegreeOfParallelism value. With this setting, we define how many concurrent operations run. If not set, it uses as many threads as the underlying scheduler provides . To determine this value for a CPU process start with the Environment.ProcessorCount . For I/O-bound processes, this value is harder to determine since it depends on the I/O subsystem, which includes network latency, database responsiveness, etc. So when working with I/O bound processes, we need to do testing with different values to determine the best value for maximum parallelization.

After, we define a ConcurrentBag for our results, which is a thread-safe collection since we use parallel execution of tasks and handle results in a loop. Allowing us to safely modify the collection without worrying about concurrency modification exceptions. Lastly, we set up Parallel.ForEachAsync() method to run three times with set options, and inside the loop, we await each result and add it to the resultBag .

One thing to mention when using the Parallel.ForEachAsync() method is that it has its underlying partitioning. This partitioning divides the input data into manageable batches and assigns each batch to a different thread for parallel processing. The exact size of the batches is determined dynamically by the framework based on factors such as the number of available processors and the characteristics of the input data. So by defining the MaxDegreeOfParallelism , we define the number of batched tasks that execute concurrently.

Regarding thread usage, since we are not explicitly altering thread assignments, threads get assigned as they usually do in the classic async/await process. One difference with the Task.WhenAll() thread usage is that most likely every task starts on its thread since we use the await keyword for each call inside the loop.

Now, let’s take a look at how the Task.Run() method behaves in this case.

Using Task.Run With Parallel.ForEachAsync

Let’s modify our method to use Task.Run() for generating tasks:

However, this may not be the best approach in this case. As we already saw, Parallel.ForEachAsync() has a built-in partitioner that creates batches of tasks and processes them in a single thread. But by using Task.Run() we offload each task into its thread. So using Task.Run() in this case, undermines the benefit of using Parallel.ForEachAsync() for chunking tasks and using fewer threads.

One more thing we may encounter when trying to parallelize the tasks is the usage of the Parallel.ForEach() method.

Pitfalls to Avoid With Parallel.ForEach

The Parallel.ForEach() method, while similar to Parallel.ForEachAsync() , lacks the designed capability to handle asynchronous work.  However, we can still encounter some examples of its usage with asynchronous tasks.

So let’s quickly check on why these approaches may not be the best workarounds and see their drawbacks.

One common thing we can see is forcing awaiting the result in synchronous code by using GetAwaiter () . GetResult() :

We should avoid this approach since by using GetAwaiter().GetResult() we block the calling thread, which is an anti-pattern of async/await . This may cause issues in deadlocks, decreased performance, and loss of context-switching benefits.

Another approach involves using async void:

In this approach, we have another anti-pattern, and that is the usage of async/void . This is a known bad practice with several reasons to avoid it. O ne such reason is that we cannot catch exceptions in the catch block.

As we can see, both of these approaches involve the use of anti-patterns to make Parallel.ForEach() them compatible with asynchronous methods. Since neither of them is a recommended way to implement parallelization, with the introduction of Parallel.ForEachAsync() in .NET 6 we have a preferable method for working with async tasks in a for-each loop.

Now that we took a look at what not to do, let’s sum up everything we’ve learned so far!

When to Use Which Approach?

As with everything in programming, how we use the knowledge from this article depends on the application’s specific requirements. Nevertheless, when choosing the right method, we should consider several factors.

When talking about CPU-bound tasks that can benefit from parallelization, the use of Parallel.ForEachAsync() stands out. Its main benefit is that it efficiently distributes the workload across multiple processor cores. Also, by setting the MaxDegreeOfParallelism we control the concurrency level we want to impose. And as we saw we can easily determine that value.

On the other hand, when dealing with I/O-bound tasks, where operations involve waiting for external resources, Task.WhenAll() becomes a preferable choice. It allows us to execute multiple asynchronous tasks concurrently, without blocking the calling thread. This makes it an efficient option for scenarios like database queries or network requests. Another benefit is that we don’t need to process results inside the loop, but we can wait on all of them and manipulate the results when they are complete.

However, it’s important to note that Task.WhenAll() lacks a built-in partitioner, and its use in a loop without proper throttling mechanisms may result in the initiation of an infinite number of tasks. So depending on the number of tasks we are executing it may be necessary to create our partition strategy or opt for Parallel.ForEachAsync() a solution.

One more thing we mentioned is initializing tasks using Task.Run() . We can use this approach when we want to have explicit control over threading but keep in mind that it can potentially lead to thread pool starvation if too many threads start at once. 

In this article, we look at two methods we use to execute repetitive tasks in parallel. We saw how both methods under the hood use threads and partition the given tasks. Also, we saw what are the differences when using the Task.Run() and how it behaves with both options. Lastly, we provide guidance on which approach is most suitable in different scenarios.

guest

Join our 20k+ community of experts and learn about our Top 16 Web API Best Practices .

task in net core

Learn about API Management from experts on today’s innovative API architectures to accelerate growth, improve discoverability, transform existing services, and more! All with a hybrid, multi-cloud management platform for APIs across all environments.

Introducing ASP.NET Core metrics and Grafana dashboards in .NET 8

' data-src=

James Newton-King

February 14th, 2024 11 12

Metrics report diagnostics about your app. .NET 8 adds over a dozen useful metrics to ASP.NET Core:

  • HTTP request counts and duration
  • Number of active HTTP requests
  • Route matching results
  • Rate limiting lease and queue durations
  • SignalR transport usage
  • Low-level connection and TLS usage from Kestrel
  • Error handling diagnostics

Metrics are numerical measurements reported over time. For example, each HTTP request handled by ASP.NET Core has its duration recorded to the http.server.request.duration metric. Tooling such as the .NET OpenTelemetry SDK can then be configured by an app to export metric data to a telemetry store such as Prometheus or Azure Monitor . Metrics are part of the OpenTelemetry standard and all modern tools support them.

Metrics data are useful when combined with tooling to monitor the health and activity of apps:

  • Display graphs on a dashboard to observe your app over time. For example, view activity from people using the app.
  • Trigger alerts in real-time if the app exceeds a threshold. For example, send an email if request duration or error count exceeds a limit.

Using metrics

ASP.NET Core’s built-in metrics are automatically recorded. How you use these metrics is up to you. Let’s explore some of the options available.

.NET Aspire dashboard

.NET Aspire is an opinionated stack for building observable, distributed applications. The Aspire dashboard includes a simple, user-friendly UI for viewing structured logs, traces and metrics. Aspire apps are automatically configured to send telemetry data to the dashboard during development.

Here you can see a collection of available metrics, along with name, description, and a graph of values. The Aspire UI includes metrics filters. Filtering is possible using a powerful feature of metrics: attributes.

Each time a value is recorded, it is tagged with metadata called attributes. For example, http.server.request.duration records a HTTP request duration along with attributes about that request: server address, HTTP request method, matched route, response status code and more. Attributes can then be queried to get the exact data you want:

  • HTTP request duration to a specific endpoint in your app, e.g. /product/{name} .
  • Count the number of requests with 4xx HTTP response codes.
  • View request count that threw a server exception over time.
  • HTTP vs HTTPS request duration.
  • Number of visitors using HTTP/1.1 vs HTTP/2.

ASP.NET Core Grafana dashboards

Grafana is powerful open-source tool for building advanced dashboards and alerts. It allows you to create interactive, customizable dashboards with a variety of panels, graphs, and charts. Once complete, a dashboard displays data from your telemetry store. Grafana is a good choice to monitor apps deployed to production and a dashboard gives a real-time view of an app health and usage.

Grafana gives you the power to build exactly what you want, but it takes time to build high-quality dashboards. As part of adding metrics in .NET 8, the .NET team created pre-built dashboards designed for ASP.NET Core’s built-in metrics.

A screenshot of the ASP.NET Core Grafana dashboard

The ASP.NET Core Grafana dashboards are open source on GitHub and available for download on grafana.com . You can use the dashboard as they are or customize them further to build a solution tailored to your needs.

Quickly try out Grafana + ASP.NET Core using the .NET Aspire metrics sample app .

  • Metrics aren’t limited to what is built into .NET. You can create custom metrics for your apps.
  • dotnet-counters is a command-line tool that can view live metrics for .NET Core apps on demand. It doesn’t require setup, making it useful for ad-hoc investigations or verifying that metric instrumentation is working.
  • Use metrics in unit tests . ASP.NET Core integration testing, MetricCollector and IMeterFactory can be combined to assert values recorded by a test.

.NET 8, ASP.NET Core Grafana dashboards and .NET Aspire (preview) are available now. Try using metrics today and let us know what you think:

  • Download the latest .NET 8 release .

Use metrics in your tool of choice:

  • Learn more about the .NET Aspire dashboard and get started .
  • Download ASP.NET Core Grafana dashboards .
  • Install dotnet-counters command-line tool.

Want to try things hands on? Checkout our new cloud-native training modules on Microsoft Learn.

Thanks for trying out .NET 8 and metrics!

' data-src=

James Newton-King Principal Software Engineer, .NET

task in net core

11 comments

Leave a comment cancel reply.

Log in to join the discussion.

' data-src=

Why is the date on this blog post six days in the future?

' data-src=

The blog post’s date metadata was wrong and is fixed.

' data-src=

@James, do these libraries work with .NET 7? Or are them .NET 8 specific?

Everything discussed requires .NET 8 or later.

' data-src=

Thanks @James for this insightful share.

Can you tell us more about the benefits of using this instead of Azure Monitor in a cloud scenario ?

I don’t have a huge amount of experience with Azure Monitor. I’m not qualified to give a good comparison between Azure Monitor vs Prometheus/Grafana.

One notable difference is that while Grafana and Prometheus have managed offerings, e.g. Azure Managed Grafana or Grafana Cloud , they can also be run on your own machine. If you want a solution that is portable then Prometheus/Grafana are a good choice.

' data-src=

Thank you James. Also think about adding a “lite” version of these metrics dashboards directly in the Aspire dashboard app. In other words, this solution requires both Prometheus and Grafana containers. It would be nice to have a ‘lite’ dashboard built in blazor/html directly inside the Aspire Dashboard app (without needing those two extra containers) – just displaying the transient in-memory telemetry. That would be a fun job for an intern to spend a few weeks on.

The Aspire dashboard has a capability to view metrics today. Because we have a simple solution, and it’s quite easy to setup Prometheus and Grafana in Aspire, we probably won’t add something like metrics dashboards to the Aspire dashboard app.

But if you think it is important then create an issue at https://github.com/dotnet/aspire and we can see how much demand there is by Aspire users.

' data-src=

Is there any recommended Grafana dashboard for http.client.* metrics?

We haven’t made a dashboard that includes client metrics, but you could copy or extend the ASP.NET Core dashboards to include them.

http.client.request.duration is very similar to http.server.request.duration so charts could be reused without significant changes.

' data-src=

fair discussion everything explained clearly

light-theme-icon

Insert/edit link

Enter the destination URL

Or link to existing content

This browser is no longer supported.

Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.

Build, test, and deploy .NET Core apps

  • 29 contributors

Azure DevOps Services | Azure DevOps Server 2022 - Azure DevOps Server 2019 | TFS 2018

Use an Azure Pipeline to automatically build, test, and deploy your .NET Core projects. This article shows you how to do the following tasks:

  • Set up your build environment with self-hosted agents.
  • Restore dependencies, build your project, and test with the .NET Core task (DotNetCoreCLI@2) or a script .
  • Test your code and use the publish code coverage task to publish code coverage results.
  • your pipeline.
  • a NuGet feed .
  • a .zip file to deploy a web app to Azure .
  • Set up your build environment with Microsoft-hosted or self-hosted agents.

For help with .NET Framework projects, see Build ASP.NET apps with .NET Framework .

Prerequisites

  • A GitHub account where you can create a repository. Create one for free .
  • An Azure DevOps organization and project. Create one for free .
  • An ability to run pipelines on Microsoft-hosted agents. You can either purchase a parallel job or you can request a free tier.
  • An Azure DevOps collection.
  • An ability to run pipelines on a self-hosted agent with Docker installed and running on the agent's host.

Create your first pipeline

Are you new to Azure Pipelines? If so, then we recommend you try the following section first.

Create a .NET project

If you don't have a .NET project to work with, create a new one on your local system. Start by installing the latest .NET 8.0 SDK .

Open a terminal window.

Create a project directory and navigate to it.

Create a new .NET 8 webapp.

From the same terminal session, run the application locally using the dotnet run command from your project directory.

Once the application has started, press Ctrl-C to shut it down.

Create a git repo and connect it to GitHub

From the project directory, create a local git repository and commit the application code to the main branch .

Connect your local Git repo to a GitHub repo .

Create a DevOps project

Sign-in to Azure Pipelines . After you sign in, your browser goes to https://dev.azure.com/my-organization-name and displays your Azure DevOps dashboard.

Within your selected organization, create a project . If you don't have any projects in your organization, you see a Create a project to get started screen. Otherwise, select the New Project button in the upper-right corner of the dashboard.

  • In a browser window, sign in to your Azure DevOps Server and select your collection.
  • Select New project .
  • Enter a project name.
  • Optionally, enter a description.
  • Select Create .

Set up your build environment

Your builds run on self-hosted agents . Make sure that you have the necessary version of the .NET Core SDK and runtime installed on the agents. You can build your .NET Core projects by using the .NET Core SDK and runtime on Windows , Linux , macOS and Docker .

You can install a specific version of .NET SDK by adding the UseDotNet@2 task in your pipeline YAML file or add the task to your pipeline using the classic editor.

Example YAML snippet:

Your builds run on Microsoft-hosted agents . You can build your .NET Core projects by using the .NET Core SDK and runtime on Windows, Linux, and macOS.

Alternatively, you can use a self-hosted agent . With a self-hosted agent, you can use preview or private SDKs not officially supported by Azure DevOps Services and run incremental builds.

Create your pipeline

You can use the YAML pipeline editor or the classic editor to create your pipeline. To use the classic editor, select Use the classic editor .

Create a new pipeline and select your source

Sign in to your Azure DevOps organization and go to your project.

Go to Pipelines , and then select New pipeline or Create pipeline if creating your first pipeline.

Do the steps of the wizard by first selecting GitHub as the location of your source code.

You might be redirected to GitHub to sign in. If so, enter your GitHub credentials.

When you see the list of repositories, select your repository.

You might be redirected to GitHub to install the Azure Pipelines app. If so, select Approve & install .

Configure your pipeline

When the Configure tab appears, select Show more and select the ASP.NET Core pipeline template from the list.

Examine your new pipeline to see what the YAML does.

You can customize the YAML file for your requirements. For example, you can specify the agent pool or add a task to install different .NET SDK .

Save and run your pipeline

When you're ready, select Save and run .

Save and run button in a new YAML pipeline

Optionally, you can edit the commit message.

Commit the new azure-pipelines.yml file to your repository by selecting Save and run .

To watch your pipeline in action, select the job in the Jobs section.

Create and run your pipeline

You can create a pipeline by using the YAML pipeline editor or the classic editor.

Go to your project and select Pipelines .

  • Select Create pipeline or New pipeline if creating the first pipeline for this project.

Select your source

Select your source repository. For this example, use GitHub Enterprise Server .

  • Enter the URL for your GitHub account. For example, https://github.com/<username> .
  • Enter your personal access token for your GitHub account.
  • Enter a Service connection name. For example, my-github .

Select your GitHub repository.

On the Configure tab, select Show more and select the ASP.NET Core pipeline template from the list.

You can customize the YAML file for your requirements. For example, you can add tasks to install a .NET SDK or to test and publish your project.

Select Save .

Screenshot showing the Save and run button in a new YAML pipeline.

To commit the pipeline YAML file to your repository, edit the commit message as needed and select Save .

Select Run to run your pipeline.

To see the build logs as your pipeline runs, select the build number at the top of the page.

Select Save and run .

To commit the new azure-pipelines.yml file to your repository, edit the commit message as needed and select Save and run .

Use these steps to create your pipeline using the classic editor.

Create pipeline

Select Create pipeline or New pipeline if you're not creating the first pipeline for this project.

Select Use the classic editor .

  • Select your source. For this example, select GitHub Enterprise Server .
  • Select Connect to GitHub Enterprise Server .
  • Enter your GitHub credentials to create a GitHub service connection to use in your pipeline.
  • Select your repository and select Continue .
  • Select your source. For this example, select GitHub .

From Select a template , find and select ASP.NET Core .

The pipeline page opens where you can configure your pipeline. Here you can add tasks, specify the agent pools and agents and configure other build options.

In the Tasks tab, select your Agent pool (usually Default )

Select the Agent specification . For this example, select windows-latest .

You can add other tasks to the Agent job by selecting + on the agent job and selecting another task from the catalog. For example, you might want to add the Use .NET Core task as the first task to install the necessary version of the .NET SDK.

  • Select Save and queue from the Save & queue dropdown list at the top of the page.
  • In Run pipeline , enter a comment and select Save and Run .

You can see your pipeline in action by selecting the job from the Jobs section on the Summary tab.

  • From the Save & queue dropdown list, select Save and queue .
  • From the Save build pipeline and queue dialog, select Save and queue .

When the Build #nnnnnnnn.n has been queued message appears, select the link to see your pipeline in action.

  • From the Save & queue dropdown list at the top of the page, select Save and queue .
  • On the Run pipeline dialog, add a Save comment and select Save and run .

You now have a working pipeline that's ready for you to customize! Read further to learn some of the common ways to customize your pipeline.

Build environment

Azure Pipelines uses self-hosted agents to build your .NET Core projects. Make sure that you have the necessary version of the .NET Core SDK and runtime installed on the agents. You can build your .NET Core projects by using the .NET Core SDK and runtime on Windows , Linux , macOS and Docker .

For example, to select a pool and agent capabilities in the pipeline YAML file:

You can select the agent pool and agent for your build job. Agents are specified based on their capabilities.

You can install a specific version of .NET SDK by adding the UseDotNet@2 task in your pipeline. Keep in mind that for agents that run on physical systems, installing SDKs and tools through your pipeline alters the build environment on the agent's host.

To install a newer SDK, set performMultiLevelLookup to true in the following snippet:

You can use Azure Pipelines to build your .NET Core projects on Windows, Linux, or macOS without the need to set up infrastructure.

For example, Ubuntu is set here in the pipeline YAML file.

See Microsoft-hosted agents for a complete list of images and further configuration examples.

The Microsoft-hosted agents in Azure Pipelines include several preinstalled versions of supported .NET Core SDKs. Microsoft-hosted agents don't include some of the older versions of the .NET Core SDK. They also don't typically include prerelease versions. If you need these versions of the SDK on Microsoft-hosted agents, install them using the UseDotNet@2 task.

For example, to install 5.0.x SDK, add the following snippet:

Windows agents already include a .NET Core runtime. To install a newer SDK, set performMultiLevelLookup to true in the following snippet:

To save the cost of running the tool installer, you can set up a Linux , macOS , or Windows self-hosted agent . You can also use self-hosted agents to save additional time if you have a large repository or you run incremental builds. A self-hosted agent can also help you in using the preview or private SDKs that aren't officially supported by Azure DevOps or are only available on your corporate or on-premises environments.

Restore dependencies

NuGet is a popular way to depend on code that you don't build. You can download NuGet packages and project-specific tools that are specified in the project file by running the dotnet restore command either through the .NET Core task or directly in a script in your pipeline. For more information, see .NET Core task (DotNetCoreCLI@2) .

You can download NuGet packages from Azure Artifacts, NuGet.org, or some other external or internal NuGet repository. The .NET Core task is especially useful to restore packages from authenticated NuGet feeds. If your feed is in the same project as your pipeline, you don't need to authenticate.

This pipeline uses an Azure Artifact feed for dotnet restore in the DotNetCoreCLI@2 task.

The dotnet restore command uses the NuGet.exe packaged with the .NET Core SDK and can only restore packages specified in the .NET Core project .csproj files.

If you also have a Microsoft .NET Framework project in your solution or use package.json to specify your dependencies, use the NuGet task to restore those dependencies.

In .NET Core SDK version 2.0 and newer, packages are restored automatically when running commands such as dotnet build . However, you would still need to use the .NET Core task to restore packages if you use an authenticated feed.

Your builds can fail because of connection issues when you restore packages from NuGet.org. You can use Azure Artifacts with upstream sources to cache the packages. The credentials of the pipeline are automatically used when it connects to Azure Artifacts. These credentials are typically derived from the Project Collection Build Service account. To learn more about using Azure Artifacts to cache your NuGet packages, see Connect to Azure Artifact feeds .

To specify a NuGet repository, put the URL in a NuGet.config file in your repository. If your feed is authenticated, manage its credentials by creating a NuGet service connection in the Services tab under Project Settings .

When you use Microsoft-hosted agents, you get a new machine every time you run a build, which restores the packages with each run. Restoration can take a significant amount of time. To mitigate, you can either use Azure Artifacts or a self-hosted agent with the benefit of using the package cache.

For more information about NuGet service connections, see publish to NuGet feeds .

Restore packages from an external feed

Do the following to restore packages from an external feed.

You can add the restore command to your pipeline using the YAML pipeline editor by directly inserting the following snippet into your azure-pipelines.yml file or using the task assistant to add the .NET Core task.

Replace the <placeholder> with your service connection name.

To use the task assistant:

To add a build task using the task assistant, do the following steps:

Go to the position in the YAML file where you want to insert the task.

Select the .NET Core from the task catalog.

Select the restore command from the Command dropdown list.

In the Path to project(s) field, enter the path to your .csproj files.

Select Add .

Select Save to commit the change.

Use these steps to add a restore task using the classic editor:

Select Tasks in your pipeline and select the job that runs your build tasks.

Select + to add a new task to that job.

In the task catalog, find and select .NET Core task and select Add .

Select the task to open the task editor.

Select restore in the Command list and specify any other options you need for this task.

In the task list, drag the task to position it before the build task.

From the Save & queue dropdown list, select an option to save the change.

Make sure the custom feed is specified in your NuGet.config file and that credentials are specified in the NuGet service connection.

Build your project

Build your .NET Core projects by running the dotnet build command. You can add the command to your pipeline as a command line script or by using the .NET Core task.

.NET Core build using the .NET Core task

YAML example to build using the DotNetCoreCLI@2 task:

You can add a build task using the YAML pipeline editor by directly editing the file or adding the .NET Core task using the task assistant.

Select the build command from the Command dropdown list.

To add a build task using the classic editor, do the following steps:

Select Tasks in your pipeline.

Select the job that runs your build tasks.

In the task catalog, find and add the .NET Core task.

Select the task and select build from the Command dropdown list.

Drag the task to position it in the correct task sequence in the pipeline.

Select the Save and queue dropdown list and select an option to save your changes.

.NET Core build using command line script

YAML example to build using dotnet build as a script:

You can add a build task using the YAML pipeline editor by directly editing the file or adding the Command Line task.

Use the following steps to add the Command Line task:

Select the Command Line from the task catalog.

Optionally, add a Display name .

Enter the dotnet build command with parameters. For example, dotnet build --configuration $(buildConfiguration) .

Enter the path to the .csproj file as the working directory.

In the task catalog, find and Add the Command Line task.

Add .NET SDK commands to your pipeline

You can add .NET SDK commands to your project as a script or using the .NET Core task. The .NET Core task (DotNetCoreCLI@2) task allows you to easily add dotnet CLI commands to your pipeline. You can add .NET Core tasks by editing your YAML file or using the classic editor.

Add a .NET CLI command using the .NET Core task

To add a .NET Core CLI command using the YAML pipeline editor, do the following steps:

Select .NET Core from the task catalog.

Select the command you want to run.

Configure any options needed.

To add .NET Core task using the classic editor, do the following steps:

Select the task and select the Command that you want to run.

From the Save and queue dropdown list, select an option to save your changes.

Add a .NET Core CLI command using a script

You can add .NET Core CLI commands as a script in your azure-pipelines.yml file.

Install a tool

To install a .NET Core global tool like dotnetsay in your build running on Windows, take the following steps:

  • Path to projects : leave empty .
  • Custom command : tool.
  • Arguments : install -g dotnetsay .
  • Script: dotnetsay .

Run your tests

When you have test projects in your repository, you can use the .NET Core task to run unit tests by using testing frameworks like MSTest, xUnit, and NUnit. The test project must reference Microsoft.NET.Test.SDK version 15.8.0 or higher. Test results are automatically published to the service. These results are available to you in the build summary and can be used for troubleshooting failed tests and test-timing analysis.

You can add a test task to your pipeline using the DotNetCoreCLI@2 task or add the following snippet to your azure-pipelines.yml file:

When using the .NET Core task editor, set Command to test and Path to projects should refer to the test projects in your solution.

Alternatively, you can run the dotnet test command with a specific logger and then use the Publish Test Results task:

Collect code coverage

When you're building on the Windows platform, code coverage metrics can be collected by using the built-in coverage data collector. The test project must reference Microsoft.NET.Test.SDK version 15.8.0 or higher.

When you use the .NET Core task to run tests, coverage data is automatically published to the server. The .coverage file can be downloaded from the build summary for viewing in Visual Studio.

Add the following snippet to your azure-pipelines.yml file:

To add the .NET Core task through the task editor:

Add the .NET Core task to your build job and set the following properties:

  • Command : test.
  • Path to projects : Should refer to the test projects in your solution .
  • Arguments : --configuration $(BuildConfiguration) --collect "Code coverage" .

Ensure that the Publish test results option remains selected.

If you choose to run the dotnet test command, specify the test results logger and coverage options. Then use the Publish Test Results task:

Collect code coverage metrics with Coverlet

If you're building on Linux or macOS, you can use Coverlet or a similar tool to collect code coverage metrics.

You can publish code coverage results to the server with the Publish Code Coverage Results (PublishCodeCoverageResults@1) task. The coverage tool must be configured to generate results in Cobertura or JaCoCo coverage format.

To run tests and publish code coverage with Coverlet, do the following tasks:

Add a reference to the coverlet.collector NuGet package.

Package and deliver your code

You can publish your build artifacts by:

  • Publishing to Azure Pipelines.
  • Publishing packages to Azure Artifacts.
  • Creating a NuGet package and publish to your NuGet feed.
  • Creating a .zip archive to deploy your web app.

Publish artifacts to Azure Pipelines

To publish the output of your .NET build to your pipeline, do the following tasks:

  • Run dotnet publish --output $(Build.ArtifactStagingDirectory) on the .NET CLI or add the DotNetCoreCLI@2 task with the publish command.
  • Publish the artifact by using the Publish Pipeline Artifact task.

The DotNetCoreCLI@2 task has a publishWebProjects input that is set to true by default. This task publishes all web projects in your repo by default. You can find more help and information in the open source task on GitHub .

To copy more files to the build directory before publishing, use the Copy Files (CopyFile@2) task.

  • Run dotnet publish --output $(Build.ArtifactStagingDirectory) on CLI or add the DotNetCoreCLI@2 task with the publish command.
  • Publish the artifact by using the Publish build artifact (PublishBuildArtifacts@1) task.

Add the following snippet to your azure-pipelines.yml file to publish your build artifacts as a .zip file:

For more information, see Publish and download build artifacts .

Publish to a NuGet feed

To create a NuGet package and publish it to your NuGet feed, add the following snippet:

The NuGetAuthenticate@1 task doesn't support NuGet API key authentication. If you're using a NuGet API key, use the NuGetCommand@2 task with the command input set to push with the --api-key argument. For example, dotnet nuget push --api-key $(NuGetApiKey) .

For more information about versioning and publishing NuGet packages, see publish to NuGet feeds .

Publish a NuGet package to Azure Artifacts

You can publish your NuGet packages to your Azure Artifacts feed by using the NuGetCommand@2 to push to your Azure Artifact feed. For example, see Publish NuGet packages with Azure Pipelines .

Deploy a web app

To create a .zip file archive that's ready to publish to a web app, add the following snippet:

To publish this archive to a web app, see Azure Web Apps deployment .

Build an image and push to container registry

You can also build an image for your app and push it to a container registry .

Publish symbols

You can use the PublishSymbols@2 task to publish symbols to an Azure Artifacts symbol server or a file share.

For example, to publish symbols to a file share, add the following snippet to your azure-pipelines.yml file:

When using the classic editor, select Index sources publish symbols from the task catalog to add to your pipeline.

For more information, see Publish symbols .

Troubleshoot

If you can build your project on your development machine, but you're having trouble building it on Azure Pipelines, explore the following potential causes and corrective actions:

  • Prerelease versions of the .NET Core SDK aren't installed on Microsoft-hosted agents. After a new version of the .NET Core SDK is released, it can take a few weeks to roll out to all the Azure Pipelines data centers. You don't have to wait for this rollout to complete. You can use the Use .NET Core task to install the .NET Core SDK version you want on Microsoft-hosted agents.

Check the .NET Core SDK versions and runtime on your development machine and make sure they match the agent. You can include a command-line script dotnet --version in your pipeline to print the version of the .NET Core SDK. Either use the .NET Core Tool Installer to deploy the same version on the agent, or update your projects and development machine to the newer version of the .NET Core SDK.

You might be using some logic in the Visual Studio IDE that isn't encoded in your pipeline. Azure Pipelines runs each of the commands you specify in the tasks one after the other in a new process. Examine the logs from the pipelines build to see the exact commands that ran as part of the build. Repeat the same commands in the same order on your development machine to locate the problem.

If you have a mixed solution that includes some .NET Core projects and some .NET Framework projects, you should also use the NuGet task to restore packages specified in packages.config files. Add the MSBuild or Visual Studio Build task to build the .NET Framework projects.

Your builds might fail intermittently while restoring packages: either NuGet.org is having issues or there are networking problems between the Azure data center and NuGet.org. You can explore whether using Azure Artifacts with NuGet.org as an upstream source improves the reliability of your builds, as it's not in our control.

Occasionally, a when new version of the .NET Core SDK or Visual Studio is rolled out, your build might break. For example, a newer version or feature of the NuGet tool is shipped with the SDK could break your build. To isolate this issue, use the .NET Core Tool Installer task to specify the version of the .NET Core SDK used in your build.

Q: Where can I learn more about Azure Artifacts?

A: Package Management in Azure Artifacts

Q: Where can I learn more about .NET Core commands?

A: .NET Core CLI tools

Q: Where can I learn more about running tests in my solution?

A: Unit testing in .NET Core projects

Q: Where can I learn more about tasks?

A: Build and release tasks

Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback .

Submit and view feedback for

Additional resources

COMMENTS

  1. Task Class (System.Threading.Tasks)

    Definition Namespace: System. Threading. Tasks Assembly: System.Runtime.dll Represents an asynchronous operation. C# public class Task : IAsyncResult, IDisposable Inheritance Object Task Derived System. Threading. Tasks. Task<TResult> Implements IAsyncResult IDisposable Remarks

  2. Asynchronous Programming with Async and Await in ASP.NET Core

    These two keywords - async and await - play a key role in asynchronous programming in ASP.NET Core. We use the async keyword in the method declaration and its purpose is to enable the await keyword within that method. So yes, you can't use the await keyword without previously adding the async keyword in the method declaration.

  3. Understanding Task and ValueTask in C#

    Task is a class that contains different methods and properties to manage the state of code execution that will complete in the future. Because Task is a class, every time a method returns a Task an object is created on the heap memory.

  4. A deep-dive into the new Task.WaitAsync() API in .NET 6

    In this post I took an in-depth look at the new Task.WaitAsync () method in .NET 6, exploring how it is implemented using internal types of the BCL. I showed that the WaitAsync ()CancellationPromise<T> instance, which derives from Task<T>, but which supports cancellation and timeouts directly. Finally, I walked through the implementation of ...

  5. Tasks vs Threads in C#

    Why Were Tasks Introduced? Before .NET Framework 4, the only way to do concurrent programming was to use the Thread class. An instance of this class represents a managed thread that the OS internally schedules. Let's see how to create a new thread: double exp1 = 0; var t = new Thread( () => { exp1 = Math.Exp(40); }); t.Start(); // ... t.Join();

  6. c#

    1 I am trying to perform the following scenario: Create multiple tasks all tasks are in the same structure (same code with different parameters) the structure is: try to do, catch if failed, and throw the exception / an exception Run them in parallel and wait for their completion

  7. Task in C# with Examples

    Task in C#. In C#, when we have an asynchronous method, in general, we want to return one of the following data types. Task and Task<T>. ValueTask and ValueTask<T>. We will talk about ValueTask later, Now let us keep the focus on Task. The Task data type represents an asynchronous operation. A task is basically a "promise" that the ...

  8. Task Scheduler in ASP.NET Core

    In this article, we will see a task scheduler in asp.net core. In the realm of building robust web applications with ASP.NET Core, there comes a time when we need to automate certain processes or tasks to run at regular intervals. Whether it's sending out email reminders, updating database records, or performing background computations, a ...

  9. How To Implement Background Tasks In ASP.NET Core

    Hosted services are the primary way to handle background tasks in ASP.NET Core. They are long-running, background processes, seamlessly integrated into the application's lifecycle. Hosted Services Basics. Hosted services in ASP.NET Core are implemented using the IHostedService interface. This interface defines two methods: StartAsync and StopAsync. public interface IHostedService { Task ...

  10. How to Execute Multiple Tasks Asynchronously in C#

    We have made the Web API simple on purpose as we don't want to deviate from the topic of this article by diving into the intricate implementation details of an ASP.NET Core Web API. Execute Multiple Tasks in Sequence using async and await. The client application has an EmployeeProfile class which we use as a presentation model to the end-user:

  11. Run and manage periodic background tasks in ASP.NET Core 6 with C#

    Follow Published in medialesson · 4 min read · Jun 10, 2022 5 Here's the intro from the Microsoft Docs, read more here:...

  12. Implementing ASP.NET Core Background Tasks In Your Project

    ASP.NET Core Background Tasks allow developers to run background operations in a web application. 💡. These operations are long-running, asynchronous tasks separate from the main application thread. IHostedService and BackgroundService are the primary interfaces for implementing background tasks. These are used to create services that run in ...

  13. Parallel.ForEachAsync() and Task.Run() With When.All in C#

    The one and only resource you'll ever need to learn APIs: Ultimate ASP.NET Core Web API ... Using Task.Run() requires caution as it might have some drawbacks. Since it offloads work to a new thread, any time it deals with a large number of tasks it can create a large number of threads, each consuming resources and possibly causing thread pool ...

  14. c#

    1 HangFire, Quartz.Net, Coravel. All come with not just timers but queues with retries, priorities, chaining, dashboards.

  15. Introducing ASP.NET Core metrics and Grafana dashboards in .NET 8

    .NET 8, ASP.NET Core Grafana dashboards and .NET Aspire (preview) are available now. Try using metrics today and let us know what you think: Download the latest .NET 8 release. Use metrics in your tool of choice: Learn more about the .NET Aspire dashboard and get started. Download ASP.NET Core Grafana dashboards. Install dotnet-counters command ...

  16. Build, test, and deploy .NET Core apps

    28 contributors Feedback Azure DevOps Services | Azure DevOps Server 2022 - Azure DevOps Server 2019 | TFS 2018 Use an Azure Pipeline to automatically build, test, and deploy your .NET Core projects. This article shows you how to do the following tasks: Set up your build environment with self-hosted agents.