Avoiding Deadlocks in Optimizely Scheduled jobs using Async/Await

When creating a Scheduled job you have the option to make the job stoppable, but what does it really stop?

Below is an example of a standard stoppable scheduled job, in it I want to use some async methods.  The built-in ScheduledJobBase does not have any async Excecute methods, we can use the stop function but that will only stop the job, not any ongoing calls to services. Also, invoking asynchronous code using GetAwaiter().GetResult() can introduce serious issues such as deadlocks, particularly in contexts with a synchronization context. This pattern forces synchronous execution of an asynchronous operation, blocking the calling thread until completion. If the underlying task lacks a proper timeout or has an excessively long one, the thread can remain blocked indefinitely, potentially leading to thread pool exhaustion or application hangs.

In a previous blog post I wrote about how to avoid common pitfalls like these in Optimizely CMS Validators using async methods. Now I wanted to show you how you can use a similar approach for scheduled jobs.

What we first do is to create a new AsyncScheduledJobBase that will handle the CancellationToken support. In my previous blog post I showed how you can include a timeout setting, but I will skip that here to make the example more readable. So what this will do is to call a cancel request on the jobs Stop call. Using ConfigureAwait(false) prevents the task's continuation from capturing the current synchronization context (such as the UI thread or ASP.NET request context). This helps avoid potential deadlocks in environments where a synchronization context is present.

Even though GetAwaiter().GetResult() blocks the calling thread, combining it with ConfigureAwait(false) ensures the asynchronous operation doesn't depend on the original context. This reduces the risk of blocking critical threads.

An now you can inherit the new async base and implement the ExecuteAsync method taking in a cancellationToken. This token should then be passed into the async methods used.

Summary

In this post, we looked at what it really means for a scheduled job in Optimizely to be "stoppable", and highlighted that it doesn’t actually stop ongoing async operations. We discussed the risks of using asynchronous methods without cancellation support, such as deadlocks and blocked threads, particularly when no proper timeout is set. To address this, we introduced an AsyncScheduledJobBase that supports CancellationToken, enabling both the job and its async calls to be cancelled correctly. By inheriting from this base class and implementing ExecuteAsync, we can now create more reliable and responsive scheduled jobs using proper async patterns.