Async Pages, Async Tasks, and AsyncTimeout

I came to the realization recently that I had oversimplified in my own mind some things regarding the development of asynchronous pages in ASP.NET 2.0 that I thought I'd archive here for future reference, and in the hopes that it might save a few people some trouble down the road.

<% @page Async=“true“ %>

With just one little attribute, ASP.NET 2.0 provides web app developers with a fairly elegant way to implement long-running pages without tying up the thread that ASP.NET dispatched to carry out the processing of the page.  Instead, a page decorated Async='true' can kick off one or more long-running operations, and then return the thread to ASP.NET to be used for other requests; deferring the completion of page processing until the long-running operations have completed and the page has the information it needs to complete the request.

Related to the the development of asynchronous pages is the mechanism that the developer uses to kick off one or more asynchonous operations when their page initially loads.  Options are numerous, and include calling the AddOnPreRenderCompleteAsync or RegisterAsyncTask methods of your page instance, calling an XxxAsync method of a component that supports the event-based asynchronous pattern, or any number of other asynchronous execution techniques.

The discussion that follows focuses on one particular mechanism - async tasks - but assumes that you're already familiar with the basics of asynchronous page development.  If you aren't, consider checking out the following resources before revisiting this page and reading further:

• Fritz's blog is, generally speaking, a must-read blog for all things ASP.NET.  But in particular, be sure to check out his posts on asynchronous pages (as well as his follow up).  Also be sure to read his post on asynchronous tasks (and the follow up to that post).

• Jeff Prosise has an article on asynchronous pages in MSDN magazine that provides a good overview of how async pages work.

• Dmitry Robsman, development manager for ASP.NET and IIS at Microsoft, gave a talk on asynchronous page development at TechEd Europe 2005.  He posted links to his presentation and accompanying sample code here.  As an aside, there's a gem of a class in his samples named CompositeAsyncResult that provides a great way to aggregate the completion of multiple IAsyncResult-related asynchronous operations into a single IAsyncResult implementation that's given to ASP.NET, and that doesn't notify ASP.NET of operation completion until all of the individual asynchronous operations you started have completed.  Be sure to check it out.  That said, I have one warning about one of the slides in that presentation that I'll refer to below.

<% @page Async="true" AsyncTimeout="5" %>

One of the features of asynchronous pages is the optional AsyncTimeout attribute on the @page directive.  This attribute specifies the maximum wall clock time that a request for your page is allowed to take, expressed in terms of seconds.  The declaration above, for example, tells ASP.NET that if you haven't completed the asynchronous operations you kicked off when the page was loaded within 5 seconds, they have your permission to go ahead and complete the page processing request.  If this happens, ASP.NET will resume the page processing steps that were suspended after the PreRender event fired, beginning with the PreRenderComplete event.

On the surface, that seems straight forward enough.  But it was only recently that I realized I'd glossed over the implications of using this attribute in my own mind, and that I'd like to mention here by way of warning/documentation.

AsyncTimeout Specifies the Total Time Budget for the Page

The first thing I wanted to clarify is that the AsyncTimeout attribute specifies the total time budget, so to speak, for your page.  It does not specify a per-task or per-async-operation timeout.  I only mention this because slide 31 of Dmitry's slides at TechEd seems to state the opposite.  On that slide, he says that the AsyncTimeout attribute “Allows for timeouts of individual tasks. Failure of one asynchronous call is not fatal to the page.”  To me, that wording means that if you kick off more than one PageAsyncTask, each will have the specified amount of time to complete.  But that's not the case.

Consider, for example, a page that's annotated AsyncTimeout="5", and which calls RegisterAsyncTask 3 times to perform operations that each take 3 seconds to complete.  If you pass true for the executeInParallel parameter, then the timeline of your page will look something like this:

T+0sec: RegisterAsyncTask called 3 times
T+0sec: First task's begin event handler called
T+0sec: Second task's begin event handler called
T+0sec: Third task's begin event handler called
T+3sec: First task's end event handler called
T+3sec: Second task's end event handler called
T+3sec: Third task's end event handler called
T+3sec: Page processing resumes & completes

Note that the begin event handler is called for all 3 registered tasks right away because executeInParallel was set to true.  Each handler then kicked off whatever it wanted to do that took 3 seconds in this scenario.  Roughly 3 seconds later, the end event handler for all 3 tasks is executed, afterwhich the page completes its execution just fine.  This was because all 3 tasks completed before the 5 second page timeout elapsed.

But here's what's going to happen if you pass executeInParallel=false to RegisterAsyncTask (which is the default if you don't specify this parameter explicitly):

T+0sec: RegisterAsyncTask called 3 times
T+0sec: First task's begin event handler called
T+3sec: First task's end event handler called
T+3sec: Second task's begin event handler called
T+5sec: Second task's timeout event handler called
T+5sec: Third task's begin event handler called
T+5sec: Third task's timeout event handler called

Note that the first task took 3 seconds to complete.  And because executeInParallel was false, the second task wasn't started until this time.  But by now, 3 of the 5 seconds budgeted for the page have been used up.  As a result, the second task had its timeout handler, not its end handler, invoked 2 seconds later.  The third tasks didn't get a chance to run to completion for the same reason.  So clearly the AsyncTimeout attribute specifies the total time budget for the page to be processing, not a per-task timeout as Dmitry's slides seem to indicate.

That said, I'm fairly certain Dmitry himself doesn't think that the timeout is per-task :-)  It's just poor wording on a slide.  But since a lot of people have probably seen those slides, and a lot of those people will probably read that bullet (like I did) to mean that AsyncTimeout specifies a a per-task timeout, I wanted to post a clarification here.

The Begin Event Handler is Always Invoked

The second impliciation of AsyncTimeout that I had not really internalized until recently is that the begin event handler for a registered async task is always invoked, even if the page has timed out before ASP.NET gets around to starting that task.  This can be seen in the timeline above.  Note that the second async task got started after the first task completed.  Since this was 3 seconds into the 5 second time budget, this is expected.  And since the task was started with only 2 seconds of page life left, it couldn't complete in time and had its timeout event handler invoked.  But note that the begin event handler for the third task is also called, even though by now the page has already timed out.  This is a subtlety that might trip you up if you're not expecting it.

In this scenario, the remainder of the tasks are basically doomed.  The page has timed out already, but ASP.NET is still going to go through the motions of calling the begin event handler for all registered tasks, followed immediately by a call to their corresponding timeout event handlers.  I have not been able to find any kind of HasTimedOut property on the Page, HttpContext, or HttpApplication classes, so it would probably be a good idea to have all of your timeout handlers set a flag somewhere that lets all of your task begin event handlers know that the page has already timed out.  This will allow you to forego kicking off an async operation, only to have to deal with the impending timeout event handler notification that's going to happen immediately afterwards.  This is especially important if the async operation you're kicking off doesn't support cancellation.

AsyncTimeout Does Not Imply Asynchronous Task Cancellation

This leads to the third thing I wanted to mention.  Just because ASP.NET has decided to timeout the processing of a page, this doesn't mean that the asynchronous operations you kicked off from within your task start event handler(s) are going to be magically cancelled.  That's what your timeout event handler is supposed to do.  This probably seems obvious as you read this, but I can pretty well guarantee that lots of people won't be thinking along these lines when they code up an asynchronous page.  They'll set AsyncPage="true", set AsyncTimeout="5" (or something), and then kick off one or more asynchronous operations using RegisterAsyncTask, passing null for the timeout event handler (which is supported).  Those people won't even know that the page has timed out.  But when the asynchronous operation they kicked off eventually does complete, the page processing may have been completed for quite some time already - leading to undefined results.

Obviously, the intent of the timeout event handler is that you'd take whatever steps are appropriate to cancel the operation you'd previously started.  If, for example, you had called the XxxAsync method of a component that supported the event-based asynchronous pattern, then your timeout handler would call the corresponding XxxCancelAsync method.

But lots of asynchronous technologies, including asynchronous delegates, the pre-2.0 BeginXxx style of asynchronous web service invocation, and Stream-based asynchronous I/O, do not support cancellation.  So be sure to think carefully about how you're going to deal with some subsequent "the operation has completed" notification from one of those technologies that arrives after your timeout handler has been invoked and page processing has been completed.  If you don't have a way to pull the plug on the operation explicitly from within your timeout event handler, be sure to at least set a flag somewhere that can be consulted when your async operation eventually does complete.  There's no telling what will happen if you fail to realize the page has already timed out and proceed to manipulate Page/Context/Request/Response/etc state long after ASP.NET completed processing that page.

Page Processing Does Not Resume Until All Timeout Handlers Return

The fourth implication of using the AsyncTimeout attribute on a page that I wanted to point out is that the page processing that was suspended between the PreRender and PreRenderComplete events does not resume until all of the timeout event handlers for all of the registered async tasks have been invoked and return.  So be careful about doing things in your timeout event handlers that further postpone (potentially indefinitely) the completion of the page.

In other words, it's possible for a timeout event handler to hang up completion of the page long beyond what you'd intended to happen by setting AsyncTimeout to something like 5 seconds; potentially to the point that the client browser gives up and times out the request 1 or 2 minutes later.

AsyncTimeout is Implemented Using a Timer

And finally, here's one that Fritz pointed out to me.  I mention it here only because he hasn't yet, and it's part of this overall topic.  In practice, I doubt this will be an issue.  It's more of a pathological case that, if it ever does happen, means you have bigger problems than the issue I'm pointing out.

Basically, the timeout handling logic for an async page is implemented by means of a System.Threading.Timer object.  When async tasks are started, ASP.NET starts one of these timers if the page has an AsyncTimeout attribute.  If page processing completes before the specified timeout elapses, great.  But if the timer callback is invoked, ASP.NET starts the timeout handling ball rolling that was discussed above.  The problem is that a thread pool thread is used to invoke timer callbacks.  So if it happens that there are no thread pool threads available, and the thread pool thread count limit has already been reached, then the timer callback will be postponed until such time as a pooled thread is freed up.

Again, this is a pathological case.  Like I said, if you run into this problem, you have much bigger problems on your hands :-)  But it was worth pointing out since it's related to the overall discussion.

Conclusion

Don't get me wrong.  I've really enjoyed delving into the asynchronous component/context architecture supported by CLR 2.0.  And I really appreciate what the ASP.NET 2.0 team has done to facilitiate the development of long-running pages.  But I've only recently realized that I'd made several oversimplifications in my mind along the way that would have led to subtle issues down the road in development.  Hopefully the information here will prevent you from making the same mistakes.

(UPDATE: just proof reading fixes I should have made before posting :-)