High peformance TCP server using completion ports

Completion ports were first introduced in Windows NT 4.0. This technology makes simultaneous asynchronous I/O possible and extremely effective. When building high performance network software one has to think of effective threading model. Having too many or too little server threads in the system can result in poor server performance.

The goal of a server is to incur as few context switches as possible by having its threads avoid unnecessary blocking, while at the same time maximizing parallelism by using multiple threads. The ideal is for there to be a thread actively servicing a client request on every processor and for those threads not to block if there are additional requests waiting when they complete a request. For this to work correctly however, there must be a way for the application to activate another thread when one processing a client request blocks on I/O (like when it reads from a file as part of the processing). (read more...)

Smart reader may admit that I/O completion ports (IOCP) are not directly available in .NET. Well, that was pure truth until SP1 of .NET 2.0. From .NET 2.0 SP1 this marvelous technology can be accessed using following Socket class methods:

• AcceptAsync
• ConnectAsync
• DisconnectAsync
• ReceiveAsync
• SendAsync
• and other XxxAsync methods

In one of my previous posts about receiving variable length messages I used BeginXXX/EndXXX asynchronous approach. Main drawback of this approach is presence of the repeated allocation and synchronization of objects during high-volume asynchronous socket I/O. That is because BeginXXX/EndXXX design pattern currently implemented by the System.Net.Sockets.Socket class requires a System.IAsyncResult object be allocated for each asynchronous socket operation.

Completion port approach on the other hands avoids the above mentioned problems altogether. Asynchronous operations are described by instances of SocketAsyncEventArgs. These objects can be reused by the application, more over, application can create as many SocketAsyncEventArgs objects that it needs to perform well under sustainable load.

The pattern for performing an asynchronous socket operation with this class consists of the following steps:

1. Allocate a new SocketAsyncEventArgs context object, or get a free one from an application defined pool
2. Set properties on the context object to the operation about to be performed (the completion callback method, the data buffer, the offset into the buffer, and the maximum amount of data to transfer, for example).
3. Call the appropriate socket method (XxxAsync) to initiate the asynchronous operation
4. If the asynchronous socket method (XxxAsync) returns true, in the callback, query the context properties for completion status
5. If the asynchronous socket method (XxxAsync) returns false, the operation completed synchronously. The context properties may be queried for the operation result
6. Reuse the context for another operation, put it back in the pool, or discard it

In How to Transfer Fixed Sized Data With Async Sockets I've presented server code that uses BeginXXX/EndXXX model for data receive handling. In the next post I'll show how that code can be rewritten to use IOCP server model.
Sample code of IOCP based TCP server