Qhttpd is a Linux specific solution. Why? Well, Linux and the Linux community have helped me out a lot over the years, and this is one way of repaying that. The reality though is that one of the goals of this solution is optimizing for performance. The more assumptions I can make, the better my optimizations will be (in theory at least).
This is not to say that the solution will be deliberately non-portable. It's just whenever I have a choice between portability and speed, or portability and ease of implementation, I'll choose the latter. I figure someone else can worry about portability at a later date.
Why an HTTP daemon? Well, first of all, lots of people are currently trying to optimize HTTP daemons. So, this allows me to compare my approach with other possible techniques very easily. Secondly, HTTP daemons are a fairly pervasive need, so making one really fast would have a fairly significant benefit.
Much like the previous point though, there's no reason this solution can't be redone to support other protocols. Indeed, Dan Kegel is working on an FTP benchmark, and it makes sense (since we're in the same city) to take advantage of that by making an FTP interface. Also, there's a batch of fresh cookies in it for me if I'm the fastest ;-). The design of this system means that only one small part of it is really focused on the protocol issues, so adding something like FTP should be easy.
Fundamentally, Qhttpd is about applying the principles of queuing as used by many middleware products like MQSeries and Tuxedo. The main idea is to represent requests as entries in a queue, as opposed to separate processes or threads. This allows most operations in the system to be done asynchronously. In theory this allows many optimizations.
As a general rule, I prefer doing OO programming. It's what I'm used to... It's how I think. I'm not sure I can write properly structured procedural code anymore. ;-) My primary programming language of choice is Java. Unfortunately, Java's support for asynchronous I/O is less than stellar, and Java's JNI interface is very slow (it would easily eliminate the benefits of async I/O IMHO). So, this lead me to C++.
I am not a C++ guru. Indeed, it's been a while since I've done any C++ programming. So, don't be surprised if the code is not as elegant or as sophisticated as it could be.
There are a variety of options for doing I/O. I refuse to use per-client threads or processes on the grounds that this adds overhead that queuing is supposed to eliminate. The main other alternative is SIGIO. In the future, I may very well incorporate SIGIO into Qhttpd, however, there are two really good reasons not to: 1) SIGIO is being examined by lots of other people, and I doubt I could shed additional light on it; 2) AIO has a nice interface for "batching up" I/O requests (aio_listio()) which more closely matches what I'm trying to do with queuing here.
Now that I've played around with AIO for a while, I have come to realize that one of it's biggest advantages is the support for a synchronization free "seek-and-read/write" operation. This of course, is probably of more benefit to databases then web servers. However, it does allow me to, for the moment, ignore the implementation of a cache, as the benefit of the cache will be smaller than I'd have otherwised guess (although benchmarks will be the one, true, answer.
Once I AIO is the interface of choice, there are only two real implementation options: glibc AIO and SGI's KAIO. I evaluated glibc's AIO, and it is broken and as best I can tell no faster than writing userland code to do the job yourself. SGI's KAIO works very well, and appears upon preliminary examination to provide meaningful performance optimizations.
The key to getting good performance is metrics. You need to measure every aspect of a system as it's running to determine where the problems are and to prioritize optimization efforts.. This system will integrate logging support into all areas of it's operations. Particular metrics of interest are:
HTTP logging can be built from this framework a well. The whole logging structure will be built around the queues in hopes of minimizing overhead. This results in delayed statistics, but hopefully very accurate statistics which can be obtained with minimal intrusion to the performance of the system. Finally, it should be possible to avoid any overhead due to logging by simply disabling the feature at compile time.
A lot of time can be spent switching between the kernel and userspace and vice-versa. Such transitions are necessary, but the idea should be to put as much information as possible into each transition so that you minimize how frequently they must occur. It probably makes sense to track syscalls.
Ok, so the key to this whole system is the queuing component. I'm planning on building this around STL's existing container libraries, probably based on a queue. I still haven't decided whether to make this a publish/subscribe messaging system, or a point-to-point messaging system. The former makes things like performance profiling and logging simpler to design, but the latter is simpler to implement in it's own right, and probably allows for better performance as well. Filtering may be an easier to manage concept if you use point-to-point.
The ProtocolAdapter listens for requests. It's responsible for managing any listen sockets as well as the expected plethora of sockets needed to talk to the clients. It will use KAIO to detect incoming messages. It will then do whatever minimal parsing of the messages that are necessary and then post a request to the queue.
The FileSystemAdapter blocks on the queue, waiting for requests. When it sees requests in the queue, it will pull off a chunk of them, and then fire off a request through KAIO to fullfil all of the requests. Upon completion it will enqueue the results to the messaging system.
The ProtocolAdapter will check the queue for results. When it finds results it will pull them off in bulk, and build up the appropriate protocol response. It will send this response down to the client through KAIO.
The hope is that this code base can mature into the fastest HTTP daemon available on Linux. My first goal will be to out perform/out scale Apache, which is already a very difficult challenge. I have few allusions about this strategy. It may very well prove that Apache's developers can keep ahead of me primarily by having more man hours to apply to the job. It's also quite possible the various SIGIO based projects will prove to be superior designs than what can be achieve through AIO. Indeed, in the long run I suspect a mix-and-match of various different strategies will result in the best performance
NEW! Ok, scratch that. I should be able to out perform SIGIO based solutions in cases of heavy load. Why? I can use realtime signals to be notified of completion, but with lio_listio() each signal will represent the completion of several IO's. In theory, this should reduce overhead. I have to look at realtime signals more closely to make sure thiscould actually work.
First of all, these are just listed here. They will not necessarily be incorporated right away (indeed, to a large degree it's probably better if I don't incorporate them at first), because my first goal is to get this up and running and get some metrics. However, this is a nice little white board to brainstorm ideas.