When I started to help Luca with feng I hadn’t worked on server software for many years; while server software was what I started working on when I started seriously working on collaborative free software projects, it really was much lower profile, yet I see that some of the things I did there seem to reflect here.
One of this is my (bad) habit of rewriting lots of parts together when I see that something does not really comply with the specifications; in this case while trying to make the parser more robust, I found that the whole thing works out of luck and started rewriting it again. The nice part is that in two days I wrote (again) the whole RTSP parser (before I only rewrote the actual parser but not the logic that reads the data), with much less lines of code, with per-client worker threads, with SCTP support and so on.
The problem now is, though, that we haven’t been able to release a stable 2.0 release yet, and since 2.0 was supposedly still using bufferpool, we’re probably just going to skip it and go with 2.1; on the other hand, the worker threads are still experimental and should not be in 2.1 either, but rather in 2.2 (I need the new parser to implement the proxy-passthrough used by QuickTime and QTSS, that encapsulates RTSP and RTP over HTTP). So the worker threads will be postponed and looked at in the future; on the other hand, the new parser was also designed to solve a series of security concerns I had with the previous parser code.
Even more complex, the message parser I wrote as my first rewrite of the request parsing and handling was split off in a separate library (related to this earlier idea I had), but the new parser is much more tied to feng’s code and cannot be easily shared directly, so the library has to go away. But does it make any sense to have a 2.1 release using a new library that the 2.2 release will blow away? Nope, so there I go merging it back.
And since I’m at that point, and merging is hard.. I’m basically working on making sure that all the non-feature non-rewrite changes that we got are merged in for 2.1, so that the walk toward 2.2 is a cakewalk. Not an easy thing but I should be able to pull it off with not an overly complex work.
The other problem is implementing IPC between two components of the lscube stack, flux and feng, to provide live streaming support. Now this is trickier because we’re actually mixing different programming languages: flux is written in C++ while feng is written in good old C. The idea for now is to use POSIX message queues to send data from one to the other, but here comes the problem: earlier the IPC mechanism was provided by the(pretty broken) bufferpool library, and thus lied away from either feng or flux; now that library is no longer present, which means we’ll have to find another way to deal with it. I’d probably look into installing some development header and libraries for feng.
I also have to design a stress-test suite to make sure that there are no buffer overflow in the old parser, which is not an easy thing to do actually; thinking outside of the box, which is what is needed for security testing, is not exactly what I do to have a standard-compliant server.
Oh yeah and there is the problem of standards. While rewriting the seeking support for feng with Luca, we discovered that either live555 or VLC are pretty stupid when it comes to seeking, and send multiple PLAY requests instead of a series of PAUSE/PLAY pairs, which in RTSP actually have a very different meaning as they create an edit list.
And all what I described up to now only relates to RTSP/1.0, not even to extensions (beside the QT/QTSS I noted before), nor to RTSP/2.0. Seems like we still have a long road ahead before even completing the basic features, or being standard compliant (did I say that we also don’t send a Connection: Close header when we’re going to close the RTSP connection?). Alas.
On the other hand I have to say that the experience is proving very interesting, and I guess that once the server part is decently ready and we can start focusing on the client part, I might be able not only to get xine in a state where the RTSP code does not decide to security mess-up every other year, and it actually works with more than just Real’s own servers.
