Programming for fun

I've been in a hiatus from blogging lately, mainly because free time has been in short supply. It's not just the very busy days at work, nights have been even busier. Instead of churning out long blog posts, I've been banging on code instead, day and night. Unfortunately I only get paid for the code I write during the day, but surprisingly (or not) I enjoy the code I write at night the most.

I have written code for banks, multinational corporations, the kind of places where the phrase "professional dress code" is not used strictly in jokes. I've helped build software that was resilient, scalable, secure and intuitive, although granted, not all those at the same time. This was mainly Custom Enterprise Software, or to put it more plainly, Software That Runs On Computers At Work. So I thought I'd try a different angle in my spare time: write software that people use at work, or at home. Or at a coffee shop. Or at the airport. Or wherever they feel like using a computer, anyway. This is sometimes called End-User Software, or to put it in layman's terms, Software People May Actually Like. After trying out some ideas that didn't seem worthwhile, I discovered that some of my buddies had similar thoughts. This is how FireStatus was born.

FireStatus is a Firefox extension that aims to be a swiss army knife for dealing with various social networks, right from your browser, without visiting any particular website. Facebook, Twitter and FriendFeed are currently supported, but more are in the pipeline. For starters, it allows you to simultaneously update your status to all or some of these services, so that all your friends see it, no matter what they are using. The notion of a status that is occasionally updated is familiar to Twitter and Facebook users, since the text field that asks 'What are you doing?' is prominent in their user pages. FriendFeed does not have the notion of a status, but its users can post short (or long) messages, just like Twitter's. FriendFeed also allows posting links to web pages, accompanied with a short description, something that many Twitter and Facebook users have been doing by constructing status updates that start with the description text and are followed by an appended link, usually shortened.

FireStatus can ease the task of posting these messages or status updates, by being always available, instead of needing to have the service pages open and without a large memory footprint, like other similar applications, since it takes advantage of the fact that most people nowadays always have one browser open. I know I do. Come to think of it, I don't know anyone who uses computers that does not keep a browser running most of the time. At least untile the darned thing crashes, which fortunately is something I haven't experienced in Firefox for quite some time.

Posting a message or status update

Clicking on the FireStatus icon, pops up a small toolbar window just above the status bar and below the window document. It is similar to the Firefox findbar that pops up when one searches for text in a page, albeit slightly larger. That was a deliberate design decision that aimed to imitate the success in the usability of the findbar. Lots of little details like this one have been carefully thought out and occasionally debated at length among the team and our beta testers:

• Enabling the spell checker in the status message field, for catching those typos when hastily typing a message.
  
• Having the URL inclusion unchecked by default, while the shortening checked, since most posts do not include URLs (and it might be embarrassing if done inadvertently), but those that do usually want them short.
• Adding a character counter to help guard against the maximum message length imposed by some services (e.g. Twitter).
• Tightly placing the available services in a way that allows for an unambiguous selection.
• Using the Escape and Enter keys as shortcuts for canceling and sending the update respectively.

As one reviewer in addons.mozilla.org so eloquently put it, "the simplicity & minimal design are the key components here".

Posting is not the whole story, though. These services provide continuous streams of updates from friends that we need to monitor. Sifting through the updates in every service, while very useful, becomes tedious after a while. It is like making a mental note to check your e-mail every once in a while for new messages. It's one way to do it for sure, but mail notifiers have been around for ages and provide more of a "live" experience.

FireStatus imitates their success in making e-mail conversations "alive", for conversations in the social network space. Every time a new incoming update is received, a notification popup appears, so I can continue using whatever application I was using, but being instantly aware of the news. Twitter notifications contain the name and the picture of the message author as well as the message sent. Low-priority or uninteresting messages can be just glanced at and then automatically dismissed, while important ones can be clicked on, so that the message can be viewed in the browser. FriendFeed notifications can be of various flavors, since FriendFeed aggregates updates from a large variety of online services.

Therefore FriendFeed updates display the service icon along with the author's name for quick identification of the type of update. Clicking on an interesting update opens a browser window to the link contained in the message. Facebook notifications currently contain new message counts, pokes and shares, but work for getting more data is underway.

Although the notifications appear instantaneous, as with e-mail notifications there is a polling process involved underneath. The polling frequency for each service can be separately tuned in the preferences window. The preference window can be opened by right-clicking on the FireStatus icon and selecting the appropriate option in the popup menu. Along with the time intervals between polling for updates, one can enable posting and/or receiving updates for the various services, as well as the authentication credentials where appropriate. For Facebook the user logs in in a Facebook-supplied browser window that pops up, similar to regular Facebook use. No credentials need to be stored separately by FireStatus. For FriendFeed the username and remote key have to be stored locally in the extension preferences. The remote key is not the same as the user password, but is provided by FriendFeed specifically for use by third-party applications, like ours. It can be found by clicking on the link displayed in the explanatory note. Twitter can work either way. If a username and password are entered in the preference window, they will be stored locally in the extension preferences and used for subsequent authentication. If, on the other hand, the fields are blank, FireStatus will consult the browser's Password Manager for any stored Twitter credentials. This will probably cause the master password dialog to pop up once, but the credentials will remain stored only in the Password Manager.

FireStatus preferences

Since we built this thing for our own personal use, pleasure and self-punishment, and as we don't plan to ever get rich out of it, we have released it from the beginning as open-source software through the permissive BSD license. You are most welcome to come by the project's home at Google Code, study the code, fix a bug (or two), help with the (embarrassingly outdated) documentation, contribute a review (otherwise it will remain an experimental extension forever) or just chat with us at the mailing list. Heck, if you feel like it, you can grab the whole code and use it in your own lucrative business and you don't owe us a dime.

With the shape the world economy currently is in, you probably couldn't afford it, anyway.

Murdered by Numbers

"JavaScript has a single number type. Internally, it is represented as 64-bit floating point, the same as Java's double. Unlike most other programming languages, there is no separate integer type, so 1 and 1.0 are the same value. This is significant convenience because problems of overflow in short integers are completely avoided, and all you need to know about a number is that it is a number. A large class of numeric type errors is avoided."

Douglas Crockford, JavaScript: The Good Parts

Bitscuits © Barney Livingston / CC

When I was twenty-something, I liked watching The Young Indiana Jones Chronicles. Besides the regular stuff that your average Indy fan loves, I was particularly fond of Indy's apparently insatiable appetite for traveling and learning different languages. So while I was contemplating what it would take for me to follow in his footsteps, I figured I'd better start with learning foreign languages. By that time I had already got my English certificate and I had a few years of learning French under my belt that had at least provided me with some means to court women (with little success, regrettably). So I started learning Italian, which everyone said was easy to pick up, especially if you had already mastered some other foreign language. That turned out to be true, and I managed to get a degree in Italian after two years of intensive studies. What made that period frustrating (and occasionally funny) however, were the times that I would inadvertently mix all three languages in the same sentence, creating my own version of Esperanto. Due to the similarities among them, I might be trying to speak Italian, but use an English noun while constructing the past tense of a verb in French. Sometimes I would even be oblivious to my mistake until someone else pointed it out to me. It all seemed very natural as I was doing it.

Lately I'm having a déjà vu, when I find myself coding in Java, JavaScript and C++, often in the same day. More than once I tried to initialize a Java object using a JavaScript object literal. Sadly, the compiler was not very accommodating. While writing a GWT-based front-end, I often transmitted POJOs without being aware that a long value was silently converted to a JavaScript Number, which essentially amounts to a Java double. Reading David Flanagan's "Rhino" book had already left me with the impression that JavaScript was rather flawed for not having separate types for integers, bytes, chars and all the other goodies that languages like C/C++ and Java spoil us with. But after getting a copy of Douglas Crockford recent, highly opinionated, "Good parts" book, his argument resonated with me: "A large class of numeric type errors is avoided." It's Richard Gabriel's "Worse Is Better" principle, or alternatively "Less Is More", in new clothes. Recent events made sure that the message was permanently bolted in my brain.

I've been writing a desktop application in Java that communicates with various Bluetooth and USB devices. The communication protocol is some sort of terminal-like commands and responses that initiate at the Java application and travel through a thin JNI layer down to the C/C++ device driver, and then to the actual device. The protocol documentation describes in... er, broad terms, the sequences of bytes that constitute the various requests and responses. Suffice it to say that my system administrator's experience in sniffing and deciphering network packets proved invaluable.

Sending the command was easy (or so I thought): fill a byte[] array with the right numbers and flush it through the JNI layer. There we get the jbyteArray and put it inside an unsigned char array, which is later send through the device driver to the actual device. When receiving responses the sequence was reversed. It all seemed to work fine for quite some time, until suddenly I discovered that one particular command caused the device to misbehave. I couldn't be sure if the device was faulty or my code was buggy, but since I had zero chances of proving the former, I focused on investigating the latter. A couple of days of debugging later I was still on square one, since as far as I could tell the command reached the device unscathed. Logic says that if a fine command reaches a fine device, then one would be entitled to a fine response. Since I wasn't getting it, I began questioning my assumptions.

I resisted the urge to blame the device, since I couldn't prove it conclusively, and started blaming the command. There was definitely something fishy about it, and to be honest, I had a bad feeling all along. The other commands were simple sequences, like:

0x14, 0x18, 0x1a, 0x3e

or

0x13, 0x17, 0x19, 0x26

This particular one however, was icky:

0x11, 0x15, 0x17, 0xf0

If you can't see the ickiness (and I won't blame you), let me help you.

Java's byte type is a signed 8-bit integer. C++'s unsigned char type is an unsigned 8-bit integer (at least in 32-bit Windows). Therefore we can represent values from -128 to 127 in Java and values from 0 to 255 in C. So, if you have a value between 128 and 255, ickiness ensues. 0xf0 is, you guessed it, between 128 and 255. It is 240 to be precise, if Windows Calculator is to be trusted.

Now, of course I am not that dumb. I knew that you can't assign 0xf0 to a byte in Java, so I had already made the conversion. You see, what is actually transmitted is a sequence of 8 bits. If you get the sequence right, it will reach it's destination no matter what. When you convert 0xf0 to bits you get 11110000. The first bit is the sign bit, which is causing all the trouble. If it was zero instead, we would be dealing with 1110000, or 0x70, or 112 if you're into decimal numbers.

So that's what I had done. I'd constructed the negative version of 112 and used that to fill my command buffer:

0x11, 0x15, 0x17, -112

Looking at it made me feel a bit uneasy, without being able to explain why. I used to think it was the mixed hex and decimal numbers. Yeah, I'm weird like that. However, the zillionth time I reread the Java's byte type definition, a lightbulb lit up over my head. I actually paid attention to the words in front of me: "The byte data type is an 8-bit signed two's complement integer".

Sure, it's 8 bits wide and yes it's signed, using the most common representation for negative binary numbers, two's complement. What's new here? I know how to negate in two's complem...

Whoa! Wait a minute. What I just described above isn't how you negate a number in two's complement. It's actually how you do it in the sign-and-magnitude variant. In two's complement you invert the bits and add one to the result:

11110000 -> 00001111 -> 00010000 or 16, or 0x10

Yep, 16, not 112. So the proper command sequence becomes

0x11, 0x15, 0x17, -16

and, yes, the device seems quite happy with that. As happy as devices get, that is.

So, basically, I wasted many many hours and suffered innumerable hair-pulling episodes for falling prey to a numeric type error. The kind Doug Crockford alludes to in his book. Now, don't get me wrong, I like mucking with bits as much as the next guy. But had I been living solely in JavaScript-land, with the single number type, I'd feel less tired and probably not hate my work as much. Though, granted, I might be needing a haircut right now.

Why you should go to conferences

One thing I like about working in a large corporation is that you get to meet all sorts of interesting people. You actually have to meet them, since you can't get anything done without coordinating with a whole bunch of them. But since this tends to get nothing done most of the time, I chose to work in a small company instead. Here if you want to do anything, you probably have to do it yourself (there are some interesting repercussions of this that I will leave for a future post). Now, if you work in a small company and want to meet lots of interesting people, you basically have three two options:

1. Use the net - blogs, mailing lists, newsgroups, twitter, friendfeed, etc. (done that)
2. Attend conferences.
3. Go to parties.

Now as you can personally attest to, I've got No.1 covered. So the last couple of days I worked on number 2 instead. The nice guys at ellak.gr organized the 3rd FLOSS conference and since it took place very close to where I work, I took the opportunity to avoid working meet some old friends and make some new ones.

Number One of course, is Wietse Venema. Since his work was an inspiration for me to pursue a PhD in computer security, I just had to see him in person once. He was more fun that I expected, to be honest. Security guys have a reputation of being a wee bit paranoid and itsy-bitsy abrupt on normal people, not to mention complete strangers. Wietse however was neither. This probably has to do with his academic background or maybe the fact that he is Dutch, who knows. He proved to be a warm, funny guy instead, who runs FreeBSD on his laptop (that always shows character in my book).

His talk was about Open Source and Security, both matters dear to my heart. The discussion of Postfix was a walk down memory lane for me. I can still remember the excitement around its initial release: "Hooray! No more sendmail exploits!". Wietse briefly discussed the architecture of postfix and how it relates to its security. He also showed some statistics comparing postfix to sendmail and qmail that I had seen in the past, but was curious to where everyone currently stands. Apparently postfix holds the second place in number of deployments (good), the first in number of lines of code (bad), but that is caused by having reached sendmail's feature list, without compromising the original architectural goals (good). For the problem of security at large, the most provocative suggestion was to make software development too hard for the laymen, so only experts could do it. Heh. As John Wayne so eloquently used to put it: "that'll be the day".

Lots of comparisons were presented in the talk by Diomidis Spinellis, titled A Tale of Four Kernels. The talk covered a code quality comparison of four major commercial and open-source operating systems, Windows, Solaris, Linux and FreeBSD. Diomidis is a well known author and FreeBSD commiter, and that was evident in the quality of his work and presentation. The room was packed, and the Q&A session lasted almost as long as the presentation itself. I suppose the sensitivity of the subject had something to do with it. Diomidis went through heaps of information so fast you could barely comprehend each slide, unless you had the forethought to study his paper before the conference, like your truly did. Although he was very delicate in his conclusions, stating that all systems had comparable quality, there were some members of the audience that seemed to have already made up their minds and didn't care for diplomacy.

Another great talk was by Jim Blandy. The talk was about Open Source Infrastructure for Software Development, which covered the evolution of version control systems (VCS), from SCCS, to the recent crop of distributed VCS. Being one of the people who gave us CVS and Subversion, his recollection of the history of this area was very exciting. Besides sharing some colorful historical tidbits, he also gave a meaningful comparison of the algorithms and data structures used by the various systems. He gave high marks to Subversion for being a fine choice in a centralized organization structure, but eloquently presented the fundamental paradigm shift that distributed VCS present. He said (as best as I can remember) that controlling access to the commit step (what all centralized VCS do) is the wrong (worst?) place to do it, since adding a change is not that big a deal. Instead the point of merging is what should be guarded, since that is when another changeset gets incorporated in our code. The fact that distributed VCS replicate whole repositories, not just working copies, makes allowing commits a natural thing, while making merge selection part of the release engineering process. I've read many pro-distributed-VCS arguments, but I found Jim's focus on the social aspect of the merge process a refreshing one. Luckily, most of my cohorts were present in this talk, so switching our team to Mercurial might not be just a dream after all.

Jim currently works on Mozilla, on the new ActionMonkey JavaScript engine. Since that work was outside the scope of this talk, I managed to corner Jim afterwards to find out more about its current status. The project attempts to integrate the current Mozilla JavaScript engine, SpiderMonkey, with the Tamarin engine contributed by Adobe. The plan was to replace various parts of the old engine with the new ones, like the garbage collector, the JIT compiler, etc. However doing so leads to performance regressions in some areas and Jim said that one lesson they have learned is that if you ship with worse performance, no matter how rare the circumstances that expose it, you will pay for it at some point. String manipulation appears to be one of those areas, since the old engine had been carefully optimized during all this time and achieving the same performance with the new one, requires work. I have been wondering for some time now whether SpiderMonkey uses some variation of the Ropes data structure for String representation, but Jim explained to me that was not the case.

He was also very excited about the experimental tracing JIT that they are working on, that appears to be a perfect fit for dynamic languages and is expected to boost JavaScript performance on future versions of the browser. I told him that I had already heard of it from Steve Yegge, and I've been meaning to read the literature Real Soon Now(TM), which I swear is the truth, the whole truth and nothing but the truth. After I finish this belated post of course. Jim said that Steve is a great story-teller, which means that Steve, if you ever find yourself in Athens and want a free dinner, just let me know. Explaining the benefits of tracing JIT led us to a detour into LISP-land and while Jim has written a Scheme compiler, I've only done LISP programming for a class as a student. Which is another way of saying I didn't understand a word he said, alas. Anyway, the cool things they are working on will probably come to us in firefox 4, so I still have time to get up to speed.

All in all, the conference was quite an enjoyable experience. Old friends, new friends, free food, stimulating discussions, what's more to ask?

What? To be on TV? Hey, that's easy. You just have to hang outside the main conference room chatting with random people, until you notice a cameraman and a journalist taking interviews from conference organizers and speakers. Then you make sure to be in the camera frame and voilà! You've got your 3 seconds of fame.

JavaScript on the server with Helma

Rumor has it that the most widely deployed programming language is JavaScript, due to being embedded in every contemporary browser. This extraordinary feat has obscured the fact that the language itself is not only suitable for client programming, but for server-side development as well. Dynamic, loosely-typed languages offer shorter development cycles, agility in meeting changing customer requirements, particularly for enterprises without well-specified business processes, or rapidly expanding enterprises that constantly adapt their core business. JavaScript on the browser is a de-facto standard; using it on the server provides the benefit of reusing a single code base and data model, and debugging it once.

It is also relatively unknown that there exist already more than one frameworks for programming on the server with JavaScript. The most mature among them is Helma, an open-source framework that has been used in production environments for years, like the Austrian Broadcasting Corporation. I've been playing with it for quite some time and I'd like to show how easy it is to get up and running, in order to make any adventurous souls less afraid to give it a try.

Setting up Helma is fairly straightforward. First you have to download the latest version from http://helma.org/download/. Follow the proper link for your platform and the zip or tar archive will be downloaded in your system. After extracting the archive it will create a folder with the latest version of Helma (1.6.1 as of this writing) in it. The only prerequisite is that you have Java installed on your system, since Helma uses the Rhino JavaScript engine underneath. This arrangement allows the reuse of existing Java code through the Rhino APIs. If your system did not come with Java preinstalled, a visit to java.com should suffice. You should get the latest version available (Java 6 as of this writing), but anything after 1.4 should be good. If you are set, go to the Helma directory and execute start.bat or start.sh depending on your platform and you should see some brief startup messages:


Mac:~/helma-1.6.1 panagiotisastithas$ ./start.sh
Starting Helma in directory /Users/panagiotisastithas/helma-1.6.1
Starting HTTP server on port 8080
Starting Helma 1.6.1 (January 8 2008) on Java 1.5.0_13


The server is now ready to accept requests. Helma embeds Jetty for its HTTP engine, something that makes the startup process a snap and provides for some interesting architectural choices with its support for continuations. Point your browser to http://localhost:8080/ and you should see the welcome page:



This is an actual application, the “welcome” application, that you are interacting with. You might want to navigate around a bit to skim through the available on-line documentation. When you've had enough, you can stop the server by getting back to the console window and hitting Control-C:


^CShutting down Helma - please stand by...
Mac:~/helma-1.6.1 panagiotisastithas$


Just to recap:

1. Download Helma from http://helma.org/download/
2. Unzip
3. Go to the unzipped directory
4. Make sure you have Java 1.4+ installed
5. Execute start.bat/start.sh
6. Point the browser to http://localhost:8080/

Creating a simple application with Helma involves only a few easy steps. First of all create a directory called 'hello' inside the subdirectory 'apps' of the main Helma installation directory. Inside 'hello' create a directory named 'Root'. Inside 'Root' create a file called actions.js with the following content:

function main_action() {
   res.write("Hello, World!")
}

The following screenshot illustrates our progress so far:

Then open the file apps.properties in the main Helma installation directory and add a single line at the end of the file with the following content:


hello


Now start (or restart if it is already running) the server and point your browser to http://localhost:8080/hello. This is what you should see in your browser:

That's it! Your first server-side JavaScript application is ready. In future posts I intend to discuss how Helma wires the various pieces together and explore some of its more interesting features. In the meantime, I suggest that you download it and experiment. You might be pleasantly surprised!

(Grease) Monkey Business

I've been reading David Byrne's journal (of the Talking Heads fame) since before I heard about blogs and blogging. In an ocean of software blogs that flood my Google Reader, it's one of the few that remind me that I have other interests besides computers. From this journal I found out that he also hosts an internet radio station that I've been often enjoying while at work or sometimes at home.

For this task I've been using iTunes on my Mac at home and Rhythmbox on Linux at work. Besides the easier subscription to the station, the one thing I miss from iTunes while at work is seeing the currently playing song on Rhythmbox. I don't know how iTunes does it, but it's always showing the correct song title and artist, while on Rhythmbox all I see is the station name.



Therefore I had resorted to having a tab open in Firefox all day that displayed the radio home page that shows the playlist and refreshes each time a new song begins. I still had to switch to the browser from whatever I was doing when a new song started, in order to glance on its title and performer. A few days ago I thought to myself: "wouldn't it be nice to have the Firefox title bar display that information and only take my eyes off what I'm currently doing and not my fingers too?".


It turns out that I'm very persuasive when I'm talking to myself, so I set off to make my wish come true. I've been also trying to find an excuse to learn more about Greasemonkey for some time and it seemed like the right time to do it. So after I spent some time reading up stuff from the web, I came up with something that made me happy:

(function() {
  window.addEventListener("load", function(e) {

   var PLAYLIST_URL = 'http://www.live365.com/pls/front?handler=playlist&cmd=view&viewType=xml&handle=todomundo&maxEntries=1';

   (function fetchCurrentSong() {
      GM_xmlhttpRequest({
          method: "GET",
          url: PLAYLIST_URL,
          onload: function(details) {
            var xmlString = details.responseText;
            var parser = new DOMParser();
            var xmlDoc = parser.parseFromString(xmlString, "application/xml");
            var refresh = xmlDoc.getElementsByTagName('Refresh')[0].textContent;
            var artist = xmlDoc.getElementsByTagName('Artist')[0].textContent;
            var album = xmlDoc.getElementsByTagName('Album')[0].textContent;
            var title = xmlDoc.getElementsByTagName('Title')[0].textContent;
            var label = title+' by '+artist+' from '+album;
            document.title = label;
            setTimeout(fetchCurrentSong, refresh);
          }
      });
   })();

  }, false);
})();

It's amazing what 20 lines of JavaScript can do. You can install this user script by clicking here, assuming you have Greasemonkey installed and enabled. If you don't have Greasemonkey or prefer to install a firefox extension that does not require it, click this link instead. It was automagically created with the very nifty User Script Compiler.

This is what I see now when I'm listening to David's playlist (although many times I just glance at the title in the minimized button on the bottom panel):


Remixing the web, indeed.

Re: I wish I could...

I can tell that spring is here, when my friend George tags me with another blog meme. Apparently it's like a ritual to him, but it came a few weeks earlier this year, so I figured I could delay posting at least until last year's anniversary. With the frequency I blog, he doesn't have much to expect anyway.

This year's theme is to write about crazy or stupid things you'd like to do, but can't, or stupid things that you do anyway. Since I'm running an old-fashioned, respectable blog here, I'll restrict my answer to appropriate stuff, namely computers and software. And with that I expect to have lost 50% of my audience, so I guess that leaves me and you George. So without further ado, here are 10 things (in no particular order) I wish I could do, but probably won't:

1. ...stop reading FreeBSD mailing lists.
2. ...convince my colleagues at work to switch to Mercurial for version control.
3. ...transfer a call at work (in a Linksys IP Phone) without loosing it.
4. ...build the zero-administration appliance I was planning.
5. ...use and contribute to JNode, an operating system written in Java.
6. ...write a book about a computer-related subject.
7. ...implement class data sharing for 64 bit VMs.
8. ...write Objective-C code using Xcode.
9. ...stop subscribing to more blogs and even ditch a few altogether.
10. ...blog more often.
    

I guess I'm not that ambitious after all...

How to create a WebDAV interface in 10 easy steps

Test Driven Development (TDD) is all the rage these days. And rightly so, if you ask me. Compare developing software against constantly changing client requirements and developing against a specification in the form of tests that your code must pass. I've done my share of the former for the better part of the last 9 years and I've got the gray hair to prove it. However, I only got a chance to practice the latter very recently and I must say I'm hooked.

My TDD endeavor concerns a small project to add a WebDAV interface to an existing server-side code base. The system already featured other interfaces for manipulating the resources stored in it, but a cross-platform, standards-based, nicely integrated, native client was deemed necessary. Enter WebDAV, a.k.a. HTTP on steroids. Most desktop systems nowadays have a WebDAV client installed by default (Windows calls it Web Folders, others prefer not to rename it), so testing a WebDAV server is not that complicated: if you can mount, browse, store and retrieve files and folders using your desktop client, the server implementation is fine. Nevertheless, we can do better than that. Using litmus, a "WebDAV server protocol compliance suite", we can make sure that our implementation respects the protocol, even in its more obscure corner cases and more importantly, perform the testing automatically, without painstaking clickety-click.

Implementing a WebDAV interface from scratch by reading the protocol specification is not the easiest nor the smartest path. You will have to read it in order to figure out what goes where, but you could also reuse some parts from an existing implementation. If you are extending an existing Java codebase, like me, your best bet is reusing the Tomcat implementation. You can run the litmus suite against a vanilla Tomcat installation, just to verify its WebDAV functionality. Not all tests pass, since the implementation is incomplete, but don't worry, most clients don't implement the spec completely either. However take a note of the results, since this is your target outcome.

For your own convenience, I have described the whole development process in 10 easy steps, ready to print, free of charge, batteries not included:

1. Download the litmus test suite.
2. Download the sources for Tomcat, particularly WebdavServlet.java and DefaultServlet.java.
3. Create your own version of WebdavServlet that will delegate the actual functionality to your own backend and fill it with empty stub methods that correspond to the ones in Tomcat.
4. Build & deploy your server.
5. Run the litmus test. All tests fail the first time, but when it eventually succeeds (i.e. matches the result of vanilla tomcat), go to step 9.
   
6. Fill one method at a time from Tomcat's sources, substituting the JNDI communications with your backend logic.
   
7. Add any other helper methods from other Tomcat classes as necessary.
8. Go to step 4.
9. Most tests succeeded, so you are done!
10. For extra credit: fix the code to pass some more tests and send your changes upstream.

You are bound to come across some hair-splitting problems when modifying the codebase to fit in you own backend, but it beats writing it all from scratch hands down, unless you are Greg Stein or Jim Whitehead. In my case, after two weeks and a submitted patch, I was done and had the tests to prove it. Furthermore, when the need for a code refactoring arises, I can rest assured that my changes won't break the protocol functionality if the litmus tests still pass. Not to mention that I have now contributed to Tomcat.

Man, I haven't felt that good in ages.