Moving On

About 10 years ago, I was invited to become employee No1 of a new software startup. I didn’t know much about startups back then, besides that they were supposedly cool, and employees weren’t treated like small cogs in a giant machine. Indeed, it felt really fun to build our first product in my 5 year-old laptop, running FreeBSD, of all things. That, and getting praised by our users, too.

Things got even better as time went by: interesting projects to build, exciting technologies to explore, new colleagues to work with, new challenges to face. I felt my work was making a real impact, getting the team to follow new and exciting technological paths, often far away from the mainstream. I became partner, pushed for launching our first large consumer product, coded, mentored, planned and dreamed about where to go next. I had a 27-inch Mac, a big desk and a great view from my office window.

And then I left.

When is it a good time to leave? When everything is over, it’s obviously too late. When things are just starting to take shape, it’s definitely too soon. I used to say, ‘when you are done’. But it turns out that, in real life, you are never done. There are always more things to do, unfinished businesses to attend to, and existing plans to follow through. Perhaps a better answer is: ‘when you are too comfortable, almost on the brink of getting bored’.

A friend asked me why would I leave this company in what appears, by all accounts, to be the pinnacle of our accomplishments, ‘now that your work is being recognised and respected’, as he put it. Another said that we’re all going through some pretty hard times, and it wouldn’t be wise to take risks. A rather unsettled family member asked: ‘aren’t you afraid?’

Of course I was.

Afraid of leaving behind the tried-and-true for something new and unknown; afraid that the new gig might not live up to the initial impressions; afraid that my work-life balance might deteriorate. However, I’ve learned long ago that fear isn’t something to be afraid of. As Eleanor Roosevelt said once: ‘You gain strength, courage, and confidence by every experience in which you really stop to look fear in the face. You must do the thing which you think you cannot do.’ What I wanted to do, is to become even better at my craft. To have an impact on even more people’s lives through my work. To inspire and be inspired. To be afraid and excited again.

So, as of a few days ago I’ve taken up an offer to work for Golden Deal. It’s a product company, something I’ve always been keen on, with a great track record and an awesome crew that is rapidly expanding (are you a senior dev or admin? we’re hiring!). I’m still getting to know the place around here, but it feels good, and I think we have a shot at becoming one of the greatest Internet companies in Greece.

I know that my former teammates will continue to rock on, and I’ll be cheering from the sidelines for their future accomplishments. As for me, I just might find more time to blog again.

P.S.: For folks contemplating a career move, here are some tips for choosing your next gig. This is not how I approached my own case, since I wasn’t really on the market for a job. I was more like seduced, from some pretty persuasive folks. However, had I been looking for a new employer, this is what I would have looked out for:

• Product company: this is where you have a better chance at learning the ropes of modern technologies, without doing pure academic exercises that have little to no importance for the business.
• Startup mentality: you want to be a part of a group that fights as a whole for the success of the company, not a bunch of bureaucrats isolated in their silos.
• Experienced team with a good track record: so you can learn from people better, smarter or wiser than you, and improve your skills.
• Teams not afraid to experiment with new and unproven technologies: because innovation is the name of the game, at least for startups and product companies.
• A ‘getting things done’ attitude: because success apparently only comes for the right kind of people.
• Honest presentation: this is rather obvious.

Bear in mind that people have a tendency to present a rosy picture of their situation, because, if nothing else, it makes them feel better about themselves. Be a little pessimist and assume things are a tad worse, and you’ll be closer to the truth most of the time. Also, do some research on what you’re being told, don’t trust their sales pitch blindly.

I know I did.

GSS/Pithos update: iPhone, Android, Mongo & getting to 2.0

It has been a while since my last post on GSS/Pithos, and there have been quite a few important announcements that I have not blogged about, only tweeted:

• a shiny native iPhone client
• a brand new Android client
• a back-office application for statistics and admin tasks
• public folders for hosting static web sites
• a slew of performance, usability and bug fixes for the web & desktop clients
• new user registration and quota upgrade workflows

And since the code base has been adopted by other European NRNs, we have also improved our release engineering process and documentation. This however was just the tip of the iceberg. What we are now close to completing, is nothing short of an almost complete rewrite of the core GSS server, aiming at even greater performance and scalability. We are about to release GSS 2.0.

The initial plan was far more modest. We had identified the PostgreSQL installation that stores all of the the system’s data besides the actual files, as a future bottleneck when user adoption would start to accelerate. In such an event there were a few low-to-medium-cost solutions that we could pursue, like replication and manual sharding, but since the problem seemed best suited for a NoSQL solution, we decided to bite the bullet and attempt a transition.

Our field search included lots of products, like Mongo, Couch, Cassandra, Riak, Redis, HBase, MemcacheDB and others. The basic requirements were high write throughput, sharding, replication and easy migration. Most of the products boasted high performance for writes and lots supported replication. In the end we discarded the ones that didn’t support sharding, which limited the options a lot, and we investigated the various APIs, trying to figure out what it would mean to rewrite GSS for one of these. In the end we chose Mongo for its rich query support, since it was the best fit for our problem at hand, making the transition of the existing code base easier. Although we’d love to have had the opportunity to try different ports for each of those datastores, it is most likely that we would have had to rewrite the entire server from scratch, which was not an option.

The initial plan was to try to isolate the changes to the data access layer, essentially building a new adapter for the middle tier that would appear almost identical to the existing code. This effort was fruitless however, since we could not find satisfactory solutions for various issues, like transactions, data mapping, entity lifecycle, etc. We came to the conclusion that even if we managed to build a not-too-leaky abstraction for Mongo, it would have probably killed the performance, making the switch from PostgreSQL pointless.

So we ditched JPA and container-managed transactions and decided to make a simpler POJO mapping to Mongo using Morphia, a Java data mapper for Mongo that takes care of the nitty-gritty details and repetitive coding, for only a minor performance hit. Since Mongo only supports atomicity for a single query, we reverted to manual transaction handling and decided to ditch the EJBs that formed the backbone of GSS altogether, since they didn’t buy us anything at that point. The new GSS server would be a set of servlets, service and domain objects, wired together via Google Guice. Since we could host the new server in a plain servlet container, we went from JBoss to Jetty, which vastly reduced the server startup time from 60 seconds down to 2. As a consequence, development round-trip times went down, and coding was fast again. Lots of object copying and conversions from entities to DTOs and back again were now unnecessary, since there was no more a transaction boundary at the session bean layer. This kept memory usage low, increased cache locality and boosted performance. There are lots of interesting details about our approach, but they would be better discussed in separate blog posts. If you are interested, you can already check out the code in the gss2 branch of the GSS repository.

The next generation GSS server is not production ready yet, but we'll be getting there soon, and so far it has been a great ride!

The Cloud Desktop

This was originally posted in the mynetworkfolders.com blog.

We live in a browser these days. E-mail, word-processing, spreadsheets, chat, are all occupying a browser window in front of me. Maybe not for everyone yet, but the trend is clear, and has been clear for quite some time now. This is a good thing. The Web of today bears little similarity to the Web of the millennium. It's fast, it's powerful and it's ubiquitous. A consequence of that evolution is the simplicity with which I can continue my work when leaving my Linux workstation at work and sit in front of my Mac at home. Since my applications are web-based, they are OS- and browser-agnostic. More importantly, my data are always there when I need them. Readily accessible from my desktop, laptop or mobile phone. I may keep local backups for archival purposes, but my master copies are always online.

Assuming this trend is indeed inescapable, one can't help but wonder: what are the applications that have not migrated to the cloud, yet? What kind of things would I like to do online, seamlessly, from every device I own? What are the barriers that make our current online experience less powerful than the desktop one?

One thing that we noticed was missing from the online desktop, was the... desktop! Yes, my spreadsheets are online. Sure, my pictures are online. Yes, I can upload arbitrary files in various storage services. But, I couldn't find a way to use a familiar file manager interface to work with my data in the cloud, as I do on my computer. This was the need we decided to fulfill with MyNetworkFolders.

MyNetworkFolders attempts to provide an answer to the question: how would my desktop look in the cloud? And even though we are nowhere near done yet, the results so far are exciting and very promising.

MyNetworkFolders offers a familiar file manager interface for my cloud storage needs, with advanced functionality and ease of use. Documents can be viewed, modified, searched or sent to the trash. They can be shared with other users or among the members of a group using flexible ACLs, or even be publicly accessible from the whole internet. They can be versioned for tracking changes more effectively. The service can be accessed from a variety of ways, including a web browser, a desktop client, a WebDAV network share or (really soon) a mobile phone. But most importantly, there is no vendor lock-in. There is a REST-like API available, battle-tested from all the above client implementations, giving everyone the opportunity to access his data in his own particular way. Or even get them out of our service altogether.

Oh, and one other thing. It's all based on open-source software. The core infrastructure, the various clients we have built, even API helpers in various programming languages, are all available for anyone and everyone to see, use and modify. Or maybe used to setup and run YourNetworkFolders, if you feel really ambitious.

If however you just want a true desktop environment, with unparalleled flexibility, and maybe also see how far the cloud desktop rabbit-hole goes, then come along. It's going to be an exciting ride.