GSS/Pithos update: iPhone, Android, Mongo & getting to 2.0
It has been a while since my last post on GSS/Pithos, and there have been quite a few important announcements that I have not blogged about, only tweeted:
The initial plan was far more modest. We had identified the PostgreSQL installation that stores all of the the system’s data besides the actual files, as a future bottleneck when user adoption would start to accelerate. In such an event there were a few low-to-medium-cost solutions that we could pursue, like replication and manual sharding, but since the problem seemed best suited for a NoSQL solution, we decided to bite the bullet and attempt a transition.
Our field search included lots of products, like Mongo, Couch, Cassandra, Riak, Redis, HBase, MemcacheDB and others. The basic requirements were high write throughput, sharding, replication and easy migration. Most of the products boasted high performance for writes and lots supported replication. In the end we discarded the ones that didn’t support sharding, which limited the options a lot, and we investigated the various APIs, trying to figure out what it would mean to rewrite GSS for one of these. In the end we chose Mongo for its rich query support, since it was the best fit for our problem at hand, making the transition of the existing code base easier. Although we’d love to have had the opportunity to try different ports for each of those datastores, it is most likely that we would have had to rewrite the entire server from scratch, which was not an option.
The initial plan was to try to isolate the changes to the data access layer, essentially building a new adapter for the middle tier that would appear almost identical to the existing code. This effort was fruitless however, since we could not find satisfactory solutions for various issues, like transactions, data mapping, entity lifecycle, etc. We came to the conclusion that even if we managed to build a not-too-leaky abstraction for Mongo, it would have probably killed the performance, making the switch from PostgreSQL pointless.
So we ditched JPA and container-managed transactions and decided to make a simpler POJO mapping to Mongo using Morphia, a Java data mapper for Mongo that takes care of the nitty-gritty details and repetitive coding, for only a minor performance hit. Since Mongo only supports atomicity for a single query, we reverted to manual transaction handling and decided to ditch the EJBs that formed the backbone of GSS altogether, since they didn’t buy us anything at that point. The new GSS server would be a set of servlets, service and domain objects, wired together via Google Guice. Since we could host the new server in a plain servlet container, we went from JBoss to Jetty, which vastly reduced the server startup time from 60 seconds down to 2. As a consequence, development round-trip times went down, and coding was fast again. Lots of object copying and conversions from entities to DTOs and back again were now unnecessary, since there was no more a transaction boundary at the session bean layer. This kept memory usage low, increased cache locality and boosted performance. There are lots of interesting details about our approach, but they would be better discussed in separate blog posts. If you are interested, you can already check out the code in the gss2 branch of the GSS repository.
- a shiny native iPhone client
- a brand new Android client
- a back-office application for statistics and admin tasks
- public folders for hosting static web sites
- a slew of performance, usability and bug fixes for the web & desktop clients
- new user registration and quota upgrade workflows
The initial plan was far more modest. We had identified the PostgreSQL installation that stores all of the the system’s data besides the actual files, as a future bottleneck when user adoption would start to accelerate. In such an event there were a few low-to-medium-cost solutions that we could pursue, like replication and manual sharding, but since the problem seemed best suited for a NoSQL solution, we decided to bite the bullet and attempt a transition.
Our field search included lots of products, like Mongo, Couch, Cassandra, Riak, Redis, HBase, MemcacheDB and others. The basic requirements were high write throughput, sharding, replication and easy migration. Most of the products boasted high performance for writes and lots supported replication. In the end we discarded the ones that didn’t support sharding, which limited the options a lot, and we investigated the various APIs, trying to figure out what it would mean to rewrite GSS for one of these. In the end we chose Mongo for its rich query support, since it was the best fit for our problem at hand, making the transition of the existing code base easier. Although we’d love to have had the opportunity to try different ports for each of those datastores, it is most likely that we would have had to rewrite the entire server from scratch, which was not an option.
The initial plan was to try to isolate the changes to the data access layer, essentially building a new adapter for the middle tier that would appear almost identical to the existing code. This effort was fruitless however, since we could not find satisfactory solutions for various issues, like transactions, data mapping, entity lifecycle, etc. We came to the conclusion that even if we managed to build a not-too-leaky abstraction for Mongo, it would have probably killed the performance, making the switch from PostgreSQL pointless.
So we ditched JPA and container-managed transactions and decided to make a simpler POJO mapping to Mongo using Morphia, a Java data mapper for Mongo that takes care of the nitty-gritty details and repetitive coding, for only a minor performance hit. Since Mongo only supports atomicity for a single query, we reverted to manual transaction handling and decided to ditch the EJBs that formed the backbone of GSS altogether, since they didn’t buy us anything at that point. The new GSS server would be a set of servlets, service and domain objects, wired together via Google Guice. Since we could host the new server in a plain servlet container, we went from JBoss to Jetty, which vastly reduced the server startup time from 60 seconds down to 2. As a consequence, development round-trip times went down, and coding was fast again. Lots of object copying and conversions from entities to DTOs and back again were now unnecessary, since there was no more a transaction boundary at the session bean layer. This kept memory usage low, increased cache locality and boosted performance. There are lots of interesting details about our approach, but they would be better discussed in separate blog posts. If you are interested, you can already check out the code in the gss2 branch of the GSS repository.
The next generation GSS server is not production ready yet, but we'll be getting there soon, and so far it has been a great ride!
No comments:
Post a Comment