We’ve been using Google App Engine at Pulse since 2010, back when we had only one backend engineer. In that time, App Engine has served us very well. There are many things Google App Engine does very well; the most obvious advantage is saving us lots of Ops work and letting us stay focused on our application. Over the last two years, it has grown with us both in terms of scale (from 200k users, to 15M+) and in terms of features.
As I’m writing this post (from Google I/O 2012), I’m happy to report that App Engine continues to grow with us. This year, Google’s App Engine team has announced that they are fixing our number one wish list item! They have also started addressing several other important concerns. For some context, here is Pulse’s App Engine wish list as of about a month ago.
- SSL support for custom domains
- Faster bulk import & export of datastore data
- Faster datastore snapshotting
- Tunable memcache eviction policies & capacity
- Improved support for searching / browsing / downloading high volume application logs
- Faster (diff-based) deployment for large applications
- Support for naked domains (without www. in front)
- Unlimited developer accounts per application
Barb Darrow from GigaOm published part of this list earlier this week (before I/O started). Check out the article Google App Engine: What developers want at Google I/O to see more common wish list items from other developers.
As of yesterday, (with the release of SDK version 1.7.0), SSL for custom domains is now officially supported either via SNI for $9/month or via a custom IP for $99/month. This means that you can now host a domain like www.pulse.me on App Engine and support https throughout your site. Previously it had only been possible to use http with your domain, and any secure transactions had to be routed to the less appealing xxxxx.appspot.com domain. This meant you had to break the user’s flow or use some complicated hacks to hide the domain switching. Now it is finally possible to present a seamless, secure experience without ever leaving your custom domain.
There were many other great features released with 1.7.0 (see the link above). As for the rest of our wish list, here’s how it stands now!
- SSL support for custom domains
– Supported now! Faster bulk import & export of datastore data
– Update 2: App Engine Datastore: How to Efficiently Export Your DataFaster datastore snapshotting
– Update 3: The internal settings for map reduce-based snapshotting have been increased to use 256 shards. It’s actually pretty fast now! Still hoping for incremental backups in the future.- Tunable memcache eviction policies & capacity
– I hear that we will soon be able to segment applications and control capacity. Eviction policy controls are likely to take longer. - Improved support for searching / browsing / downloading high volume application logs
– It was announced that this is coming very soon!! Faster (diff-based) deployment for large applications
– Update 4: This is supporting and working for us now!- Support for naked domains (without www. in front)
– Pending. No ETA. - Unlimited developer accounts per application
– This is now supported for premier accounts!
Let me know in the comments if you have any questions about these or want to share some of your wish list items. I’m always happy to discuss App Engine issues with other developers.
Update: Just now, at the second Google I/O keynote, Urs Hölzle has announced Google’s push into the IaaS space with Google Compute Engine. It should be interesting to see if this offers serious competition to Amazon’s EC2 for future Pulse systems and features. 771886 cores available to the demo Genome app was pretty impressive! I’ll post here and/or at eng.pulse.me when we get a chance to try it out!
Tweet
you think google will continue to focus on appengine in parallel? where is diff based deployment documented? i’ve never even come across that before..
also, is pulse running aws or gae?
http://eng.pulse.me/scaling-to-10m-on-aws/
I definitely think they will continue to focus on App Engine in parallel with Compute Engine (with more focus on the former).
As far as I know, diff-based deployment is not documented. Talking with some of the engineers here at I/O, it’s supposed to just work with the standard deployment tool, but for some reason it doesn’t for us. Do you see it working for you? For example, if you have 1000 files, but have only changed 3, do you see “3 files cloned” or “1000 files cloned”? I plan to follow up with Google on this soon.
We actually use both pretty heavily. We use App Engine wherever possible, since it means less Ops work for us. But sometimes we need to do something at a little lower level, so we use AWS.
i definitely have not seen diff’d deploys.. are you also running separate machines that continuously deploy? we’re definitely moving towards that, and i’m pretty confident that it will help alleviate some of the pain in the process