App Engine For The Enterprise?

I like writing software, not the process of deploying it. The idea of abstracting away configuration, hosting and complex network setup has always been very appealing to me. This is one of the big promises of cloud computing; so I have been looking at and playing various cloud technologies as they have been popping up over the last 3-4 years.

Recently I have started using Google App Engine for non-trivial web apps. My experience so far has been a pleasure! You may say: What!? that's not a very precise description, true, but this is not meant to be a thorough analysis or comparison of every feature that one can think of or use in a real-world applications. What I can say, is that it's been painless to do all the regular things that you have to during a standard application development and deployment process. The API's are elegantly simple to learn and to use, and make it easy for you to start handling requests, persisting data, doing background processing, all on a scalable platform that can handle from 1 to thousands concurrent requests (at least in theory).

At the same time, having worked at a number of large corporations I have seen a number of good and bad attempts at streamlining the deployment processes and abstracting away architecture complexities. Again, I have mostly been in the end user role, not the implementor of the abstraction/automation frameworks and systems, so my view may be biased/limited/etc, but at least it represents an experience of a typical developer at a large organization. Typically it's been a rather arduous experience. Some firms are better than others, but in almost all cases you have to pay lots of attention to fairly low level architecture of your application. You can't just write a simple script and deploy it into the cloud.

Given my recent experiments with the App Engine, and extensive experience with large enterprise IT environments, I have starting thinking about how cloud computing can apply to the enterprise.

Low Level Infrastructure ------> High Level API

At the low level, you have to worry about how you are going to scale your application and the exact technology stack needed for your particular application. This is essentially what is being offered by various virtual machine cloud providers: Amazon EC2, Rackspace Cloud, and others. You have to figure out what VM you want to run (ie pick OS: linux flavor, windows), and what goes inside the VM (mysql/postgres/couchdb/other db, apache, lighthttpd, j2ee, IIS), how many nodes you want to have for failover and scalability, how many tiers you need to scale, etc. This gives you ultimate flexibility, but requires a LOT of extra architecture work. This comes at the expense of agility.

At the mid level of the spectrum, you have something like Windows Azure. You get to use familiar development tools and components (SQL, asp.net). But you still have to think about the architecture: how many worker/web/db nodes you want to have, some design of the schema.

At the highest level of architecture abstraction you will find Google App Engine. As a developer on this platform you don't have to worry about configuring VMs, figuring out the number and size of logical tiers. You are able to focus directly on solving your business problem. To Be Continued...