General Discussion

Nested
Google App Engine
User: kmaclean
Date: 4/15/2008 8:13 pm
Views: 3235
Rating: 28

I've been playing with the Google App Engine SDK since it has been released (and am on the waiting list to get on the actual site).  It runs Python, and can use the Django framework.  It has a 500 meg storage limitation.  From the Google App Engine blog:

Google App Engine -- a developer tool that enables you to run your web applications on Google's infrastructure. The goal is to make it easy to get started with a new web app, and then make it easy to scale when that app reaches the point where it's receiving significant traffic and has millions of users.

Google App Engine gives you access to the same building blocks that Google uses for its own applications, making it easier to build an application that runs reliably, even under heavy load and with large amounts of data. The development environment includes the following features:

  • Dynamic webserving, with full support of common web technologies
  • Persistent storage (powered by Bigtable and GFS with queries, sorting, and transactions)
  • Automatic scaling and load balancing
  • Google APIs for authenticating users and sending email
  • Fully featured local development environment

Google App Engine packages these building blocks and takes care of the infrastructure stack, leaving you more time to focus on writing code and improving your application.

The reason why I was very interested in this service was to use it to host the SpeechSubmission application.  This might addresse a scalability issue with the current Java applet setup.  The Java Applet is stored on a regular hosting server, and the speech uploads are directed there also.  Some people have commented that when the hosting server is busy, the upload takes too long.   This would also important if we get Slashdotted again.  The last time this occured, many users were not able to get to the site.  Although we did not have the Java Applet running at the time, many of these might have contributed some speech if they would have been able to.

I've been playing with the App Engine SDK on my desktop computer (learning a bit of Python along the way), and have been able to get the Java SpeechSubmission Applet to display.  I have also managed to get binary files to upload to persistent storage (the local, flat file version of Google's bigtable).  All that would remain would be for the Java Applet to talk to the uploader and upload an audio file to App Engine - which should not be a big problem.  I would also need to create a script that would download and purge any newly uploaded speech submissions from the App Engine repository to address the 500 meg application storage limitation.   

The only problem is that I just came across is a 1 meg file size limitation for uploads and downloads.  From the Google App Engine blog:

Here are are some of the general areas we're focusing on right now:

...
  • Support for large files. Google App Engine currently imposes a limit of 1MB on all requests, both inbound and outbound. We're interested in providing efficient support for much larger uploads and downloads.

 ...

I am glad that addressing this limitation is one of their priorities.  Once this is addressed, I will look into using Google App Engine again. 

A work-around might be direct the uploads to Google Sites, though the upload code (originally from JotSpot) has some nasty looking Javascript.  Google Sites is also interesting because its storage limit is 10 Gig.  If there are any Javascript experts out there who can figure out which fields the SpeechSubmission applet should be sending to the Google Sites "File Cabinet" uploader, please let me know.

thanks, 

Ken

--- (Edited on 4/15/2008 9:13 pm [GMT-0400] by kmaclean) ---

PreviousNext