but there is an issue with it seems many do not realize:

the “database” backend of google-app-engine (i will call it BigTable in the following text) is not a relational (read “SQL”) store, and it will never be. for example, it does not support SQL JOINs. but it’s worse than that. because of it’s architecture, JOINS will never be fast there. BigTable is essentially a collection of spreadsheet-tables, where you can do some basic searches, that’s all. oh, and transactions.

for this reason, there probably never will be a BigTable django-ORM wrapper. of course technically it’s possible to implement in python all the missing features, but it’s performance characteristics will not be the same as of a relational-database. so you will not be able to simply take your mysql/postgresql-optimized application, and deploy on it, and all is fine. you will have to restructure your application.

and if you have to restructure your app anyway, why do you need the django-ORM? you can as well write google-app-engine-specific code.

(on the other hand, maybe there could be a more stupid django-orm, that does not assume a relational-db-backend, and it could work with the various non-relational databases like BigTable or hBase or other column-oriented databases…)

the basic idea when writing BigTable code is that read-operations will happen much more often than write-operations. so do more at write-time, and less at read-time. denormalize tables.

for example, take a simple forum-application. it stores discussions. a discussion has comments. now let’s see how we could implement 2 basic features: “add comment” and “list discussion-names with comment-count”.

SQL:

• “add comment”: store a new comment-entry, which contains a link (a foreign-key) to it’s discussion
• “list discussion-names with comment-count”: do an SQL query like: SELECT discussion.name,count(1) from discussion LEFT OUTER JOIN comment GROUP BY comment.discussion_id; (let’s not discuss right now if it’s inner or outer join etc. it’s quite late at night here, so maybe it’s not 100% correct. but it should be enough to demonstrate the situation)

BigTable (one possible solution):

• “add comment”: store a new comment entry, which contains a link to it’s discussion. also, count the number of comments for this discussion, and store this value in the discussion-table
• “list discussion-names with comment-count”: select * from discussion

of course the whole denormalize-your-database-if-you-want-performance mantra is nothing new. if i remember correctly, Flickr also does this. but still, for most developers, (also for me), it’s just painful to give up our nice, clean, normalized db-tables.

p.s: please note, that all the info is not based on my own performance-benchmarks. it’s more a summary of what i’ve read in the the google-appengine documentation and the google-appengine mailing list.

let’s say you want to create a web-app which:

• shows a form to the user, where he can enter a number and submit it
• then you show him a second form, where the user can enter a second number and submit it
• then you show the user a page which displays the sum of those 2 submitted numbers in Seaside, you can implement it in something like this: (pseudocode)

first_num = get_number()
second_num = get_number()
display(first_num + second_num)

my language of choice is python, so i was curious if it’s possible to achieve something similar in that language. seside uses continuations to achieve this, so i turned to python generators. they allowed me to write this:

def sum_two_numbers():
    first_num = yield InputPage("enter first number:")
    second_num = yield InputPage("enter second number:")
    sum = int(first_num) + int(second_num)
    yield TextPage("the sum isresult is: %s" % sum)

the full code is below, but first some notes about it:

• it requires python 2.5 or higher, because i use the “advanced” version of yield, which appeared in python2.5
• this is only proof-of-concept code, and i know that it uses global-variables, ugly names, get-where-it-should-do-post etc.
• i know Seaside does much much more (backbutton-support etc.)

the code:

#!/usr/bin/env python2.5
from wsgiref.simple_server import make_server
from cgi import parse_qs
 
 
class TextPage(object):
    def __init__(self,text):
        self.text = text
 
    def get_response_content(self):
        return '<html><body>%s</body></html>' % self.text
 
    def get_request_data(self,environ):
        return None
 
class InputPage(TextPage):
    def get_response_content(self):
        return """<html><body>%s
            <form method="get" action=".">
            <input type="text" name="data" value=""/>
            </form>
            </body></html>""" % self.text
 
    def get_request_data(self, environ):
        get_req_dict = parse_qs(environ['QUERY_STRING'])
        return get_req_dict['data'][0]
 
 
def sum_two_numbers():
    first_num = yield InputPage("enter first number:")
    second_num = yield InputPage("enter second number:")
    sum = int(first_num) + int(second_num)
    yield TextPage("the sum isresult is: %s" % sum)
 
s = sum_two_numbers()
cur_item = None
 
def simple_app(environ, start_response):
    status = '200 OK'
    response_headers = [('Content-type','text/html')]
    start_response(status, response_headers)
 
    global cur_item
 
    if not cur_item:
        cur_item = s.next()
    else:
        cur_item = s.send(cur_item.get_request_data(environ))
 
    return [cur_item.get_response_content()]
 
 
httpd = make_server('', 8000, simple_app)
httpd.serve_forever()

i did a quick google-check about this topic (“python web framework continuations”), and only found Nevow with Wolf, which uses a less-elegant approach, probably because when it was implemented, python2.5 was not yet available. if anyone knows about other python web-frameworks that use generators, leave me a comment.