Back-references are your friends

A very handy feature of database references is their automatic back-references pairs.

class User(db.Model):

name = db.StringProperty()

class Phone(db.Model):

number = db.PhoneNumberProperty()

user = db.ReferenceProperty(User)

# fetch one user

username = 'John'

user = User().all().filter('name = ', username).get()

# get the first associated phone number for a user

phonenum = user.phone_set[0].number

# list all phone numbers of the user

for phonenum in user.phone_set:

# phonenum here contains the user's actual phonenum in the list

The name of the back-reference property defaults to {model name}+'_set' in lower case, that's how 'phone_set' was made in the example. More about back-references can be read here.

Interactive Shell - a must have

Do you want to debug and see a memcache value on the live server? Or any other property? Deploy this interactive shell into your application's directory. Don't forget to set the rights to admin only in app.yaml.

Full Text Search

I thought it's impossible but then I've found an article about Full Text Searching in GAE - the nasty thing is, it's entirely undocumented. I'll give it a try soon. How-to can be found here.

Custom Django Filters in Google Application Engine

Here is a very good description how can you use custom Django filters in Google Application Engine.

I've modified a filter found on Django Snippets to convert ISO8859-2 / ISO8859-1 strings into URL friendly strings:

match_non_alnum = re.compile("[^a-zA-Z0-9]+")

def dig_url_encode(string):

"""Converts common ISO88592/1 chars to ASCII and removes any special chars"""

r = {

u'á': u'a',

u'é': u'e',

u'í': u'i',

u'ó': u'o',

u'ö': u'o',

u'õ': u'o',

u'ő': u'o',

u'ú': u'u',

u'ü': u'u',

u'ű': u'u',

u'û': u'u',

u'Á': u'A',

u'É': u'E',

u'Í': u'I',

u'Ó': u'O',

u'Ö': u'O',

u'Ő': u'O',

u'Ú': u'U',

u'Ü': u'U',

u'Ű': u'U'

}

for key, value in r.items():

string = string.replace(key, value)

return match_non_alnum.sub(' ', string).strip().replace(' ', '_')

Never pass an extended db.Model to Django

Despite it will work locally, Django will not recognize a property added to a db.Model.

Example:

class Example(db.Model)

username = db.StringProperty()

password = db.StringProperty()

examples = Example().all().fetch(10)

for example in examples:

example.address = 'hello'

template_values = {

'examples': example

}

Django won't see example.address only with the local Appengine SDK environment! It might work with the db.Expando mode but quite confusing since the added property (address) will be unreachable only after the application is uploaded to the GAE servers.

Possible solution:

class Example(db.Model)

username = db.StringProperty()

password = db.StringProperty()

examples = Example().all().fetch(10)

elist = []

for example in examples:

eitem = {

'example': example,

'example_address': 'hello'

}

elist.append(eitem)

template_values = {

'examples': elist

}

Thanks for Eoghan for finding a solution.

Building an App Engine Google Query - Terror of the missing white space

Lesson learned:

Always use white space between query filters and the equality signs otherwise the equal sign will be interpreted as part of the index key.

Example:

Wrong:


cattoplist = dbmodels.DailyStats.all().filter("kc_date=", today).filter("kc_category=", cat.cat_id).order("-visitor_new").fetch(catnum, catoffset)

Good:


cattoplist = dbmodels.DailyStats.all().filter("kc_date =", today).filter("kc_category =", cat.cat_id).order("-visitor_new").fetch(catnum, catoffset)

Introduction

This blog is dedicated to follow my adventures in the cloud. I'm working on high performance, distributed, scalable systems since many years and recently I've started to test Google's infrastructure, the Google Application Engine. I'll log every useful thoughts and experiences to avoid myself - and hopefully other developers as well - to fall into the same trap.