90

When researching Google App Engine (GAE), it's clear that using Django is wildly popular for developing in Python on GAE. I've been scouring the web to find information on the costs and benefits of using Django, to find out why it's so popular. While I've been able to find a wide variety of sources on how to run Django on GAE and the various methods of doing so, I haven't found any comparative analysis on why Django is preferable to using the webapp framework provided by Google.

To be clear, it's immediately apparent why using Django on GAE is useful for developers with an existing skillset in Django (a majority of Python web developers, no doubt) or existing code in Django (where using GAE is more of a porting exercise). My team, however, is evaluating GAE for use on an all-new project and our existing experience is with TurboGears, not Django.

It's been quite difficult to determine why Django is beneficial to a development team when the BigTable libraries have replaced Django's ORM, sessions and authentication are necessarily changed, and Django's templating (if desirable) is available without using the entire Django stack.

Finally, it's clear that using Django does have the advantage of providing an "exit strategy" if we later wanted to move away from GAE and need a platform to target for the exodus.

I'd be extremely appreciative for help in pointing out why using Django is better than using webapp on GAE. I'm also completely inexperienced with Django, so elaboration on smaller features and/or conveniences that work on GAE are also valuable to me.

Improve this question
3
  • holy crap terry bradshaw writes code?
    – Woot4Moo
    Commented Dec 20, 2009 at 5:05
  • 4
    Django is beneficial because it's awesome. That's really it. :)
    – jathanism
    Commented Dec 21, 2009 at 23:27
  • I am new to Google app engine as well and this is a terribly well formed question even for 2018(although Django ORM it seems is much better supported on GAE now). :)
    – Divij Sehgal
    Commented Mar 31, 2018 at 12:34

8 Answers 8

Reset to default
51

Django probably isn't the right choice for you, if you're sure that GAE is right for you. The strengths of the two technologies don't align very well - you completely lose a lot of Django's wonderful orm on GAE, and if you do use it, you write code that isn't really directly suitable to bigtable and the way GAE works.

The thing about GAE is that it gets the great scalability by forcing you to write code that scales easily from the ground up. You just can't do a number of things that scale poorly (of course, you can still write poorly scaling code, but you avoid some pitfalls). The tradeoff is that you really end up coding around the framework, if you use something like Django which is designed for a different environment.

If you see yourself ever leaving GAE for any reason, becoming invested in the infrastructure there is a problem for you. Coding for bigtable means that it will be harder to move to a different architecture (though the apache project is working to solve this for you with the HBase component of the Hadoop project). It would still be a lot of work to transition off of GAE.

What's the driving motivator behind using GAE, besides being a Google product, and a cool buzzword? Is there a reason that scaling using something like mediatemple's offering is unlikely to work well for you? Are you sure that the ways that GAE scales are right for your application? How does the cost compare to dedicated servers, if you're expecting to get to that performance realm? Can you solve your problem well using the tools GAE provides, as compared to a more traditional load-balanced server setup?

All this said, unless you absolutely positively need the borderline-ridiculous scaling that GAE offers, I'd personally suggest not letting that particular service structure your choice of framework. I like Django, so I'd say you should use it, but not on GAE.

Edit (June 2010): As an update to this comment sometime later: Google has announced sql-like capabilitys for GAE that aren't free, but will let you easily do things like run SQL-style commands to generate reports on your data.

Additionally, there are upcoming changes to the GAE query language which will allow complex queries in a far easier fashion. Look at the videos from Google I/O 2010.

Furthermore, there is work being done during the Summer of Code 2010 project which should bring no-sql support to django core, and by extension, make working with GAE significantly easier.

GAE is becoming more attractive as a hosting platform.

Edit (August 2011):

And Google just raised the cost to most users of the platform significantly by changing the pricing structure. The lockin problem has gotten better (if your application is big enough you can deploy the apache alternatives), but for most applications, running servers or VPS deployments is cheaper.

Very few people really have bigdata problems. "Oh my startup might scale someday" isn't a bigdata problem. Build stuff now and get it out the door using the standard tools.

Improve this answer
6
  • Thanks for the thoughtful response, Paul. We're evaluating GAE largely because the cost model matches our proposed business plan well. The ability to start free and then scale incrementally with a very granular cost model is very attractive. Additionally, we have no expectation of needing to move or migrate away from GAE in the future and platform lock-in for this project is acceptable. I included the "exit strategy" comment in my question primarily in an effort to be fairly comprehensive in what I've learned through reading and research before posting this question. Thanks again!
    – Travis Bradshaw
    Commented Dec 20, 2009 at 5:26
  • How do you rate the cost of development time? One of the nice things about Django is that you spend less time worrying about defining your data models than you would with Bigtable. Bigtable forces you to be much clearer about your usages prior to being able to use it at all. Certain queries are significantly easier with "normal" sql.
    – Paul McMillan
    Commented Dec 20, 2009 at 5:38
  • 3
    Be careful not to pinch pennies too excessively. Free is nice, but the service rapidly does cost real money. If you're coasting in the "free" level of service, host it on some other server/hosting you're already paying for. If you're getting into the non-free level of service, the $20/mo for a VPS that you can easily scale later is in the noise as far as cost.
    – Paul McMillan
    Commented Dec 20, 2009 at 5:42
  • 11
    tbradshaw, do not forget to consider how often you will need ad-hoc reports run on your data set. I am involved in a growing social application and GAE is becoming... I won't say a nightmare, but it is extremely resource-intensive to derive knowledge from our data. Between Google purging old logs and the extreme lengths required to sweep across all the data, it makes reporting way, way more expensive than an SQL db. That is a cost I did not consider when getting started. Secondly, if you really do grow and start making money, the control you lose concerning backups is, well, a factor.
    – JasonSmith
    Commented Dec 20, 2009 at 8:55
  • 2
    For lock-in concerns check out AppScale, which is a Google App Engine clone. We've been working on the platform since GAE first came out and have many users on it for their production python and java applications. You have direct access to the machines it runs on so you have more control on the infrastructure. github.com/AppScale/appscale.git
    – Navraj Chohan
    Commented Feb 17, 2014 at 20:36
20

We use django on our appengine instances mostly when we have to serve actual websites to the user. It has a great template engine, url routing and all the request/response/error handling built in. So even while we can't use the magic orm/admin stuff it has a lot going for it.

For api services we built something very simple on top of webob. It's far more lightweight because it doesn't need everything that django offers, and therefore a little quicker in some situations.

Improve this answer
4
  • 1
    Thanks Koen. Part of my confusion as to the appeal of Django stemmed from the idea that url routing and request/response/error handling were also features of the provided webapp and that the template engine can be used without Django as well with webapp. Am I mistaken? Does Django provide these services better than the webapp framework?
    – Travis Bradshaw
    Commented Dec 20, 2009 at 17:19
  • They're more extensive and flexible in django I would say. So it's better if you actually need that :-)
    – Koen Bok
    Commented Dec 20, 2009 at 21:49
  • 2
    I think that this is the answer I'm looking for! That Django is largely redundant to webapp, but in the functionality that they share Django does in a more flexible and robust way. It seems like it's surely a decision "on the margin," but I think all of the other suggestions, plus yours, makes for a compelling answer. Thanks.
    – Travis Bradshaw
    Commented Dec 20, 2009 at 23:49
  • 1
    Python modules written in C are not supported as well.
    – Ryu_hayabusa
    Commented Aug 11, 2014 at 11:26
16

I've done lots of projects on GAE. Some in django, some in their normal framework.

For small things, I usually use their normal framework for simplicity and quickness. Like http://stdicon.com, http://yaml-online-parser.appspot.com/, or http://text-twist.appspot.com/.

For large things, I go with django to take advantage of all the nice middleware and plugins. Like http://metaward.com.

Basically my litmus test is Will this take me more than 2 weeks to write and be a REAL software project? If so, go with django for the addons.

It has the added benefit of, if your project is badly suited for BigTable then you quickly port off (like I did Is BigTable slow or am I dumb?)

Improve this answer
3
  • +1, bigtable is just bad for some kinds of projects and queries. It's great for what Google does, and may be terrible for what you want to do.
    – Paul McMillan
    Commented Dec 20, 2009 at 5:43
  • Thanks Paul! Could you link me to any resources that describe the Django middleware and plugins that work on GAE? If there are add-ons useful to our project, that would certainly be a reason to go with Django rather than webapp. The more obviously useful ones (like session and authentication) appear to have clear Django ORM dependancies.
    – Travis Bradshaw
    Commented Dec 20, 2009 at 6:04
  • 2
    Basically any addon that doesn't have a models.py will work perfectly. And if they have a models.py you can probably do the 1-to-1 conversion to bigtable and still use it if you want. Some I use are django_annoying, django_debug_toolbar, and from the contrib section csrf, humanize, and of course admin.
    – Paul Tarjan
    Commented Dec 20, 2009 at 10:40
11

I think all this answers are a bit obsolete.

Now you can use Google Cloud SQL

Django is a popular third-party Python web framework. When coupled with Google Cloud SQL, all of its functionality can be fully supported by applications running on App Engine. Support for using Google Cloud SQL with Django is provided by a custom Django database backend which wraps Django's MySQL backend.

https://cloud.google.com/python/django/appengine

one more fresh news is, that there is BETA support for PostgreSQL

Improve this answer
3

I have experience using Django and not GAE. From my experiences with Django it was a very simplistic setup and the deployment process was incredibly easy in terms of web projects. Granted I had to learn Python to really get a good hold on things, but at the end of the day I would use it again on a project. This was almost 2 years ago before it reached 1.0 so I my knowledge is a bit outdated.

If you are worried about changing platforms, then this would be a better choice I suppose.

Improve this answer
2
  • 1
    Thanks for your quick response! While I agree that Django is a framework that's speedy to get started, that's not really a concern for us. We have four quite experienced Python developers with web development backgrounds, so getting started with any framework is going to be quick and painless. But the question remains, when choosing between Django and webapp on GAE, which is the better choice and why?
    – Travis Bradshaw
    Commented Dec 20, 2009 at 5:10
  • @Woot4Moo if no experience with GAE were do you deploy it to, I am new to GAE but the price is confusing me quite a lot, random hughe charges, I am thinking pythonanywhere, could you pass me some recommendations?
    – Manza
    Commented Aug 28, 2018 at 20:55
0

I cannot answer the question but you may want to look into web2py. It is similar to Django in many respects but its database abstraction layer works on GAE and supports most of the GAE functionality (not all but we try to catch up). In this way if GAE works for you great, if it does not, you can move your code to a different db (SQLite, MySQL, PostgreSQL, Oracle, MSSQL, FireBird, DB2, Informix, Ingres, and - soon - Sybase and MongoDB).

Improve this answer
0

If you decide to run you app outside of GAE, you can still use Django. You won't really have that much luck with the GAE webapp

Improve this answer
1
  • Thanks, though I mention exactly that in the original question: "Finally, it's clear that using Django does have the advantage of providing an "exit strategy" if we later wanted to move away from GAE and need a platform to target for the exodus."
    – Travis Bradshaw
    Commented Dec 22, 2009 at 4:57
0

I am still very new to Google App engine development, but the interfaces Django provides do appear much nicer than the default. The benefits will depend on what you are using to run Django on the app engine. The Google App Engine Helper for Django allows you to use the full power of the Google App Engine with some Django functionality on the side.

Django non-rel attempts to provide as much of Django's power as possible, but running on the app-engine for possible extra scalability. In particular, it includes Django models (one of Django's core features), but this is a leaky abstraction due to the differences between relational databases and bigtable. There will most likely be tradeoffs in functionality and efficiency, as well as an increased number of bugs and quirks. Of course, this might be worth it in circumstances like those described in the question, but otherwise would strongly recommend using the helper at the start as then you have the option of moving towards either pure app-engine or Django non-rel later. Also, if you do switch to Django non-rel, your increased knowledge of how app engine works will be useful if the Django abstraction ever breaks - certainly much more useful than knowledge of the quirks/workarounds for Django non-rel if you swap the other way.

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.