Archive for October, 2010

Prototyping a todo list in Google App Engine

Thursday, October 14th, 2010

In this walkthrough we’ll build a simple to-do list on Google App Engine so that users can log in and add new items to the to-do list, which are stored and displayed back to them later. (Much of this walkthrough follows the Getting Started documentation from the Google App Engine docs, and I’ll refer to those sections as they match up.)

The first thing to do is to install Google App Engine (available on Mac, Windows and Linux) and the Google App Engine Launcher (available on Mac and Windows platforms). These instructions for installation are helpful; you’ll need Python 2.5 installed, on a Mac it should already be present.

Once it’s installed (on a Mac be sure to copy the App Engine Launcher to your desktop rather than running it from the disk image), open the Google App Engine Launcher, which will show an empty list. To create a new application, click the + button. Enter the name of your app (use a name that’s all lowercase letters and numbers) and where you want to store the files (putting this in your repository would be great, for example).

This will create a folder with three files in it: app.yaml (for configuration), index.yaml (for advanced use of the database) and the file we most care about where our server-side code exists. You can open this folder in the Finder/Explorer and open in your favorite Python IDE or text editor.

Back to the App Engine Launcher: select your application from the list, click “Run” and (hopefully) the small icon next to your app will turn green. Now just click “Browse” to open the application in your browser. “Hello World!” without writing a line of code. Congratulations.

Also, now your application is running on a mini-web server on your machine. This is a huge advantage because you can quickly and easily make changes to the code on your own machine and immediately see the effects by refreshing your browser. You don’t need to install, maintain or configure a web server, App Engine’s mini-server is fine. To test this quick updating, change “Hello World” in to “Hello IO Lab” and refresh to prove to yourself that updates work.

Now, Google provides all these neat services, so let’s take advantage of one: users. This way we can welcome the user personally rather than welcoming the entire class. (Getting Started explanation | Reference Documentation)

Copying from the documentation, add this new Python library import at the top of your

	from google.appengine.api import users

And inside the get(self): method, let’s replace the existing single line with the following:

    user = users.get_current_user()

    if user:
      self.response.out.write('Hello, ' + user.nickname())

(Be careful about indentation. Python requires that all indentation be done the same way: either all with spaces or all with tabs. And indentation must line up or your program will fail to execute.)

If you refresh the page now, your local mini-server will come up with a little fake login page, where you can enter any username you want and click “Login”. You should then be welcomed by email address. How would this work in the real world? Let’s upload our app to Google and see. (Getting Started docs)

From the Launcher, click “Dashboard” for this app. Oh noes, it doesn’t exist! Click “Return to Applications screen” to see your list of Applications (you’ll probably have none) and click the button to “Create Application”. Pick an available subdomain name (which is also called your appid) and give your application a title. If this is your first app, Google will make you jump through some hoop that includes SMS verification. This shouldn’t take too long, but you can keep testing on your own machine without uploading, if you want.

You’ll need to specify your selected appid in the app.yaml configuration file in your application’s folder. Enter it after application: .

Now, back in the Launcher, just click “Deploy”, enter your Google username and password and your application files will be uploaded to Google’s servers. Then go to http://<your-appid> and your code is running, except with a real Google login. Congratulations, your application is running in the cloud! Clicking “Dashboard” in the Launcher now will show you a web page with stats and various information about your app’s execution on Google servers.

Most of the time, though, we want to test by using the mini web-server, which picks up changes automatically. So let’s switch back to that by clicking “Run” again. (Deploying your application to the production server will stop it on your mini server.)

Now, we’re combining our presentation and behavior here, which we’ve told you is a bad thing. So let’s move the presentation into an HTML file and keep just the logic in Python.

Create a new file in the same folder called index.html (the HTML skeleton file on the course web site is a good place to start). Here you can fill in the <title> element, make things look nice, and not be mixed up with Python. We’ll leave a little placeholder for the user’s nickname, using a templating language. (Getting Started docs | Django Templating docs)

    <h1>Hello, {{nickname}}</h1>

Now, in, we’ll import two additional libraries:

    import os
    from google.appengine.ext.webapp import template

And then render the template with values we store in a simple Python dictionary:

    nickname = user.nickname()
    template_values = {'nickname': nickname}

    path = os.path.join(os.path.dirname(__file__), 'index.html')
    self.response.out.write(template.render(path, template_values))

Refreshing the page in your browser now should show a nicely formatted page with a large heading welcoming the user by their nickname. Now let’s make this a little more interactive, by adding a form where a user can add a to-do item. In index.html:

	<form action="" method="post">
		<input type="text" name="item" value="To-do item"/>
		<input type="submit"/>

I’m using action="" just because it’s the simplest possible thing: it’ll call back to the same page, but use post instead of the usual get, since we’re submitting data. To respond to the post form submission, we define a new function in the same class:

    def post(self):

Just enough to prove that it works.

Now, we’ll actually do something with the entered data, like storing it. (Getting Started docs | API Reference) To do this, we need to define a Model, a representation of the data we’re going to store. Import the db library, then define the Todo class which will be stored in the database.

    from google.appengine.ext import db

    class Todo(db.Model):
      author = db.UserProperty()
      item = db.StringProperty()
      completed = db.BooleanProperty()
      date = db.DateTimeProperty(auto_now_add=True)

And then use that class to save new Todo’s in the post function:

    todo = Todo() = users.get_current_user()
    todo.item = self.request.get("item")
    todo.completed = False


    self.response.out.write("Saved: " + self.request.get("item"));

Now, that data has supposedly been stored, let’s see it. Back in the Launcher, click “SDK Console”. In the Datastore Viewer, you can choose a type of entity and then list the entities in that store. (When you’re running live on the server, you’ll have a similar view from the Dashboard.)

Let’s expose that data back in our little web app. Change the get handler to query for all the Todo items in the datastore and store them in the template_values dictionary:

    todos = Todo.all()

    template_values = {'nickname': nickname, 'todos': todos}

And then change the template to loop through them and create list items for each one:

    {% for todo in todos %}
    	<li>{{ todo.item }} by {{ }}</li>
    {% endfor %}

Great, now when we add something and go back to the main page and refresh, we see the new content. Let’s make that step automatic though.


(This goes at the end of the post function; be sure to remove or comment out any self.response.out.write lines.)

What else could we add to this? You could add CSS and Javascript to this HTML page (and with a small workaround, you can add them as static files). Since these are to-do items, you could add checkboxes and make clicking a checkbox into an AJAX post that would mark the item as completed. You could filter to only show recently added todo items, or only your todo items.

A working version of this code is available in the repository.

Working around the same-origin policy

Sunday, October 10th, 2010

As part of the basic security model of the Web, sites can’t usually make requests to pages on other domains — if they could, then just visiting any random site on the Web having recently logged in to your email could reveal the entire contents of your email to an attacker! In class we briefly mentioned three ways to work around the same-origin policy:

  1. Use a server-side proxy.
  2. Make a JSONP request to a server that supports it.
  3. Make a request from a privileged context (like in a Chrome Extension).

This blog post will cover generic uses of the first two methods. If you’re creating a standalone page for your projects, you’ll either need to use APIs that already support JSONP, or use our provided proxy to call an existing API for you and wrap it in JSONP. Ryan’s walkthrough tutorial uses a version of this technique for posting new bookmarks to Delicious from a standalone page, but here we’ll access any generic JSON API.

The internal details aren’t vitally important, but in case you’re curious: JSONP works by loading a new <script> element to the page, where the contents of that <script> element just happen to be (no, I’m kidding, it’s not the least bit coincidental) calling a function with a name you defined with a single parameter which is the response from the API. To take advantage of this in jQuery (which, as usual, does all the hard work for you), just use the $.getJSON() function and include a special callback parameter in the URL '?callback=?'.

To use this with the New York Times API, for example, which doesn’t support JSONP, we instead make a JSONP call to a proxy on our own Berkeley servers and that proxy makes the call to the New York Times and then responds using the JSONP callback standard described above. We just need to pass the URL of the New York Times API and all the query parameters as parameters to the proxy.

var query = $('#search').val();

var apiKey = 'yournytimesapikey';
var proxyUrl = '';

$.getJSON(proxyUrl + '?callback=?', {"url": '', 
                                     "query": query, 
                                     "api-key": apiKey}, 

The full sample page is in the iolab10 repository, including the full path to the PHP proxy that we’re running and even a sample NYTimes API key. We’ve also made the PHP code for the proxy available in case you want to inspect it or modify it for your own purposes. (If you install and run the proxy on the same domain that your page runs on you don’t even need to use JSONP!)

A couple of caveats:

  1. If you’re using JSONP from a content script in a Chrome Extension (or from a Greasemonkey extension, for that matter), you’ll receive an error that a function named json123456789 (or some similar nonsense name) doesn’t exist. This is because jQuery created the callback function in its own sandboxed area, but when the script was inserted into the page it called a function in the original window context. To work around this, cross-domain requests in Chrome Extensions shouldn’t use '?callback=?' and should be made from the background page or a pop-up page with cross-domain permissions declared in the manifest. (For more detail, see the relevant Chrome Extensions documentation.)
  2. When you pass a URL to the proxy as the ?url= parameter, the URL itself shouldn’t include the parameters you’re passing on to the API, those parameters should just be additional parameters to the proxy page. Really, this just means that you should use an ampersand for the first parameter to the API. For example, call proxy.php?url= rather than proxy.php?url=
  3. There are potential security implications here that we haven’t gone into yet. Requests through a PHP proxy on Berkeley servers may be logged by Berkeley, which you might not want (particularly if you’re passing a key, password or other secret data). And if the address for your (or our) proxy becomes widely known, it could be abused by others for denial-of-service or other malicious purposes.