May 13 2010

Publishing a REST API?

Aww, Facebook. We all know your backend is a Frankenstein of legacy foundation overlaid with new-and-cool features that could probably make the LiveJournal codebase actually appear organized. In day to day usage of your service, it doesn't really matter. Somehow it is all glued together into a cohesive experience for the end user, and I rarely notice the .php extension in the address bar. And your user interface team remains so very good. (I mean that.) It shows, however, when a developer cracks the hood of your REST API.

So, so ghetto.

And you're not alone. I'm noticing a disturbing trend with API designers. I'm looking at you, Twitter. Don't think you're getting away with it either, Foursquare. You guys are the cool kids. You shouldn't have a bunch of cruft lying around. You still can fix this, and your developer userbase would actually appreciate you for it.

Here's the gist: REST is designed to piggyback on HTTP, and HTTP is surprisingly full featured. You get an amazing array of features available to you as part of the protocol, like document caching via last-modified times, byte-range fetches, pipelining requests, a huge assortment of authentication options, and content negotiation. You can pick and choose what you want to support, and HTTP provides for a standard way to accomplish it. HTTP compliments the REST design philosophy, and if you're creating a REST API that doesn't take advantage of that fact (or worse, re-invents portions of it), then you just look either lazy, or like you never read any HTTP RFCs.

HTTP verbs

A verb is a synonym for an HTTP method. Given a URI, what is your intent? Reading it? Adding new content to it? Removing it altogether?

HTTP has a lot of verbs. But for purposes of REST, it's pretty straightforward. We'll just use 4 of them, which should cover 95% of the REST use case:

GET     Read from a resource
POST    Create a new resource
PUT     Update an existing resource
DELETE  Remove a resource

Foursquare gets this largely right (GET a list of checkins vs POST a checkin), as does Twitter. Both strangely omit the PUT verb completely though, using POST for all writes -- regardless of whether the write is creating new content or updating existing stuff. Even Wikipedia confuses what PUT should do, but the HTTP RFC is clearer. PUT is for updates!

There is even a handy dandy error code dedicated to HTTP method handling. As an example, if you try and POST/PUT to a resource that should always be read only, your server should respond with a 405 Method not allowed, along with an Allow header that lists the verbs you can use at that URI. See? HTTP takes care of you like a gentle lover, and any decent HTTP client (web, desktop, or whatever) should handle your API in the correct fashion.

With the exception of adding video, Facebook's API completely ignores the concept of verbs. Everything is a GET request, and the verb is embedded into the resource. getSomething. addSomethingElse. At that point, you're immediately not using REST. You're just publishing a loose collection of random URIs and calling it an API. You might as well be using SOAP, because that's what you just reinvented, without all the XML suck. Oh, wait... that brings me to

Content Negotiation

Everyone is getting this wrong. Here's the litmus test -- if you need to embed your desired response format in the resource URI, the API is being lazy.

Again, HTTP supplies a standard method to do this. A client can send an Accept header, that can even specify order of preference it can deal with the data. All format types are MIME. Realistically, most clients will have such a strong preference for a specific serialization format, that they won't ever use Accept parameters or qvalue weights. They'll just say "gimme the resource in application/json format!" and the server will say SURE. Alternatively, the client requests a format that the server can't comply with, and lo and behold -- there's an error status for that, too -- 406 Not Acceptable. Servers, if they want to be -really- nice, can provide alternate resource URIs in the body of the error that explain to the client what to do instead, in the originally requested format!

If the APIs actually used this handy header, then they could rapidly start supporting things like compressed JSON streams, or new serialization formats on the fly, all while continuing to support the other client applications out there that don't support the new and fancy formats. This opens up the API to a whole audience of clients that might normally not be interested, while keeping your resource URIs completely unchanged and static.

Facebook's new Graph API gets around the whole mess of serialization formats by only supporting JSON. Ever. Want something else? Working with a language with subpar or no native JSON support? Too bad. I personally really like JSON, but why would you intentionally cut off other formats when it would be so easy to support?

POST (kinda)

I update a plan file regularly, and script updates in my plan file to Twitter. (Then a FaceBook application syncs from Twitter.) This way, I can continue using my "update the world before social networking existed" technology, while still telling Mom what's up on FaceBook. That's right. Get off my lawn!!

I like calling Twitter updates 'twits'. Twits + plan file? Plits! A CLIENT IS BORN. Here's what a status update URI looks like to Twitter.

Ok, what the hell is this lame excuse at REST? Why do I have to include my 'source' in the URI, when there is a perfectly good User-Agent header? Why on earth must I embed the content of the POST (thank you, it IS at least a POST) in the URI after encoding it appropriately? This is HTTP dammit! It has a very solid concept of a content body!

Just look at how pretty this would be if REST ideals were actually applied.

POST /statuses/update HTTP/1.0
Accept: application/json
User-Agent: plit
Content-Type: text/plain
Content-Length: 13

I eat burritos

What if someday, Twitter suddenly supports posting audio updates to your feed? Like a giant social-networking Nextel walkie talkie! (Please, for the love of all that is holy, lets hope this never happens.) Watch the magic.

POST /statuses/update HTTP/1.0
Accept: application/json
User-Agent: plit
Content-Type: audio/x-wav
Content-Length: 273

[audio stream here]

Notice that the API is essentially unchanged! Twitter could simply reply with what formats it can support, and the client can do the brainy work of converting the audio to a supported format. How would this ever work with putting content in the URI?

Much better dev experience, and less work for developers making clients for your services. A clear path for you, the service publisher, to optionally provide all sorts of rich media options, while supporting protocol standards between clients.


Please guys, lets take advantage of HTTP if you continue using it as a transport. Or if you stay the course, stop calling what you have REST. It's misleading and just plain... not.