The Highs and Lows of REST
By Ryan Tomayko under Then you win. on 19. March 2006Discussion on REST seems to be moving in a constructive direction all over the place with Tim Bray and Don Box both contributing. I think Dare Obasanjo’s notes from a talk given by Google’s Nelson Minar almost a year ago may have put some valuable terminology behind ideas that have been floating around for some time and may have even brought us to a place where we can have an actual conversation about this stuff:
To begin he defined what he called ‘low REST’ (i.e. Plain Old XML over HTTP or POX) and ‘high REST’. Low REST implies using HTTP GETs for all API accesses but remembering that GET requests should not have side effects. High REST involves using the four main HTTP verbs (GET, POST, PUT, and DELETE) to manipulate resource representations, using XML documents as message payloads, putting metadata in HTTP headers and using URLs meaningfully.
The definitions of low REST / high REST could be tweaked a bit (maybe they were when I wasn’t looking?)
but I do like the idea of putting terms we can all agree on to the
subset that’s the reality behind most of machine/web interfaces that
seem to be working today. POX
isn’t that term because it unnessarily constrains the media type; “low REST”
should include the mostly non-XML web (i.e. the text/html, image/*,
text/css, text/javascript, etc. web) where it conforms to the
constraints of GET and POST.
The reason I like this approach is because it could remove the artificial and often harmful separation made between the web people interact with and the web machines interact with. Exposing resources for machine consumption isn’t any different than exposing resources for human consumption until you get down to the media type (and not then if you subscribe to Microformats). That is, some form of hyper-media (links / URIs) and a simple, universal interface (HTTP) are the really big wins you get with REST because those things are what creates the possibility of having a hugely distributed and uncoordinated system of interaction. The choice of document format is something else entirely and the decisions there ought to be based on a whole other set of concerns.
If I cared at all about the term anymore, I would go so far as to say that anything conforming to the constraints of at least one HTTP method and that also exposes references to other resources using URIs should be considered “a web service”. But “web service” has been defined to death and it doesn’t seem to be helping.
The first time I heard someone attempt to describe the GET/POST subset as something we should pay attention to was Michael Champion on xml-dev. This was usually thrown out when someone championing REST used the “REST is the web and the web is working” argument (I use that argument to this day, btw). Michael would often be the one to point out that it wasn’t exactly true and that a subset of what REST describes seemed to be working well on the web (GET/POST, URIs, caching), while another subset was clearly—sometimes knowingly—being violated (GET w/ side-effects, uncool URIs), and yet another subset was used only rarely (PUT/DELETE/OPTIONS, content negotiation).
Not being able to distinguish which of these subsets of applied REST one is refering to is hurting our ability to have meaningful conversations about this stuff. For example, consider the following conversation; I’ve seen it hundreds of times now:
Restafarian: 80% of developers use the XXX REST API because they find it easier to deal with.
Tool Vendor: That’s not really REST. It only uses GET and POST and breaks rule #5034
Restafarian: Oh yea, well SOAP/WS-* sucks.
Tool Vendor: Go write some PHP you dirty hippie!
I’d like that to stop happening.
But I don’t think adopting wrong or ambiguous terminology is going to help either. I’d like to register my nits with the few sentences that Dare provided and see if anyone else is thinking along similar lines.
To begin he defined what he called ‘low REST’ (i.e. Plain Old XML over HTTP or POX) and ‘high REST’.
Like I said before, POX is kind of, ummm, bad, I think. It focuses too much on the format. XML is not why the web is working. URLs and HTTP is why the web is working. HTTP and URLs work so well they’re even capable of being useful in spite of complexities in media-types based on XML.
Low REST implies using HTTP GETs for all API accesses but remembering that GET requests should not have side effects.
Fully constrained GETs are really important but POST should fall under low-REST too. By the definition above, low-REST can never modify anything. Throw POST in there, it’s a big part of the present day human/web and is likely to play a bigger role in the machine/web.
High REST involves using the four main HTTP verbs (GET, POST, PUT, and DELETE) to manipulate resource representations, using XML documents as message payloads, putting metadata in HTTP headers and using URLs meaningfully.
I think we ought to downgrade XML to a “bestcommon
practice” or something. Microformats and JSON are two notable examples
of non-XML representations that seem to be working quite well and are
absolutely capable of being used high-RESTfully.
I’m pretty sure I’ll always be a high-RESTafarian but I could definitely see myself coming out as a lo-RESTafarian as well if some of this were grinded out a bit more.
Aristotle Pagaltzis:
Why the XML hate?
Microformats are an excellent approach to enable serendipitous scraping of information intended primarily for human consumption so that it can subsequently be reused. This is what the contemporary open web consists of, so putting more of it in microformats would be great. I would never store data in HTML-based microformats, though, neither as the producer (in whose position I’d keep the data in more structured storage and only render it to microformat via templates prior to serving) nor as the consumer (in whose position I’d scrape the data off the page and then dump it into a more structured and more easily queried store). I’d always prefer to work with an API that provides the same data in machine-readable format. The strength of microformats is broadcasting: they make a great generic back-channel for apps not versed in specific APIs and wire formats.
JSON’s strength is expressing highly structured data where the structure has low variability: basically, for pushing objects across the wire. JSON makes a great choice with minimal fricition for off-the-cuff APIs.
But for standardized APIs that will have many different implementations on both ends of the wire, XML really is the wire format of choice. Namespaces provide a builtin hook for implementation-specific extensions. It goes without saying that XML/SGML-type markup is great for weakly structured data such as documents – XPath also makes it suitable at the other extreme of the spectrum, for highly variable nestable structures. (I love this little language more than anything. If I never have to write another piece of ad-hoc tree-walking code to locate bits and pieces in a tree, it won’t be soon enough.) In addition, doing XML means that meeting the minimum requirements for i18n is a prerequisite. (What’s the status of JSON with regard to charsets? I admit ignorance here.)
If you want to see a great example of the stuff I’m talking about (high-REST POX that plays to XML’s strengths), read the Atom Publishing Protocol spec, which is winding down to its finishing touches. That stuff’s gonna catch on, tell you what.
comment at 19. March 2006
Aristotle Pagaltzis:
On that note, I have to add that I already love this low-REST/high-REST terminology. It feels natural, and it does indeed make talking (and thinking) about the topic easier. The importance of names and naming…
comment at 19. March 2006
alexking.org: Blog > Around the web:
[…] The Highs and Lows of REST [@lesscode.org] […]
pingback at 19. March 2006
Ryan Tomayko:
Aristotle: XML definitely hits a sweet spot with all the things you mentioned (that’s exactly where my head was w/ Kid) but doesn’t it seem like we’ve been putting too much emphasis on content type? I wonder if things might have worked out better if the black-hole created by XML didn’t dwarf all the other considerations.
I think RSS is actually a perfect example of what I’m talking about. It sucks, is almost never directly parsable XML, has compatibility issues too numerous for a normal person to fully understand, and yet it’s still doing really well somehow. It exhibits very few, if any, of the benefits we’ve come to associate with XML. I try to rationalize the situation by telling myself that HTTP and URIs have to be the key ingredients and that, in practice, there’s a high level of tolerance for bad, broken, incomplete, etc. media types. I’m completely behind Atom in every way but I think RSS says a lot about the nature of the web.
And Microformats is probably a bad example of a non-XML representation since they are based on XHTML (but without the media type). The point I was trying to make, though, was that you should be able to attack the question of document format based on your needs starting with the simplest thing that can possibly work. If you can extend your existing HTML markup a bit and get 80% of the integration capabilities you require, that’s probably going to be a good decision. If, later, you find yourself requiring that additional 10%-15%, you roll something in alongside your Microformats approach using a more specialized XML format (e.g. actually use content negotiation or have a whole other URI hierarchy, whatever). If you’re into that top 5% range, you start considering how you might make the various WS-* specs work with the web.
That’s the approach I like to work with and I just wonder if the constant focus on XML for so long hasn’t twisted our minds up in a way that makes it hard to see what’s really important.
comment at 19. March 2006
Alex Bunardzic:
Ryan,
You are one of the rare techno geeks I know who keeps insisting on placing common sense at the very top of our concerns. I like that. God knows it’s incredibly hard to get geeks in general to become sensitive to and to start differentiating between what’s possible and what’s probable.
Even if we’re building a system that’s 98% idempotent in its functionality, the geeks will still hijack the proceedings and force the 98% of operations to adopt the heavy load that would satisfy the remaining 2%. That’s definitely the biggest source of all my frustrations when working with software development teams.
You’ve nicely illstrated that with your analysis of the lopsided emphasis on XML.
comment at 19. March 2006
Koranteng Ofosu-Amaah:
On matters of advocacy I suggest we should stick to the REST elevator pitch.
comment at 19. March 2006
assaf:
I think the best advise here is “starting with the simplest thing that can possibly work”.
In some cases, I get most mileage out of using microformats, and everything else is just a layer of indirection. But microformats have their limits, and in other cases XML works better. RSS are great for some stuff, they’re an overkill if you’re doing an authentication API. And let’s not forget JSON.
You can write a whole article on what to use when, I think that would be useful. But the key point is discretion: use all of these, but for each use case pick the ones that are most effective.
The only barrier to using XML and RSS and microformats and JSON is learning. The barrier to using XML when you should be using microformats, or RSS when you should be using JSON is complex code that’s difficult to write and maintain.
As for REST, I follow the same approach. Which means most of my Web stuff break REST by relying on cookies, and most of my APIs break REST by asking you to construct URLs. And so far I’ve only had use for GET/POST.
I call it lightweight services. It’s about keeping services simple to implement, use and maintain. And it so happens that a lot of REST gets used because REST fits the bill.
comment at 19. March 2006
Aristotle Pagaltzis:
Ryan,
I am strongly convinced that the reason that RSS seems to be holding up (which I will assume means RSS 2.0, since earlier versions are RDF, even if noone parses them that way) is that it hasn’t been pushed. It’s not used for anything much other than shovelling weblog content into readers, which content consists mostly of tag soup anyway. So the fact that RSS seems to work is really no more to the point than the fact that HTML tag soup seems to work.
I do agree with your basic thesis that REST succeeds because of the REST principles and does not depend much on the wire format. The position you put forth in the article just seemed to veer into “we’d be better off without XML,” rather than staying on the “XML isn’t what matters” course, so that’s what I addressed in my comment.
Alex,
don’t be silly. You won’t find many who will argue that excessive model complexity is worth its diminishing coverage returns. People aren’t stupid. The problem is that one man’s 98% are another’s 55%. People disagree about how much of the problem space constitutes the 80% worth addressing. Your portrayal of the problem conveniently skips over the hard part; unfortunately, reality is rarely that tractable.
comment at 20. March 2006
Alex Bunardzic:
Aristotle wrote:
All I was talking about is an idempotent portion of a system (an exactly measurable feature). How you inferred that one man’s 98% of idempotency is another’s 55% is beyond the grasping capabilities of my feeble mind.
comment at 20. March 2006
Aristotle Pagaltzis:
Care to give an example?
comment at 20. March 2006
Aristotle Pagaltzis:
Ryan,
Excellent commentary on the high/low REST coinage:
Mark Nottingham: Don’s False Choice
comment at 20. March 2006
Quoderat » The REST schism and the REST contradiction:
[…] Pick one, and only one of these arguments, please. As far as I can see, apart from a few rare exceptions (like WebDAV), Don’s lo-REST — HTTP GET and POST only — is what’s been proven on the web. The pure Book of Fielding, hi-REST GET/POST/PUT/DELETE version is every bit as speculative and unproven as Web Services/SOAP/SOA themselves (that’s not to say that it’s wrong; simply that it’s unproven). Some REST supportors, like Ryan Tomayko, are already starting to acknowledge this contradiction. […]
pingback at 25. March 2006
Patrick Logan:
The amount of “stuff” in the less-used aspects of HTTP is a good deal less than the amount of “stuff” in SOAP/WSDL/WS-* so I am not sure it is fair to say it is every bit as unproven. It seems to have less to prove. That also reflects my impression than WS-* seems to be overly ambitious.
comment at 25. March 2006
Sandeep Shetty:
A lot of web developers fail to see this.
comment at 01. April 2006
Let’s REST! » Wagalulu - Microsoft » » Let’s REST!:
[…] "Halloween Problem" for XML APIs » Bookmark on del.icio.us A bunch of links to RESTful resources I’ve collated here and there - should be a goodstarting point to get a handle on some of the basics of REST, with links to example APIs / documentation as well as some posts ReprESenTative of the discussion in this space, plus bouns randomly related stuff (rss). If there are some obvious holes here, let me know…. REST. Representational State Transfer represents an architectural style for building distributed applications. or Really Extremely Simple Transfer. Intro to REST Roy Fielding - Architectural Styles and the Design of Network-based Software Architectures, Chapter 5 Ryan Tomayko - How I explained REST to my wife Wikipedia - REST Sam Ruby: How to Create a REST Protocol To REST or not to REST REST and Web Services: The ZapThink Take ZapFlash “In many ways, however, the debate about Web Services and REST is as pointless as arguing whether a hammer or a screwdriver is a better tool.” Paul Prescod: on REST - Second Generation of Web Services Don Box: SOAP vs. REST Dion Hinchliffe: Creating Open Services That Last (And Anyone Can Use) Danny Ayers - Is REST too complicated? Ryan Tomayko - The Highs and Lows of REST REST APIs, examples, documentation and bits eBay’s REST API Amazon REST API Tagtooga’s List of REST APIs Random, but related, RESTful thoughts: John Udell - Amazon’s pragmatic approach to metered infrastructure “SQS is a Web-based queue to which you post messages and from which you read them back — without worrying about pesky details such as scale, concurrency, reliability, or guaranteed delivery.” Tim O’Reilly - A Week in the Valley: GData “The big point for me was that GData is just Atom/RSS for reading, Atom Publishing for writing, and A9 stored queries for searching.” Optimistic concurrency (versioning) (We’re not talking about dating here….Google Data APIs Protocol) sqlREST and 15 minutes Guide to sqlREST Don Box: HTTP, XML, REST and $100 - Popular REST tagged articles in Del.icio.us […]
pingback at 21. July 2006