lesscode.org

There’s a big battle of words raging on to define what the hell we’re doing and why we’re doing it all wrong.

Are Web services on the Web, or do they have to use SOAP? How is low REST different from high REST? Does XML/HTTP work better if we call it POX? When is AJAX not AJAX? Who owns the semantic landscape?

There are also a lot of people building applications, that don’t have time to argue how you call it or what it means, just as long as it works. We call them, “the pragmatics.” This post is for them.

When it comes to designing Web services, there’s a few choices of architecture style, and stacks of technologies to choose from. It’s still undecided which one will rule them all, the race if far from over. Most people who write about that stuff hope their horse is the winning one.

I’m no exception. So I’m going to play pundit and tell you which architecture style I think works best for the Web, which technology stack I prefer to use:

• Architecture style: use the simplest most common solution to solve your problem.
• Technology stack: see above.

XML over HTTP (can we please not use an acronym that means a disease?) is great for moving structured data around, and shines at encapsulation. Its loosely coupled document style has too many merits to mention. Except when you’re building a UI that needs to talk back to the server, where tight coupling is good enough, and too much abstraction means you’ll never deliver on time.

RSS is a wonderful way to stream data updates from server to client, not to mention the magical combination that are blogs and feed readers. It’s the ultimate query API. Except when your UI is trying to query the number of unread e-mails in your inbox, and you find out that RSS is anything but simple.

Microformats are amazing in their beauty and simplicity, and let you easily microchunk your content. Until you fall in the 80% of problems they were never intended to solve and… well, just don’t solve.

JSON (not to be confused with Jason, though they both love AJAX) is the simplest form of API you can think of, concise and easy to use. It’s a wonder it wasn’t invented earlier. Except that it’s tightly coupled and still envious of what XML can do for you.

AJAX is great even though almost everybody is using it for HTML. But let’s not get too tangled up in detail, acronyms are not for describing but for branding.

There’s nothing like REST, it powers the Web (can we get a logo for that?). You’ll just have to ignore all those big name Web sites we all like to mention that use cookies for the benefit of their users. And all the REST APIs that violate more constraints than a drunk driver speeding down the wrong side of the road.

RDF is semantic bliss, it gets semantics right and has a thoughtful model for representing and querying it. You can do amazing things with it. But most likely your semantic data is not RDF, and still happens to work, and you’re cursing every time you need to fix Firefox configuration files in all their semantic glory.

The Web thrives not because it uses a strict architectural style and a coherent technology stack. It thrives because so many sites pay little attention to REST and choose to focus on their users instead. It thrives because malformed HTML pages include GIFs and PNGs and Flash and badly written JavaScript that just works. It thrives because people go on the Web to send e-mail, IM, do VoIP and trade BitTorrent files.

The search for the holy grail, the one technology to rule them all, is as old as technology itself. I’m fine knowing we’ll never settle the score, never know whether vi is truely better than Emacs, or whether Eclipse shadows them both. But unless you’re a vendor making money on one horse, it doesn’t matter to you.

If you’re a pragmatic developer, you have one tool in your toolbelt that always wins the day. It’s the ability to think, ask questions and make choices. To choose solutions that are best for the problem you’re tackling right now. And to keep learning. Because there will always be some new technology that’s better at solving some use case or another.

The only architecture that matters is the simplest one you can get to solve the problem at hand.

This post: written in Vim, formatted as HTML, available over RSS.

I wonder if it’s time to adopt a variation of Gandhi’s famous formulation: “First they ignore you, then they laugh at you, then they fight you, then you win”. It goes something like “First they ignore you, then they keep on ignoring you but you win anyway”. One nice thing about the new formulation is you get to skip the whole fighting part. I think Gandhi would approve.

On March 7, in his talk at the O’Reilly Emerging Technology Conference, Ray Ozzie, CTO Microsoft, demonstrated (short screencasts, live demo page) a method by which users can cut/copy/paste structured content between web applications and between web applications and Windows ones. Alert lesscode readers will notice that Ray’s blog post describing his inspiration and motivation for the idea bears a striking resemblance to my own post from last October. A deeper look at the screencasts, the live demo and the technical introduction remove any doubt that what Ray demonstrated, and in fact what Microsoft is offering under the Creative Commons Attribution-Share Alike license (you heard me right) is in fact the Web Application Clipboard as described in that original lesscode post and followup.

This is in no way meant to diminish the Microsoft folks’ brilliant implementation. The portable trick used to actually hook the browser’s built-in cut/copy/paste functionality is inspired. The design tradeoff resulting in a visible representation (scissors icon) of every clipboard-capable element is all their own, and quite frankly, is probably a necessary tradeoff if the Web Application Clipboard is to actually get traction. There’s a more technical analysis posted on my personal blog, but the executive summary is:

Live Clipboard (Microsoft’s name for Web Application Clipboard) is awesome and I believe sites will adopt it. I wish the name weren’t “infected” with the Windows Live brand but that won’t matter a whole lot so long as the specification is good and open. Right now though I can’t find any specification at all but I’m keeping hope alive on that front anyway (you listening Gandhi? MLK?)

What I find more interesting than that technical stuff though is the amount of mindshare this thing got “all of a sudden”. First off, note that Ray’s March 7 blog post on this subject comprises the only thing he’s blogged since the end of January. A whole month of the Microsoft CTO’s time — how many dog-years is that? Now realize that the good guys over at the Gillmor Gang podcast spent a full 48 minutes on this single topic on March 16 (Ozzie Gang I, Ozzie Gang II), most of which was spent interviewing Ray Ozzie himself. They were downright effusive. From Ozzie Gang II:

• Mike Arrington, 4:45

  Does this signify any sort of shift organization where people in microsoft who have ideas that can help the Web… that they can also get things through the politics and through the system fast enough so it doesn’t take and years?

• John Udell, 17:52

  Ray this is great. I just personally want to thank you for doing it. I think it’s a tremendous step.

• Steve Gillmor, 17:57

  Yeah, and I think that everybody is knocked out that somebody with your intuition is in such a powerful position at Microsoft. You know, bringing Microsoft into the community has been very difficult to do and you seem to be doing it.

• Dan Farber, 18:31

  I think as everybody on this call has said, we’re quite impressed with this very simple little notion that seems to be well executed and that it comes from Microsoft now really injecting itself into the big old Web community. It’s cool!

• Dan Farber, 19:13

  If you go to the desktop and you consider cut and paste, copy and paste, you know it’s everywhere… to be able to do that on the web… it’s like a big gift — it’s a contribution to the Web.

• Mike Arrington, 22:18

  I think the big story here is that we’re starting to see with Ray’s leadership that Microsoft is able to do things that seem selfless and simply good for their own sake and SSE is one example and (Live Clipboard) is another stunning example…

• Steve Gillmor, 24:05

  I think that Microsoft may well be on the threshold, whether they fully realize it or not at all levels of the organization, of winning by accepting a part in the community, and I think that’s a huge story.

Strange and wonderful times indeed.

In public and private people whom I would have never expected, are now getting interested in how to roll out in the REST style. “Restian” is the term I hear bandied about. Not “RESTful”. Not “RESTafarian”. “Restian”. I like it.

This is good news, though I’m a bit taken aback it’s happening so soon. Some people would say it should have happened half a decade ago, but I’m something of an optimist.

REST has always been a solid basis for designing distributed systems, it comes with a theory (I know! It’s crazy!), appropriate and specific to the web domain. It has not been a populist approach, which can be summed up with – “where are the REST toolkits?”.

All that is about to change. REST has won. Fire up your editors.

Have nothing in your house that you do not know to be useful.

Now what?

When REST becomes a mainstream/default development approach, as opposed to a fringe/implicit one, a couple of things come to mind on the plus ca change front. First off, I think you’ll see better support for URI design and mapping onto code in server libraries. Also, better access to headers (especially caching and timestamping directives). More awareness and support for the full range of HTTP methods and response codes. Attribute modifers and declarations on methods that have side-effects. Exception designs based around 5xx and 4xx response codes. Media type dispatching. Less emphasis on cookies, more support for digest authentication. More flexbility in LAMP stacks for writing out stuff that isn’t HTML (esp. JSON and Atom serializers). That sort of thing. Overall a gradual lifting on HTTP/REST concepts and idioms into application code.

But probably the most important initial effect is how minimal and frugal REST will seem to the working developer.

This Spartan Life

If REST has an analog in building architecture, it’s Modernism. On the web, hard lines are where it’s at, and less, really, is more.

REST is about design constraints – what you can’t say. That’s contrary to industry practice, where it’s all about about getting cool new features piled on, and to heck with principles. If anything, most developers are cynical about architecture and architects. Developers, Developers, Developers. Give ‘em what they want. Keep ornamenting the stack with pointy-clicky solution cruft.

There’ll be a bit less of that. On the web, there are less assumptions you can make and those you can make are rigid and need to be treated as invariant. Developing for the web, as opposed to developing around it, is spartan work. Coming from middleware and enterprise development, could be a bit like coming from the Victorian ornateness and richness of Ruskin and Morris to the urban cold and minimalism of le Corbusier and Mies van der Rohe. It’ll be disorienting at best.

For example, tool support will go from “non-existant” to “sucks”, initially.

All one can say is that the laws and liberties, such as they are, which govern local/LAN-based software development don’t always apply when it comes to Internet based development. In point of fact they’re often a hinderance, and there’s a history of getting burnt on bad assumptions on the Internet. You can try to make inter-network, cross-administrative systems development transparent, to look and act like middleware. But you won’t. Noone has.

The solution is invariably to take the assumptions and patterns and cute tricks away, and replace them with forcing functions in the form of design constraints that keep developer and system behaviour aligned. Even stuff that enterprise developers would tend to know, like caching, needs a different approach. Them’s the apples. REST is a collection of such constraints, formalised into an architecture.

Perhaps the key thing is education, and teaching.

I think we’re done more or less with the evangelism and technical food fights with the WS-* and Enterprise crowds. It’s time for REST to proceed on its own terms and not in contrast and counterpoint to something else. Teaching, on how to target and design with REST, particulary for HTTP as deployed, needs to ramp up. Certainly we need a few good practical books on REST/HTTP to ship.

At least one of those fine books should cover the fundamental principles of Web-centric development, the way Gamma et al did for C++ Patterns, Fowler did for Refactoring, or Norvig/Graham did for CLisp. Something that nails it. As things go, Fielding’s REST thesis is pretty accessible, a decent read even, but it’s a doctoral thesis, not a textbook.

Tickbox Gartnerfication

Hype could be a problem for REST if the “industry” gets on board with it as a commmercial factor that requires ticking off rather than some dorky engineering stuff – “Do they have REST? Check.” So will keeping an eye out for land-grabbing on the term “REST”, so it remains a crisp technical term (It has a theory?! I know! It’s crazy!).

Avoiding the descent into marketing farce, a problem that has plagued “SOA” and “WS” can only be a good thing. To be honest, I’m not holding much hope on this one.

God is in the details

Let’s be realistic. No software architecture can truly withstand implementation. REST is really great, but on the web, there is plenty of detail work to be cleared out, like push, containership, encoding, side-effected GETs, curse-of-popularity, queuing, 2 versus 4 methods, universal format junk, and authentication, among others. I could go on, it’s messy out there.

Issues like these will become more important, or more distracting, depending on your point of view, as the style gets adopted. So run away screaming from anyone who dismisses these as “just implementation details” – implementation details matter. They cost money, and break hearts.

Lot’s of work has been done. There’s lots of work to do.

---

PS.: in case anyone thinks there’s some kind of collective REST triumphalism going on @lesscode, there isn’t. I’ve been sitting on this post over the St Pats weekend, only to find out that Sayre and Tomayko have posted REST stuff in the meantime. You’d swear they weren’t Irish.

Discussion on REST seems to be moving in a constructive direction all over the place with Tim Bray and Don Box both contributing. I think Dare Obasanjo’s notes from a talk given by Google’s Nelson Minar almost a year ago may have put some valuable terminology behind ideas that have been floating around for some time and may have even brought us to a place where we can have an actual conversation about this stuff:

To begin he defined what he called ‘low REST’ (i.e. Plain Old XML over HTTP or POX) and ‘high REST’. Low REST implies using HTTP GETs for all API accesses but remembering that GET requests should not have side effects. High REST involves using the four main HTTP verbs (GET, POST, PUT, and DELETE) to manipulate resource representations, using XML documents as message payloads, putting metadata in HTTP headers and using URLs meaningfully.

The definitions of low REST / high REST could be tweaked a bit (maybe they were when I wasn’t looking?) but I do like the idea of putting terms we can all agree on to the subset that’s the reality behind most of machine/web interfaces that seem to be working today. POX isn’t that term because it unnessarily constrains the media type; “low REST” should include the mostly non-XML web (i.e. the text/html, image/*, text/css, text/javascript, etc. web) where it conforms to the constraints of GET and POST.

The reason I like this approach is because it could remove the artificial and often harmful separation made between the web people interact with and the web machines interact with. Exposing resources for machine consumption isn’t any different than exposing resources for human consumption until you get down to the media type (and not then if you subscribe to Microformats). That is, some form of hyper-media (links / URIs) and a simple, universal interface (HTTP) are the really big wins you get with REST because those things are what creates the possibility of having a hugely distributed and uncoordinated system of interaction. The choice of document format is something else entirely and the decisions there ought to be based on a whole other set of concerns.

If I cared at all about the term anymore, I would go so far as to say that anything conforming to the constraints of at least one HTTP method and that also exposes references to other resources using URIs should be considered “a web service”. But “web service” has been defined to death and it doesn’t seem to be helping.

The first time I heard someone attempt to describe the GET/POST subset as something we should pay attention to was Michael Champion on xml-dev. This was usually thrown out when someone championing REST used the “REST is the web and the web is working” argument (I use that argument to this day, btw). Michael would often be the one to point out that it wasn’t exactly true and that a subset of what REST describes seemed to be working well on the web (GET/POST, URIs, caching), while another subset was clearly—sometimes knowingly—being violated (GET w/ side-effects, uncool URIs), and yet another subset was used only rarely (PUT/DELETE/OPTIONS, content negotiation).

Not being able to distinguish which of these subsets of applied REST one is refering to is hurting our ability to have meaningful conversations about this stuff. For example, consider the following conversation; I’ve seen it hundreds of times now:

Restafarian: 80% of developers use the XXX REST API because they find it easier to deal with.
Tool Vendor: That’s not really REST. It only uses GET and POST and breaks rule #5034
Restafarian: Oh yea, well SOAP/WS-* sucks.
Tool Vendor: Go write some PHP you dirty hippie!

I’d like that to stop happening.

But I don’t think adopting wrong or ambiguous terminology is going to help either. I’d like to register my nits with the few sentences that Dare provided and see if anyone else is thinking along similar lines.

To begin he defined what he called ‘low REST’ (i.e. Plain Old XML over HTTP or POX) and ‘high REST’.

Like I said before, POX is kind of, ummm, bad, I think. It focuses too much on the format. XML is not why the web is working. URLs and HTTP is why the web is working. HTTP and URLs work so well they’re even capable of being useful in spite of complexities in media-types based on XML.

Low REST implies using HTTP GETs for all API accesses but remembering that GET requests should not have side effects.

Fully constrained GETs are really important but POST should fall under low-REST too. By the definition above, low-REST can never modify anything. Throw POST in there, it’s a big part of the present day human/web and is likely to play a bigger role in the machine/web.

High REST involves using the four main HTTP verbs (GET, POST, PUT, and DELETE) to manipulate resource representations, using XML documents as message payloads, putting metadata in HTTP headers and using URLs meaningfully.

I think we ought to downgrade XML to a “bestcommon practice” or something. Microformats and JSON are two notable examples of non-XML representations that seem to be working quite well and are absolutely capable of being used high-RESTfully.

I’m pretty sure I’ll always be a high-RESTafarian but I could definitely see myself coming out as a lo-RESTafarian as well if some of this were grinded out a bit more.

SYS-CON Media publisher/editor Jeremy Geelan, a fine chap who interviewed The Father of Java just the other day, contacted me about re-publishing Gosling Didn’t Get The Memo on sys-con.com. This is the same site that published Gosling’s original remarks.

It took me about five seconds to agree, within an hour we were sharing the front page with the original Gosling quote, and we’ve just now broken into the top three most heavily read articles on the site for the day. The republished post is here and is pretty much an exact copy (comments and all).

So yea, I’m stoked.

I should also mention that there was likely some back-channel prodding that helped lead to this. Adaptive Path’s Jesse James Garrett was panelist at the SYS-CON hosted Real-World AJAX event earlier this week in New York (see: Geelan interviews Jesse). From what I can gather, Jesse played no small part in getting the article picked up.

Huge thanks to both Jesse and Jeremy from lesscode.org.