No SOAP Radio
By Bill de hÓra under Talk on 17. September 2005A significant problem in the trenches for Web Services isn’t the ability to play nice or not with existing Internet protocols, or its complicated type system, or Object/XML impedence. It’s optionality. When you get down to it, too much in Web Services is optional. Calling the specs composable and thus dressing that up as feature is pointless - optionality is a curse. Yes, WS specs are designed to be composable, but this underplays the fragmentation costs of not having an interoperable core.
Optionality here means: a) the state space for communications is huge , b) everything has to be negotiated, c) no clear layering resulting in dependency issues. These introduce complexity and thus drive up costs. As a result I disagree that implementing something like WS-Eventing is “simple”, exactly because there is freeform composition. And that’s just WS-Eventing. What about addressing, transfer, policy, wsdl, faulting, the various reliable messaging approaches? The list goes on and on. Even the basic SOAP stack seems to be difficult to get right; we’re on the third generation inside 7 years of what is supposed to be stable computing infrastructure.
Favouring optionality can’t possibly be a good idea for protocols, or even protocol frameworks like SOAP. In that sense RSS or Atom have more interop potential, precisely because they take options away by design, even though they wouldn’t present an obvious choice for mainstream enterprise purposes.
Ur-Protocol
Here’s Elliotte Harold and Ted Neward arguing over the merits of HTTP as a Highlander Protocol.
“Web Services Addressing is trying to define an addressing scheme that can work over HTTP, SMTP, FTP, and any other protocol you can imagine. However, each of these protocols already have their own addressing systems. Developers working with these protocols don’t want and don’t need a different addressing system that’s marginally more compatible with some protocol they’re not using in exchange for substantially less compatibility with the protocol they are using. Besides nobody’s actually doing web services over anything except HTTP anyway. Doesn’t it just make more sense to use the well understood, already implemented debugged HTTP architecture for this instead of inventing something new? “
Ted:
“No, it doesn’t. There’s no sense in trying to take bits that were designed for a distributed hypermedia system (Fielding’s words, not mine) and trying to bend it to fit a problem space that isn’t distributed hypermedia. Can we learn from the REST architectural style? Absolutely–and the new WS-* specs, including SOAP, do exactly that, favoring self-descriptive messages over RPC calls as the principal abstraction to deal with. But does that mean we tie ourselves solely to HTTP? Oh, hell no.”
The problem with using HTTP as the Highlander protocol has been described best by Marshall Rose:
"HTTP has become the reuse platform of choice, largely because: - it is familiar - it is ubiquitous - it has a simple request/response - it usually works through firewalls These are all good reasons, and - if HTTP meets your communications requirements - you should use it. The problem is that widespread availability of HTTP has become an excuse for not bothering to understand what the requirements really are. It's easier to use HTTP, even if it's not a good fit, that to understand your requirements and design a protocol that does what you really need."
HTTP bias aside, sometimes you do have to work across multiple protocols. That results in the problem of managing and preserving addresses and message identity. And when you do have to span protocols and systems for messaging purposes, especially where there are multiple self-interested administrations, you want to have a standard envelope that provides at minimum a means of identifying the message, the sender, the recipients, and probably time stamping. Some kind of ability to correlate messages. These are baseline criteria - without them you’re toast. You need standard headers not just for routing traffic (important) but because you need to know what the heck is going on operationally - you want a standard means of asking the various endpoints (or nodes, or gateways, or hubs, or whatever you want to call them) questions. Where is my message? How many messages did you drop? Which messages did you drop? When? Whose messages are you dropping? Will you resend this message? How many messages have you sent to so and so? What open items do you have? And so on. However WS, and in particular, SOAP, doesn’t provide a standard (as in non-optional) set of headers and thus it doesn’t provision this kind of generic communication facility. I’ll confess that this has never made sense to me, given SOAP’s protocol framework and inter-systems ambitions.
Ted has a specific rant about broadcast and push being unsuitable to run over HTTP. I don’t think there’s any argument that, formally speaking, HTTP is not the right protocol for those jobs. But here’s the kicker. Blogs and RSS syndication are exactly that, done over HTTP infrastructure. The world has not gone running to push RSS over XMPP because there is obviously a scale problem when you draw things out on a napkin. [Which is not to say this writer wouldn’t like to see more Atom over XMPP; it’s clearly a good idea. Nonetheless the Web sky is not falling in yet. ]
And even in the push scenario, what’s the point of a protocol framework? Why not use XMPP for push/pubsub? Does this mean when people on the ground start getting Eventing systems and EDAs built out over commodity IM, can we expect to see another rounds of the “It’s Not Sufficient” meme played out again, this time for messaging protocols?
What now for SOAP?
If you want to span systems asynchronously in a manageable fashion you’ve got to have standard models for addressing and identity baked in. These are not optional if this stuff is expected to work. It’s taken some time for the WS community to get round to dealing with addressing and naming, but this is key stuff for getting something done over a gateway and for me, SOAP’s value is largely in dealing with gateways asynchronously. Addressing being an optional add-on to SOAP is quite strange then, although now that it exists that’s good news. Down the line then what would be useful is a subset and hardening of SOAP; a 1.3 revision where WS-A and some key message headers are non-optional.
WS should be starting out with, and not ending with, profiles like RAMP.
Bob Wyman:
You wrote: “Why not use XMPP for push/pubsub?”
This is precisely what we do at PubSub.com. We use XMPP with the pub/sub extensions defined in JEP-0060 to instantly notify users of updates to 16+ million RSS/Atom feeds, 50K+ newsgroups, SEC Edgar filing, earthquakes, etc. SOAP based web services and HTTP based systems are simply not adequate to this task since they either require holes in firewalls, polling or gateway services in order to get messages to desktop. To provide instant notifications, one really must have a push protocol… HTTP might be popular, but it isn’t much good for notifications.
bob wyman
comment at 17. September 2005
Bill de hÓra:
Hi Bob,
Yep, I put in a link in to PubSub; and I’m guessing you got here because you’re using PubSub with a subscription to “pubsub” and “xmpp” ;) (incidentally you should check out the URI filter/query mechanisms being proposed for APP at the moment and see if it’ll map onto XMPP…)
While I was thinking about system integration scenarios rather than pushing feeds to users here, the one thing that bugs me about the current push setup is that IM clients… suck. They’re much too intrusive - flashing, popping up; that’s fine maybe for chat, but they could take a cue from mail clients in the case of feed subscriptions.
On the infrastructure level I see a lot of places that block XMPP and you have to argue to get the ports opened (resonating with Marshall’s point about firewalls). It really does seems that some folks find the idea of running a protocol over its correct port rather than 80 being more secure counter-intuitive.
comment at 17. September 2005
Ryan Tomayko:
This is where I think we can clearly see the fundamental difference in two popular world-views about the web and “web services”.
IMO, Ted’s world view makes the assumption that we have the luxury of imposing an entirely new set of techniques that are on an order of magnitude more complex than what is currently being used to accomplish ~80% of what is currently possible. Elliotte’s world view disallows him from making such an assumption. Any design that suggests we are capable of causing adoption of this kind of unprecedented amount of new technology should be heavily scrutinized.
No one is arguing that these new techniques (WS-*) would not provide additional robustness when considered soley from the perspective of “an adopted distributed system”. The question is whether we have the fu required to make adoption of this stuff actually happen on the scale of “all business systems”. You must consider basic human tendencies, skill-set required for implementation, and whether previous systems in similar situations were successful. It is important that a design answer these questions fully. A design that does not will remain a design only.
How does WS-* answer these questions? “We have big companies, standards bodies, and a tech press that will spend lots of money to convince people to adopt and everything will be great.”? It’s not working. It hasn’t been working for some time now. When will we begin a real conversation about these fundamental issues?
comment at 18. September 2005
Ryan Tomayko:
By the way, Bill, where can I pick up that conversation between Ted and Elliotte? It looks like typical xml-dev but I’m not seeing the thread. Thanks.
Nevermind, I can see that its happening on the blogs. Cool.
comment at 18. September 2005
Laurent Szyster:
This is a late comment, I’m quite too busy to read that much.
“If you want to span systems asynchronously in a manageable fashion you’ve got to have standard models for addressing and identity baked in.”
Indeed.
“It’s taken some time for the WS community to get round to dealing with addressing and naming, but this is key stuff for getting something done over a gateway and for me, SOAP’s value is largely in dealing with gateways asynchronously.”
Did they get round addressing and naming?
They did not yet got round the riddle of asynchrony.
In 2000 I reviewed SOAP 1.0 gateway, built a whole test case for some reinsurance network company, complete with SSL/TLS client certificate and integrated as a Python web peer and interfaced with some EDI/XML tool chain and a qmail queue. Simple, fast, cheap. It’s the only software they managed to sell when the network was disolved: it showed the London buyer how to match a team of IBM Java swiss experts with two british VB programmers and some free software.
My judgement from that experiment is that SOAP is broken.
Asynchronously broken.
The envelope is before the body, which is fine for a synchronous process, but deadwrong when handling asynchronously pipelined requests. Because:
the envelope is the state of the transtaction instance.
the body is the state returned by a method of that transaction
the common transaction is usually made of a sequence of calls
that modify the transaction state
Yet there is no way to actually program an asynchronous SOAP server because the f****ing state as modified by the transaction must precede the result of the function that just happen to modify that state!
Too bad.
Really too bad, because back in 2000, an asynchronous Python server running on a 133mhz Pentium III was flooding some Java Application Server running on the kind of hardware Swiss Reinsurer can afford. Too many processes, not enough memory.
SOAP should read like this, translated in an XMLization of Python keywords:
And often, that is what most XML transported by SOAP gateways will contain if there is an asynchronous handler in the process.
comment at 25. September 2005