lesscode.org

My favorite part of Agile Web Development with Rails is the section on testing. I’ve found the framework around testing included with Rails to be a wonderful blend of simplicity and power and this chapter in the book is the perfect compliment. It’s based largely on the core Ruby Test::Unit module but adds some important features on top.

One of those features is Fixtures. Fixtures provide a simple, YAML-based file format for storing database state that should be loaded before test runs. The database is wiped and the fixtures are loaded before each individual test executes, providing consistent state for tests. See the section on fixtures in A Guide to Testing the Rails for more information. Here’s an example fixture from said section:

# low & behold!  I am a YAML comment!
david:
 id: 1 
 name: David Heinemeier Hansson 
 birthday: 1979-10-15 
 profession: Systems development

steve:
 id: 2
 name: Steve Ross Kellock
 birthday: 1974-09-27
 profession: guy with keyboard

The cool thing is that the top level fixture names (in this case, “david” and “steve”) become instance variables in your test case, allowing you to access fixture data in a very intuitively way from tests. The result of this is test code that reads like a story and is often almost humorous.

The book includes a sidebar with a little David head lamenting the importance of “Picking Good Fixture Names”:

Just like the names of variables in general, you want to keep the names of fixtures as self-explanatory as possible. This increases the readability of the tests when you’re asserting that @valid_order_for_fred is indeed Fred’s valid order. It also makes it a lot easier to remember which fixture you’re supposed to test against without having to look up p1 or order4. The more fixtures you get, the more important it is to pick good fixture names. So, starting early keeps you happy later.

But what to do with fixtures that can’t easily get a self-explanatory name like @valid_order_for_fred? Pick natural names that you have an easier time associating to a role. For example, instead of using order1, use christmas_order. Instead of customer1, use fred. Once you get into the habit of natural names, you’ll soon be weaving a nice little story about how fred is paying for his christmas_order with his invalid_credit_card first, then paying his valid_credit_card, and finally choosing to ship it all off to aunt_mary.

Association-based stories are key to remembering large worlds of fixtures with ease.

Taking this advice, I started in on testing a part of an application I’m working on now. The result is worth posting in it’s entirety (hint: it gets interesting towards the middle):

require File.dirname(__FILE__) + ‘/../test_helper’

class EnrolleeTest < Test::Unit::TestCase
  fixtures :coverage_types, :plans, :coverages, :plan_levels, :ppo_options,
           :elections, :enrollees, :enrollments, :tpas, :tpas_users, :users,
           :roles

  def setup
    @joe = Enrollee.find(@joe_the_policy_holder.id)
    @rita = Enrollee.find(@rita_the_spouse.id)
    @billy = Enrollee.find(@billy_the_dependent.id)
    @alice = Enrollee.find(@alice_the_dependent.id)
    @the_family = [@joe, @rita, @billy, @alice]
    @enrollment = Enrollment.find(@test_enrollment.id)
  end

  
  def test_common_attrs
    @the_family.each do |enrollee|
      assert_kind_of Enrollee, enrollee
      assert_equal @enrollment.id, enrollee.enrollment_id
      assert_equal @enrollment, enrollee.enrollment
      assert_equal @joe.id, enrollee.policy_holder_id
      assert_equal @joe, enrollee.policy_holder
    end
  end

  def test_get_gender
    assert_equal :male, @joe.gender, "Joe is a male"
    assert @joe.male?, "Joe is a male"
    assert_equal :female, @rita.gender, "Rita is a female"
    assert @rita.female?, "Rita is a female"
    assert !@billy.female?, "Billy is not a female"
    assert !@alice.male?, "Sally is not a male"
  end

  def test_set_gender
    [:female, ‘F’, 2].each do |tok|
      @joe.gender = tok
      assert_equal :female, @joe.gender, "Joe is now a female"
      assert @joe.save, "Joe could not be saved after sex change"
      @joe.reload
      assert_equal :female, @joe.gender, "Joe is now a female"
    end

    [:male, ‘M’, 1].each do |tok|
      @rita.gender = tok
      assert_equal :male, @rita.gender, "Rita is now a male"
      assert @rita.save, "Rita could not be saved after sex change"
      @rita.reload
      assert_equal :male, @rita.gender, "Rita is now a male"
    end

    # make sure that we can set it to nil
    @joe.gender = nil
    assert_equal nil, @joe.gender
    assert @joe.save(false), "Joe couldn’t be saved after neutering…"
    @joe.reload
    assert_equal nil, @joe.gender

    @joe.gender = ‘’
    assert_equal nil, @joe.gender
    assert @joe.save(false), "Joe couldn’t be saved after neutering…"
    @joe.reload
    assert_equal nil, @joe.gender
  end

  def test_marital_status
    assert_equal :married, @joe.marital_status
    assert @joe.married?, "Joe is married"
    assert !@joe.single?, "Joe is not single"
    assert @billy.single?, "Billy is single"
    assert !@billy.married?, "Billy is not married"
    assert_equal :single, @billy.marital_status
  end

  def test_set_marital_status
    [:married, 2].each do |tok|
      @billy.marital_status = tok
      assert_equal :married, @billy.marital_status
      assert @billy.save, "Billy could not be saved after he got married"
      @billy.reload
      assert_equal :married, @billy.marital_status
    end

    [:single, 1].each do |tok|
      @rita.marital_status = tok
      assert_equal :single, @rita.marital_status
      assert @rita.save, "Rita could not be saved after her divorce"
      @rita.reload
      assert_equal :single, @rita.marital_status
    end
  end
end

:)

Note: the title should make sense by the end of the post…

Kevin Dangoor’s recent announcement of TurboGears has resulted in a dramatic increase of interest in Kid. Kid was first announced on November 30, 2004 as a Pythonic XML-based Templating Language. I remember thinking I was going to do a series of articles on why I wanted this specific combination of features in a library. I never did and Kid progressed into its current form, growing a small community along the way.

Although it’s now used primarily for HTML templating, that wasn’t the initial goal of the project. What I really wanted to do was to illustrate a different way of thinking about “Web Services”.

Not what you were expecting, eh?

But it’s true - Kid was supposed to be a simple device that I would use to start a narrative exploring a variety of topics related to building distributed systems atop the web (i.e. “Web Services”). I had decided that the direction being taken by the industry mainstream was incorrect and that web services would languish until they became more like the existing, working, proven, web.

There was a lot of talk about “Web Services Infrastructure” and framework and that talk continues today. The assumption by nearly everyone was (and still is) that web services would require a whole new set of tooling and paradigm. SOAP/WS was still very much RPC oriented and so the focus was on language bindings, typing, discovery, and the like. Web Services programming had almost no resemblance to existing web programming.

This has only accelerated in the time since I was heavily involved. Now it seems the industry is calling for some mysteriously unspecified SOA toolkit, ESB, or some other insignificant combination of alphanumerics to save Enterprise IT.

I was in the middle of the situation a year ago or so and it troubled me deeply. Long days and nights at work were spent combining technologies like SOAP, WSDL, WS-Security, and BPML with massive Java infrastructure. I ate it all up and knew it like the back of my hand. It is not impossible for someone dedicating 8-12 hours a day to understanding this stuff to love it. I loved it. It’s all very intriguing for someone with my particular constitution.

But so is masturbation.

I’m not sure what happened. I guess I had what alcoholics refer to as “a moment of clarity”. I distinctly remember going over to grab a co-worker for lunch. He was on the phone and so I was talking to some guy in an adjacent cube who did “services work”. The services people were developers but were considered a different department than product development. They dealt with specific customer needs and were tasked with using our platform to craft actual real solutions for actual real business problems. Their world is much different from product development, whose main customers were marketing and the executive team.

Anyway, I noticed a very large stack of printed material on this guys’ desk, entitled “Introduction to Web Services”. Development had collected a set of introductory materials that were to be distributed to all of the services people in the field and, I assume, eventually to external developers using our platform. I had a small part in compiling these materials and had reviewed it at various stages in electronic form.

The hard copy put me on my ass. (I say that metaphorically but if it were to hit me with any significant velocity it very well could have literally put me on my ass.) I picked up the tomb (an “Introduction”, mind), slapped it on my co-worker’s desk, who was now off the phone, and whispered, “this isn’t going to work.”

The services guy doesn’t love this stuff - at all. I’m sure he was completely capable of digesting all of it had he the time and interest of, say, someone in my position, but he doesn’t. He has a customer that wants a bunch of systems to talk to each other and they all have very large and amazingly different framework and infrastructure. More framework and infrastructure is the last thing he needs. The services guy was a wake-up call.

At the time this was happening I had been nursing a long-time interest in true web architecture. This didn’t have anything to do with work - a very long time ago I threw together a little download tool thingy that was capable of resuming failed downloads (with 14.4 baud modems this was a big deal and none of the browsers had native support for resuming). This required that I read bits of RFC 2616 and implement a basic HTTP client. I remember being amazed at some of the capabilities of HTTP because I had assumed that it was mostly a simple file transfer protocol (closer to FTP than, say, CORBA). The spec ended up staying with me and I continued to explore different ways people were using the web and HTTP to do new and exciting things.

At some point, I became convinced that existing, basic web architecture solved many of the problems we were seeing with Web Services adoption. The problem wasn’t that WS was incapable of solving the technology issues (it’s quite adequate), the problem was that it was incapable of solving the social issues. It far exceeded the threshold of acceptable complexity for true adoption by this large community of people whose primary goal is to solve business problems.

The web, on the other hand, was invented to solve the same basic integration problems businesses are experiencing now. Tim Berners-Lee’s dilemma is not so different than our own: a bunch of systems with a bunch of closed data formats and processing capabilities that should be universally accessible. Berners-Lee realized early on that solving this problem would require, above all else, a virus: something that was so simple and lightweight that it would be hard not to adopt. And that’s what the web is: a virus. Just like C and UNIX, Windows, Visual Basic, and a slew of other technologies that kept, as a primary requirement, the ability for real people to solve real problems.

I shifted my thinking drastically and tried to imagine how the integration problems we were seeing would be solved using existing web architecture, which we know to have the traits necessary for mass adoption. I should mention here that by “web architecture”, I mean W3C Tag Web Architecture but also all of the tools and techniques that have evolved above and below it. The servers, proxies, template languages, mod_rewrite, sessions, cookies, load balancing, liberal feed parsers, virtual hosts, MVC, MultiViews/content negotiation, monitoring systems, automated testing tools, dynamic languages, view source, and on and on. All of these little tricks and techniques add up to an extremely powerful and yet fairly simple and understandable toolkit for building distributed systems. What’s more is that there are an unmatched number of people who understand how to build these systems using all variations of platform and language.

Which brings us to Kid as “Web Services Infrastructure”. The concept is simple: for whatever reason, template languages (PHP, ASP, JSP, CFML, ERB, Cheetah, Tapestry, Velocity, etc.) have become a fundamental tool for web development and their usefulness is in no way limited to presentational data (HTML). Templates are simple, templates are cool. You throw some junk in there and look at the result. If the result isn’t right, you tweak your template until it does look right. There’s no layers of magic to get in your way. When something doesn’t come out right, you change the template. There’s no “management” or “container” involved.

There’s no rule that says templates must only be used to generate HTML. Indeed, many of the RSS and Atom feeds in the wild are generated from some form of template. They are never automatically-generated-behind-the-scenes using language bindings and are very rarely generated using some kind of DOM/SAX API.

<rant> RSS is the most successful web services data format in existence (after HTML, of course ;). Successful web services in the future are likely to resemble it. Is there a use-case for RSS or Atom in WS-* land? There are thousands of pages of spec text and no one could throw together a simple use-case for the most successful web service in existence? That’s irresponsible. </rant>

The point I’m trying to make is that template based web services are a reality and that we should be thinking about making incremental improvements to the general model to facilitate more machine-centric data formats instead of creating whole new paradigms.

There are a variety of really important issues with using templates for general purpose web services programming, most having to do with the first part of Postel’s Law:

“Be conservative in what you do; be liberal in what you accept from others.”

The problem with using templates to produce XML (including XHTML) is that it is exceedingly hard to be conservative in what you do. Most template engines are text based, making it easy to miss well-formedness errors. There are also a range of character encoding issues that template languages could ease but often simply ignore and sometimes make worse.

Kid is a simple attempt at building features that aid in conservativeness into the template engine. I actually considered tag-lining it The Ultra-Conservative Template Engine as a play on Mark Pilgrim’s Ultra-Liberal Feed Parser whose name comes from the second part of Postel’s law. This is, of course, the whole point of being conservative in the first place: so that we don’t need an Ultra-Liberal parser for each variation of “web service”.

I think some of these features are compelling and would like to see them pursued in other tools that are in common use for web development. For instance, one of the most important and least talked about features of XML is that it provides a reasonable system for encoding the entire range of unicode code points in any character set, including ASCII. A template engine with a basic understanding of XML could process templates authored in utf-8, interpolate data encoded in ISO-8859-1, and output in 7-bit ASCII — if that’s what Postel demanded. Kid does that.

One of the most important features of XSLT is that well-formed templates are guaranteed to yield well-formed output (with some well understood exceptions). If the template runs, you know it will provide a basic level of conservativeness. Kid does that. We’ve wontfix‘d feature requests because they would break this contract.

Most template languages require you to explicitly encode data that may contain reserved characters. Kid takes the opposite approach and assumes that all content is textual and should be encoded unless you explicitly state that something is XML (in which case it must be well-formed).

There’s also some interesting features around serializing the resulting XML infoset with different variations. For example, you can author templates in XHTML 1.0 and output in HTML 4.01. The output serializer takes care of all the little quirks for things like empty elements, non-escaping of SCRIPT and STYLE content, boolean attributes, etc. The result is a clean authoring environment and an ultra conservative output format. I bring this up in the context of web services infrastructure only to show that the ability to filter template output can be very useful when dealing with different types of user agents.

All this to say that if you’re looking for “Web Services Infrastructure” for exposing processes and information, you’re probably looking to hard. If you have a database, templating, and a web server, you have most of the infrastructure and framework required to begin exposing information from each of your systems in a proven and established way. What you want to be on the lookout for are small and specific enhancements to these existing pieces that allow you to interact with other machines in a more predictable manner or in new and different ways.

The next time someone is selling you infrastructure for Web Services, or SOA, or ESB, or whatever they’ll call it next, make sure you ask “Why?” After they tell you, make sure you understand, agree, and have the problems they propose to solve. If not, ask again. New framework and infrastructure is extremely expensive in more ways than one: you have to ramp people on it and then deploy, manage, monitor, and support it. Sometimes new infrastructure is unavoidable but when it overlaps a large portion of your existing infrastructure, you should make sure it’s bringing back a significant return.

This is what extremely powerful looks like. This is what extremely powerful is like.

Gates had this to say in a recent interview with Jon Udell.

“There’s only really one metric to me for future software development, which is — do you write less code to get the same thing done?”

I’d be lying if I said that didn’t put an immediate grin on my face.

I personally don’t get that feeling from Microsoft’s present toolset but I’m interested in understanding this situation better. I’ve been impressed by their ability to resist being sucked into much of the Enterprise hype that seemed to drive J2EE to bloatedness, but “less code” as the metric for innovation in development tools and methodology? Is there really a strong concious regard for simplifying things in Microsoft’s developer culture?

I’ve had this in draft for a few days because I wanted to talk a bit about the false assumption that GUI = simplicity (not saying that’s what is going on here but it is a falacy MS has subscribed to in the past). But now Michael Champion, who is one of the very few reasons I have left to respect MS’ technology, is beating up on us for a comment.

So bravo to Gates and all that but is it true and does Gates’ personal view on simplicity trickle down into the trenches at MS?

Via Tim.

I’ll take it as a given that if you’re reading this then you agree that, for the sake of sanity and productivity, it’s time coders gave up on roll-your-own, and moved over to modern frameworks where one can concentrate on business logic rather than request parsing (and get all those AJAX goodies for free ;-) ).

I’ve been looking on with interest for the last year and a bit, and as I’ve watched the pioneers blaze their XP, RoR, lesscode trail across the web-firmament, I’ve begun to suspect that I must have missed something. Yes, it’s powerful stuff, and yes it isn’t all smoke and mirrors - there really is “gold in them thar hills…” - but, and for me it’s a big but, we seem to be missing the big picture. Where are the map makers? Where’s the documentation for the second (or third) wave?

Self-documenting code is all very well, and having a common vocabulary of design patterns helps when discussing solutions to individual problems. But what second-wavers really need (and I include myself here - no, actually, put me down as a third-waver) are more pictures. More exposition. Road maps.

Is there a way to add XD (eXtreme Documentation?) back into the XP mix? Writing elegant code is hard, and people who do it earn the admiration they received. But I would argue that writing good documentation is harder, and that it shouldn’t be left for the second-wavers to do.

People who’ve moved to XP have already gone thorugh the pain barrier of

• write the tests
• then write the code
• (then refactor)

and have proved that in the long run, it means better code, less debugging, in less time. But having proved that that works, might there be some benefit in switching to;

• write the spec
• then write the tests
• then write the code
• (then refactor)
• then write an overview!!!

Might this result in (my h-nought) more easily modfied code, quicker adoption by other coders, greater community support?

I’m genuinely interested in other people’s views on moving documentation down(?) the food-chain, so that it’s non-optional, and as integral to writing new code (and frameworks) as writing good tests. Yes, there are good auto-doc tools and methodologies out there, but that right now they still seem to be seen as secondary to the process by Joe Frontiersman, and they only deal with what’s in the file, not what’s in the coder’s/architect’s head. (There’s the nine diagrams of UML, yes, but who on the bleeding-wrist of open-source technology is actively using/sharing designs via UML?)

Let me know if I’ve missed a trick somewhere.

[A few thoughts for the pot: I believe the that the reason the Open Source model works because it’s based on non-coercive collaboration. But Source Forge is littered with unfinished, half-baked projects because someone didn’t think to check that there wasn’t already a project out there that they could use. (How many PHPUnits does the community really need?) Should there be a ‘requirement’ for documentation before a project gets listed? Perhaps it’s time for ‘ideaforge’, or ‘architectureforge”?]