lesscode.org


'Then they fight you...' Archives

More Developers, Less Code  10

Cat.: Then they fight you...
14. August 2005

Why is it that when any institution has a need for software, they immediately start looking for pre-built “solutions”, instead of consulting a team of developers? Is it because that management feels that it will be less expensive to buy a “turnkey” software product than to pay developers to do the job right? Or is it that the higher-ups are looking for a stronger chain of responsibility if something goes wrong? I would assert that the real reason is somewhere in between the two.

Many of us have worked in houses whose primary business is something other than software. Our jobs ranged from being a one-man (or woman) IT department to simply being that person who picks up the phone when a user calls the helpdesk. In any event, if you’ve worked in this environment, it will quickly become clear to you that non-technical management seems to be its own worst enemy. I’m not saying that all management is bad, just that when it comes to making decisions about technology, a person’s self-importance can be their undoing. Software developers, we are guilty of this too. We put on a smile for the managers, when secretly we think to ourselves how much smarter we are. This attitude only pushes the divide further.

With such a sharp difference between IT and management, it’s no surprise that there is an internal conflict when it comes to making IT decisions. I’m going to describe a scenario that took place at place I had previously worked. This place is an educational institution, with a fairly reasonable IT budget. There are 2 people on staff in the IT department, both of whom are able programmers, with skills concentrated in dynamic web applications, namely PHP. We’ll call this place “the Institute”.

For years, student grading at the Institute was done by pencil and paper. At the end of every term, each teacher would tally up all his students test scores, and issue a grade. Once all a student’s grades were tallied, a small team in the registrar’s office would type up report cards on a typewriter, sign them, and mail them out to the parents. One copy of the report card was kept in a file cabinet, and the only method of access control was a key, having 5 copies, distributed to people who would need access to the information. Times were simple, and everything seemed to flow smoothly.

The Institute had been buying computers in small lots for quite some time. They were distributed sparsely - in the library, in the academic offices, and in the classrooms of teachers who managed to get themselves some kind of priority. Slowly but surely, an IT sprawl was occurring. The Institute saw it fit to hire a small IT staff - 2 people. Both of them young men who really had a passion of technology, and who liked the idea of growing their own infrastructure from the beginning. Eventually, the Institute had computers in most classrooms, a library full of PCs, and even a computer lab that students could drift in and out of in their spare time. The IT staff had been setting up a Microsoft-centric shop, because that’s what the management wanted. The two of them didn’t complain too much about it since the job was still fun, and Microsoft clients and servers seemed to work pretty well together - despite the occasional hiccup. Nothing a reboot couldn’t fix! There was server space and e-mail for everyone. Again, times were good.

There was one problem that nagged the IT pair, though. Grading was still being done with pencil and paper. The higher ups had started to catch on, too. We could make things so much faster if this was all done by computer. By this time, every teacher had his/her own laptop, and some had even taken to doing their grades in Excel - which sped up but didn’t unify the process. With this problem at hand, the IT staff started drawing up plans for a grading system. It would be a simple web application where teachers could enter test scores incrementally throughout the semester, grades would be tallied, and report cards could be printed. It could simplify the whole report card process to a single batch job. They had estimated it would take 6 months to develop and test, with another couple of months for small modifications and training. They could write it in their spare time, considering most of their day was comprised of reading Slashdot between taking support calls. The best part of the whole deal, though, was that it would all be free. The entire framework for the program would run on Open Source software, and since the management already paid the IT staff a salary, there would be no overhead for development.

The management, however, was less than receptive to this idea. When it went down, who was responsible? Who could they call for support? If it took 6 months to develop, wouldn’t we be in action faster if we just bought a “turnkey solution”? IT pleaded with the management to be more reasonable. If it went down, IT would support it. That, after all, was their job. A 6 month development time was a fine wait if it means you’re getting a good piece of software. The managers, though, decided to press on. They decided on a piece of software that had first come as a free sample in the mail. It cost $100 per workstation. IT, suspicious of any software that arrives in the mail unsolicited, reluctantly installed it on all of the faculty laptops. At first, the faculty was excited about this software, but then the questions started to roll in. Why can’t I import all my old grades from Excel into it? Why does it crash when I do X, Y and Z? With the animosity growing between IT and management, the IT people were not as responsive as they should have been when it came to answering user support calls. They were upset that the management had gone behind their backs, and wanted to exact some punishment. Management, on the other hand, was upset that their shiny new $5,000 investment wasn’t the silver bullet that ended all their problems. After a few heated meetings between IT and management, it was finally agreed upon that the Institute would look for a new electronic grading system. The highest authority at the Institute a well-educated fellow named John, had recently come back from a demo of a software product that would supposedly give the Institute electronic grading in one fell swoop. It was a custom-designed program that ran on Filemaker: the pseudo-database. John had loved it, and this was what the Institute was going to use. It was not open for debate. $25,000 later, the Institute had a Filemaker system. The sales force had convinced the Institute to not only buy the software, but a brand new dual-processor server as well. IT gritted their teeth as this new system came online for the first time.

Of course, the Filemaker system wasn’t compatible with the previous program, so teachers had to once again re-enter their grades. Another problem that IT foresaw but was unable to do anything about was the issue of access control. In days past, there had been 5 copies of a key to the file cabinet where all the grades were kept. Management, used to this concept, was very receptive to the idea of a single password for the whole Filemaker database. The IT staff knew it was only a matter of time before this password got out to the students, and sure enough, they were right. It took a month for the students to start changing their grades.

Amidst this crisis, IT pleaded with the management again. John, however, wouldn’t budge. He had made the decision, and it was final. Filemaker was it, and it was IT’s job to fix all of the problems. The teachers had lost their trust in the software, and started doing their grades in Excel again. The administrators, knowing that the students had gotten access to the database in the past, didn’t trust the software either. At the end of every semester, the teachers would e-mail their grades to the registrars, who would combine a student’s grades into a report card, written in Word. They would print 2 copies of this report card. One to mail to parents, and another to put in the 5-key file cabinet. After this process was complete, they would enter the student’s grades into Filemaker, but only for “archival” purposes. And so it remains to this day at the Institute.

The moral of this story is twofold: IT and management need to cooperate on a new level, and for internal software, roll-your-own is almost always superior to shrinkwrapped. Internally developed software is better suited to the problem than anything you can buy, and it humanizes IT more than purchased software does. To know that the software was made for the company, by the company gives users a much better attitude about using it, as long as it is done well. All that needs is for IT and management to trust eachother; something, I think, that will take a long time to happen.

P-Languages on the Wane?  9

Cat.: Then they fight you...
04. August 2005

Hot on the heels of O’Reilly’s report of strong demand for dynamic languages we have The Register reporting that Perl, PHP, and Python are on some kind of massive wane:

PHP and Python have each experienced a 25 per cent drop in the last 12 months while Perl fell 20 per cent. Those not planning to evaluate or use either of the three in future projects, meanwhile, is growing - up by as much as 40 per cent in the case of PHP.

The Reg goes on to predict folly for IBM, Oracle, SAP, and Intel - each of which has thrown support behind PHP.

EDC blamed the drop on a failure for PHP, along with Perl and Python, to penetrate the enterprise space. An EDC spokesman predicted that the decline could be halted and reversed with this year’s work between Zend and its new backers.

Hmmm.. I’m not very satisfied with the response they got from Zend, either:

Disputing EDC’s numbers, though, Zend said that its backing from IBM, Oracle, Intel and SAP proved that PHP is a growing market and that these companies have accurately read developer trends.

This argument is crap - backing from large corporations proves nothing about developer trends.

EDS’ sample seems a bit narrow. They polled 400 developers in small, medium and large enterprises across Europe, the Middle East and Africa (EMEA).

This goes strongly against much of what I’ve been observing over the past year. Where are we to assume these huge percentages of developers going? Java? .NET? I don’t buy it.

Or is this some kind of massive migration to Ruby? ;)

Complexity, Bureaucracy, Fairness  7

Cat.: First they ignore you.., Then they laugh at you..., Then they fight you...
30. July 2005

The starting line for the first Oklahoma Land Rush, April 22, 1889. (Library of Congress)

Tantek Çelik: “The namespace problem? The short answer is that it’s nothing more than academic chicken-littling. The internet has worked just fine (at many levels, with many applications, specifications) without namespaces. Namespaces try to solve a 1% problem while burdening the 99%, which is always bad economics.”

I disagree. Namespaces can sometimes be a technical hassle, but I’ve noticed that people who rail against them are usually squatting on a set of short strings. Would it be OK for Microsoft to define the meaning of the word “friend?”

Dan Connolly: “There is a time and a place for just using short strings, but since short strings are scarce resources shared by the global community, fair and open processes should be used to manage them.”

If RSS didn’t restrict extensions to namespaced elements, we would have a very ugly scene on our hands by this time. At what point does technical convenience trump social concerns? In this case, RSS favors a little technical complexity in return for less bureacracy and more fairness. Maybe morecode is occasionally worth it.

Web Architecture Roundup  5

Cat.: Then they fight you...
23. July 2005

I’ve been on a language and infrastructure kick for the past month or so and posts have reflected that. Unfortunately this means I haven’t had the time to report on some of the excellent discussion around simplifying practices at the protocol and format level. Let’s run through some of this real quick:

Sam Ruby has been churning out good content on the practical benefits of HTTP and REST architecture, challenging SOAP/WS-* practioners to consider the immense benefits resource oriented design offers over the more popular service oriented (SOAP/WS-*) approaches. The Atom Wiki was slashdotted but he was able to push the troll-hoards over to an uneditable site due to HTTP’s redirect capabilities. He uses the occasion to illustrate one of the many basic features HTTP provides to accomodate massively distributed and un-coordinated environments that SOAP/WS-* completely ignores:

The foundation for most of this is rooted in the tantalizing phrase “hypermedia as the engine of application state”. If one views a wiki as a state machine, then being able to do a redirect is effectively changing the programming of the application.

The fact that I was able to do all this with but a single line of code — and using only the protocols and products that were already installed — was of great value to me yesterday. In fact, if one were predisposed to use such terms, I would go so far as to say it was of significant business value to me.

Sam provides more thoughts and conclussions in REST vs. API, which he sums up nicely:

At some point, you need to question the wisdom of having an API which abstracts away that which is important.

In the meantime, WS-* supporters are still trying to figure out how to fit the square peg of narrowly distributed systems architecture into the round hole of the web. Infoworld just published a story that tries really hard to slap some lipstick on the WS-pig:

Although the vendors backing Web services are confident they are properly addressing any kinks, Web services remains hobbled by issues such as standardization and WSDL complexity. To make matter worse, its impacts are growing, with both the .Net and Java development technologies focusing on Web services. Additionally, Web services is doing double duty as the lynchpin of the growing SOA trend.

And just in case there was any doubt that many leading vendors are completely clueless when it comes to providing tools to aid their customer’s operations moving to the web:

Microsoft’s Bixhorn emphasized the company’s planned Indigo technology as the answer to many issues cited with Web services. Due in 2006 in the Windows Longhorn client, Indigo removes the reliance on Web servers by extending beyond HTTP to support protocols such as SMTP, TCP, and REST (Representational State Transfer).

No, no, no! I don’t even know where to begin disecting the problems with this statement.

  1. You don’t want to “remove the reliance on web servers,” you want to provide tools that let customers use it for what it’s worth. Web servers are good! We need to make more and better use of them.

  2. Extending HTTP to “support REST” would be a truly interesting demonstration as REST is largely a description of the architecture HTTP facilities. UPDATE: it’s been pointed out that I may have mis-represented Krill’s intent here. I’m not sure I see a huge difference but in the interest of rational debate and all that…

  3. <outrage>TCP?!!@%##! WHAT THE FUCK!</outrage>

I can only assume this was a horrible misprint or that Microsoft has an automated press statement request robot capable of combining a random set of acronyms with upcoming product names.

The blogosphere, at least, is injecting some sanity into the discussion. Mark Baker wrote a nice piece, Towards truly document oriented Web services, which does a good job of explaining why the web’s basic architecture has been and continues to be a perfect medium for “Web services” since long before the term was even coined.

There were also at least two enlightening articles on Atom’s potential as a SOAP/WS-* competitor. Bill de hÓra’s Atoms in a small world is a response to a computer weekly article making the following claim:

It is clear that there are lots of server-based applications (not blogs) that could work by syndicating packets of XML data to client applications (not RSS aggregators). These will not rival Soap/web services as a system for executing transactions.

Bill responds:

I don’t see why it would not be a good basis for enterprise concerns like messaging, authentication, or management interfaces. Atom, RSS and the Atom Publishing Protocol (APP) as a complement to REST could disrupt SOAP and WS.

I agree but have a slightly different take on how this will be most disruptive. In my opinion, REST and web architecture is just plain misunderstood. There’s a lightbulb moment that’s required to grasp it fully and it just hasn’t went off for a big portion of the business IT community. RSS and Atom are showing how following web architecture can yield huge benefits. These technologies are disruptive because it forces people to ponder whether the web in its basic, unmolested form may not be limited to web pages. If Atom can find a niche here than that’s great but I’m more excited by its potential as an agent for enlightenment.

Kurt Cagle follows Bill’s post with some truly excellent examples of how, in many cases, RSS/Atom can be superior to WS-* techniques in How RSS/Atom Is Replacing Web Services, which he ends with a demonstration of “good web thinking”:

While I suspect that this commentary will be poo-pooed by web services vendors, the history of the web makes me suspect I’ll be right in the end. RSS syndication has emerged in a largely ad-hoc fashion to fulfill a specific need - syndication - that seems to be an intrinsic characteristic and requirement of the Internet. Syndication in turn is simply a messaging protocol built around a subscription-oriented PUSH model. If you make the syndication format too complex for people to use (as I believe is the case with SOAP) then they will migrate to simpler formats as they become available. Because ATOM is blessed by both the IETF and the W3C and is seen as an open standard (whereas questions remain concerning the precise nature and implementation of SOAP) it will likely continue to gain the mindshare of developers and users alike.

To sum up, there’s been good stuff all around from the REST / web architecture guys and the WS-* death march continues on…

Guns, Germs, and Open Source  Comments Off

Cat.: Then they fight you..., F/OSS
16. July 2005

A colleague of mine had suggested on numerous occasions that I read the book, “Guns, Germs, and Steel“, by Jared Diamond. Wherein the author explains that the rise of Western civilization was due largely to a series of lucky developments that turned out to be so disruptive when introduced into other environments that there was just no chance of their old ways surviving.

Tim O’Reilly recently pointed to a bout scheduled for this years OSCON where Byron Sebastian, CEO and Founder of SourceLabs, will go toe to toe with Bob Sutor, VP of Standards at IBM Corporation, on whether F/OSS is to the Enterprise as Guns were to the Incas.

The format is interesting with Sebastian taking the extreme position of “open source takes all” and Sutor the position of “protracted struggle between proprietary and open source enterprise software”. It’s interesting to note that they couldn’t find anyone to champion the “proprietary takes all” scenario.

I’m not able to attend this years OSCON but I’m hoping this debate makes it’s way onto IT Conversations. If anyone attending wants to live blog the debate for lesscode.org, it would be much appreciated.