More Developers, Less Code
By teddziuba under Then they fight you... on 14. August 2005Why is it that when any institution has a need for software, they immediately start looking for pre-built “solutions”, instead of consulting a team of developers? Is it because that management feels that it will be less expensive to buy a “turnkey” software product than to pay developers to do the job right? Or is it that the higher-ups are looking for a stronger chain of responsibility if something goes wrong? I would assert that the real reason is somewhere in between the two.
Many of us have worked in houses whose primary business is something other than software. Our jobs ranged from being a one-man (or woman) IT department to simply being that person who picks up the phone when a user calls the helpdesk. In any event, if you’ve worked in this environment, it will quickly become clear to you that non-technical management seems to be its own worst enemy. I’m not saying that all management is bad, just that when it comes to making decisions about technology, a person’s self-importance can be their undoing. Software developers, we are guilty of this too. We put on a smile for the managers, when secretly we think to ourselves how much smarter we are. This attitude only pushes the divide further.
With such a sharp difference between IT and management, it’s no surprise that there is an internal conflict when it comes to making IT decisions. I’m going to describe a scenario that took place at place I had previously worked. This place is an educational institution, with a fairly reasonable IT budget. There are 2 people on staff in the IT department, both of whom are able programmers, with skills concentrated in dynamic web applications, namely PHP. We’ll call this place “the Institute”.
For years, student grading at the Institute was done by pencil and paper. At the end of every term, each teacher would tally up all his students test scores, and issue a grade. Once all a student’s grades were tallied, a small team in the registrar’s office would type up report cards on a typewriter, sign them, and mail them out to the parents. One copy of the report card was kept in a file cabinet, and the only method of access control was a key, having 5 copies, distributed to people who would need access to the information. Times were simple, and everything seemed to flow smoothly.
The Institute had been buying computers in small lots for quite some time. They were distributed sparsely - in the library, in the academic offices, and in the classrooms of teachers who managed to get themselves some kind of priority. Slowly but surely, an IT sprawl was occurring. The Institute saw it fit to hire a small IT staff - 2 people. Both of them young men who really had a passion of technology, and who liked the idea of growing their own infrastructure from the beginning. Eventually, the Institute had computers in most classrooms, a library full of PCs, and even a computer lab that students could drift in and out of in their spare time. The IT staff had been setting up a Microsoft-centric shop, because that’s what the management wanted. The two of them didn’t complain too much about it since the job was still fun, and Microsoft clients and servers seemed to work pretty well together - despite the occasional hiccup. Nothing a reboot couldn’t fix! There was server space and e-mail for everyone. Again, times were good.
There was one problem that nagged the IT pair, though. Grading was still being done with pencil and paper. The higher ups had started to catch on, too. We could make things so much faster if this was all done by computer. By this time, every teacher had his/her own laptop, and some had even taken to doing their grades in Excel - which sped up but didn’t unify the process. With this problem at hand, the IT staff started drawing up plans for a grading system. It would be a simple web application where teachers could enter test scores incrementally throughout the semester, grades would be tallied, and report cards could be printed. It could simplify the whole report card process to a single batch job. They had estimated it would take 6 months to develop and test, with another couple of months for small modifications and training. They could write it in their spare time, considering most of their day was comprised of reading Slashdot between taking support calls. The best part of the whole deal, though, was that it would all be free. The entire framework for the program would run on Open Source software, and since the management already paid the IT staff a salary, there would be no overhead for development.
The management, however, was less than receptive to this idea. When it went down, who was responsible? Who could they call for support? If it took 6 months to develop, wouldn’t we be in action faster if we just bought a “turnkey solution”? IT pleaded with the management to be more reasonable. If it went down, IT would support it. That, after all, was their job. A 6 month development time was a fine wait if it means you’re getting a good piece of software. The managers, though, decided to press on. They decided on a piece of software that had first come as a free sample in the mail. It cost $100 per workstation. IT, suspicious of any software that arrives in the mail unsolicited, reluctantly installed it on all of the faculty laptops. At first, the faculty was excited about this software, but then the questions started to roll in. Why can’t I import all my old grades from Excel into it? Why does it crash when I do X, Y and Z? With the animosity growing between IT and management, the IT people were not as responsive as they should have been when it came to answering user support calls. They were upset that the management had gone behind their backs, and wanted to exact some punishment. Management, on the other hand, was upset that their shiny new $5,000 investment wasn’t the silver bullet that ended all their problems. After a few heated meetings between IT and management, it was finally agreed upon that the Institute would look for a new electronic grading system. The highest authority at the Institute a well-educated fellow named John, had recently come back from a demo of a software product that would supposedly give the Institute electronic grading in one fell swoop. It was a custom-designed program that ran on Filemaker: the pseudo-database. John had loved it, and this was what the Institute was going to use. It was not open for debate. $25,000 later, the Institute had a Filemaker system. The sales force had convinced the Institute to not only buy the software, but a brand new dual-processor server as well. IT gritted their teeth as this new system came online for the first time.
Of course, the Filemaker system wasn’t compatible with the previous program, so teachers had to once again re-enter their grades. Another problem that IT foresaw but was unable to do anything about was the issue of access control. In days past, there had been 5 copies of a key to the file cabinet where all the grades were kept. Management, used to this concept, was very receptive to the idea of a single password for the whole Filemaker database. The IT staff knew it was only a matter of time before this password got out to the students, and sure enough, they were right. It took a month for the students to start changing their grades.
Amidst this crisis, IT pleaded with the management again. John, however, wouldn’t budge. He had made the decision, and it was final. Filemaker was it, and it was IT’s job to fix all of the problems. The teachers had lost their trust in the software, and started doing their grades in Excel again. The administrators, knowing that the students had gotten access to the database in the past, didn’t trust the software either. At the end of every semester, the teachers would e-mail their grades to the registrars, who would combine a student’s grades into a report card, written in Word. They would print 2 copies of this report card. One to mail to parents, and another to put in the 5-key file cabinet. After this process was complete, they would enter the student’s grades into Filemaker, but only for “archival” purposes. And so it remains to this day at the Institute.
The moral of this story is twofold: IT and management need to cooperate on a new level, and for internal software, roll-your-own is almost always superior to shrinkwrapped. Internally developed software is better suited to the problem than anything you can buy, and it humanizes IT more than purchased software does. To know that the software was made for the company, by the company gives users a much better attitude about using it, as long as it is done well. All that needs is for IT and management to trust eachother; something, I think, that will take a long time to happen.
Seth Vidal:
I’ve seen this happen many times. There is however the double-edged sword. Internally developed software has the benefit and detraction of being developed to meet internal needs. Sometimes the developers are working in a vacuum away from the rest of reasonable, open source, development communities. That’s how we get things like an all-pascal or all-haskell grading system written with some ‘interesting’ decisions about how to do IPC and RPC.
Essentially, internal devel is extremely dangerous if your internal developers aren’t good enough. Or are unwilling to play outside of the organization.
comment at 16. August 2005
Ryan Tomayko:
Or are not permitted to play outside the organization as is probably the more common case until recently.
comment at 16. August 2005
grahamc:
Yes, a familiar story. Usually though, the system gets declared a huge success and “John” gets promoted, the 2 IT people quit in disgust, a new IT department gets hired who promptly throw up their hands at the appalling standards of the last IT department, thereby ensuring that any messups in the first 6 months can be blamed on their predecessors. One of the 2 new people quits after 6 months. The remaining one soldiers on putting out fires on a daily basis, and comes to be regarded as a hero becuase of their dedication and service orientation.
A new person is hired who quietly writes a simple grading system in PHP and MySQL without asking permission, and quietly gets all the teachers to do “beta testing”. This becomes the new system. After 2 years, all use of the previous commercial systems stop and no mention of them are ever made again, as if they never existed.
Then management hires a new senior teacher, “Fred”, who throws up his hands at the appalling IT standards, and pushes through approval for a new $100,000 grading system which has integrated payroll, accounting, inventory and teaching modules. The students crack the security in 1 week flat. It is declared a huge success. “Fred” gets promoted…
I have seen the same thing repeated many times, although usually with many millions of dollars being wasted. It’s so easy if the right salesperson takes the right executive out to lunch at the right time!
comment at 16. August 2005
martijn:
If they were really able developers they whould not have proposed a 6 month project.
They wourk have proposed a phased project:
2 weeks:
-authentication + adding grades to the database.
4 weeks:
??
6 weeks:
??
comment at 17. August 2005
james governor:
the language of “solutions” is off-putting and generally unhelpful. a solution is what i used to mix in chemistry…
http://www.redmonk.com/jgovernor/archives/000127.html
comment at 17. August 2005
Digitalia » Blog Archive » Lesscode.org:
[…] This looks like it might be worth keeping up with: lesscode.org, a multi person blog about, basically, producing applications that actaully get something done, rather than getting lost in ciomplexity. I particularly like this article, outlining the benefits of in-house code and the management process by which is doesn’t happen. I, of course, have never worked anywhere like that. No. […]
pingback at 17. August 2005
Kevin Dangoor:
One other thing to remember is that a very large percentage of IT software projects “fail” (for various definitions of failure). There is a whole host of reasons why this is so (many of them non-technical), but there’s no guarantee of success for the custom-built solution.
comment at 18. August 2005
DLeBlanc:
There is a lot to be said of the ability of a small, focussed, and talented developer (or part time developer) team to create something like this. Usually though, this just ends up in a special purpose, poorly tested implementation.
Now imagine that this project evolves into a slightly more general purpose, yet still light weight and focussed system. It becomes open source, and gains just enough (actually useful) features to be popular. It also has enough clients that it gets pounded. Some of the students even look at the code and break it in certain ways.
I think I’d rather look at using the second option - a lot is to be said of software maturity (even if it does come with an initial learning/installation curve). Assuming the software in question is well tested, and solves the important problems, I think this cost is far lower than employing a team to create a custom one-off and hope they have the craftsmanship to create a well tested and usable application. Of course if this product doesn’t exist, it could be because this group needs to write it :)
I think this rule applies as the software in question becomes a larger burden to reimplement. I don’t think anyone would try to re-write MS Word because it doesn’t do something a particular way. For trivial applications, a custom one-off might be just the trick.
I think there should be a cut-off point at which the costs of reimplementation exceed those of adapting an existing solution. Of course, for the example above I think the one off was far superior to the other solutions.
comment at 18. August 2005
Bill Brown:
This entry really resonated with me. At my last job, we regularly were forced to work with so-called turnkey products that were bought as a result of a slick presentation. We spent $1.2 million dollars to buy and implement Vignette for our external web site because they didn’t want to take the time for us to develop it in-house. Ostensibly, we would just implement the bastard (with help from a consulting firm that had done it so many times that it would be a cinch) and move on to things that better used our talents. The six months turned into 13 months and the monstrosity barely was able to cope with traffic. None of the supposed extensibility was easy and we gradually adopted a hands-off approach to it, touching it only when absolutely necessary. Everything was like this: lip service to the highly-qualified developers while complete distrust evidenced by seeking “better” solutions.
Contrast that with Go Daddy, my present employer. We build practically everything we use internally. There’s a couple of exceptions, but by and large management has confidence that we can do anything they want. I have never been happier and the difference was palpable within weeks of joining. It’s amazing when your knowledge and skill is appreciated.
comment at 20. August 2005
netsNbytes » Blog Archive » More Developers, Less Code [@lesscode.org]:
[…] More Developers, Less Code [@lesscode.org] […]
pingback at 20. October 2005