lesscode.org

This is a two-parter in two parts.

A few months ago I became partial owner of a small technology company. A long time friend and colleague had started in on a set of web applications for managing (US) health benefits (wait - don’t go anywhere yet, this will get on topic in a second) and succeeded in convincing me that this was an area of business IT that could benefit excessively from being part of the web. About 60% of this particular application’s functionality was there when I signed on and was built using Microsoft’s .NET, SQL Server, and IIS. My partner had little experience with the technologies I tend to recommend on this site and so the decision was made to move forward with the safety of .NET while researching potential alternatives for future work.

My days have been spent refining and growing the .NET app while my nights were spent researching and assembling a platform that would allow us to work in a much more agile environment. I have a fair bit of experience working with LAMP (where “P” is for Python and “M” is for PostgreSQL). That experience served as a starting point but what I’m after is much more than a simple running environment. I’d like to pull together the various tools necessary to develop, test, manage, maintain, and deploy applications built on the LAMP foundation. That means everything from Subversion to Mailman, Trac to Wordpress, and on down into the guts of a hacking environment like build tools (make, etc.), automated documentation utilities, continuous integration and testing, etc.

That’s the briefest possible history to bring us up to present time where I’d like to talk about two important observations I’ve made over the past few months of being in this situation. I’ll be talking primarily about Microsoft but everything here is applicable to the Java platform and, in some cases, is more so because I’ve made a few assumptions about what is available for .NET based on Java experience (for instance, I’ve barely used Nant or NUnit but assume they’re roughly on par with Ant and JUnit).

The first observation (and the one I will be exploring for the rest of this post) comes from being in extremely close, and somewhat handicapped, competition with the Microsoft platform. Please remember that I am partial owner of this company, have a working application written in .NET, and feel extreme amounts of pressure to create revenue quickly while keeping cost (in both time and money) at a minimum. This situation has forced me to use extreme prejudice when evaluating tools and technologies. In some ways I’ve inherited the responsibility of championing Microsoft’s platform (if only to myself) as my situation requires that I provide real evidence that shows not only that gains can be had by using something different but that these gains are significant enough to justify the cost of that change. In general, the situation just plain sucks - I have to work much harder than if we were to secure a couple million in R&D budget for the year or were juiced up on venture cap. I’m not saying that because I wouldn’t give 100% of my capacity in those situations but it’s hard to imagine manufacturing the level of discipline created when your house is sunk into something.

Back to Microsoft’s platform: they have all the major pieces in one place, are fairly organized, and are extremely accessible. However, time and again I’m finding that I can beat them in almost any given area with lightweight F/OSS technology but I can’t beat the platform as a whole. It’s just all there. The sense of safety this creates cannot be ignored (I’ve tried).

I want to be absolutely clear about that last point because I’d hate to be misrepresented: in a great number of cases, I’ve found individual F/OSS technologies to be superior to their Microsoft counterparts but the lack of an overall platform whole is distressing. I don’t care about indemnity, paid support, certification, finding people to hire, TCO (blech!), etc. because I already understand the F/OSS answer to those questions and they are compelling. I do care about being able to answer the phone, talk to a customer, and feel confident in what and when I can deliver. Microsoft either gives me that ability or gives me the illusion of having that ability - it’s assuring and troubling at the same time.

All this is to say that Kevin Barnes’ Freedom Languages really hit a chord with me. I have very little doubt in the area of language: Python or Ruby just completely destroy any of the mainstream .NET languages and Java to boot. What makes me feel safe is the consistency of the rest of the tool-chain, the massive amount of component libraries, and the breadth of the core platform.

I keep going back to Kevin’s idea of measuring things on the axes of Freedom vs. Safety, where “freedom” isn’t so much GNU-freedom-type-freedom but rather a sort of aggregate measure of the qualities we find most compelling in software design and architecture. Weighing things on this scale helps explain some of the intuition I’ve been feeling over the past months. If we measure the three major platforms competing for business mind-share, I think you find first that Microsoft is extremely safe while offering the least freedom. Next we have Java, which is pretty safe too but the established F/OSS communities (e.g. Apache, Codehaus, etc.) and even the vendor’s (Sun, IBM, BEA, etc.) positions provide a bit more freedom. We then arrive at LAMP and friends which tend to sacrifice safety but provide unmatched freedom and thus, when measured along the web’s core axioms, quality.

Given those basic metrics, choosing a platform comes down to making two key decisions: where do you fall on the safety/freedom scale and in what manner would you like to proceed? Your remaining time (e.g. forever) will be spent doing one of two things: making the platform more safe or making the platform more free. You have infinitely greater control over one of these situations.

We played the fence but I believe we’re arriving at a turning point. I’ll talk about that in part two because the areas explored there are different enough to warrant a separate URL - mainly my experiences in making the LAMP platform more safe and complete but with twists and turns that will eat your brane.

Take Me To Part Two - Assembling The Platform

I am indebted to Stefan Arentz (RoR Tunnel Vision) for helping me articulate better the issue with the impasse software development community is faced with today. His keen observations, while aimed toward explaining how no trace of crisis in the world of software development (and in particular in the world of Java-based development) is detectable on the radar, actually helped me tremendously in corroborating my initial hypotheses.

Stephan begins his rant by admitting how he couldn’t get a clear indication from my original post on what kind of complexity am I referring to when it comes to developing applications using J2EE platform. And here I admit my mistake, caused by my desire to be brief and to the point. I did use the J2EE moniker, but what I actually meant was ‘J2EE-flavored frameworks’. By that I mean any and all of the official and open source frameworks available on the market today. Frameworks such as JCA, JMS, JSF, Velocity, Expresso, Tapestry, Jetspeed, Hibernate, Spring, the list goes on and on.

Any of the above listed and implied frameworks would do; I’ve given a chance to almost all of them, only to turn away in disgust. There’s gotta be a better way! Come on, we’re battle-scared professionals, admit it, there simply has to be a better way.

Stefan tried to reassure me that Java didn’t sit still during its 10 years tenure here on earth (“Alex, you just have to be open minded and stop thinking that Java has not progressed in the last 10”). Yes, I am painfully aware of the fact that Java has progressed (there are days when, in desperation, I think how I wish it did indeed stand still, instead of turning into this Frankenstein that we have to deal with today). As an early adopter and a veritable Java evangelist, I’ve watched the language of my choice transmogrify from seductive little agile, compact product, into the monstrosity that it is today. Needless to say, I’m not amused.

I did buy into Java 10 years ago, hook line and sinker, because I really believed in the power of developing commercial grade business applications without having to depend on moronic, bloated IDEs and the surrounding environmental support (I’ve managed to successfully desert Microsoft’s ‘DLL Hell’, with all its implied complexities and monstrosities, back in 1996). Nine years ago (heck, even seven or eight years ago), only me and my notepad, armed with a small footprint JVM and JSDK on my laptop could walk into a giant corporation and get things rolling in a matter of hours. I could easily build agile applications in Java, quickly cobble up fully functional prototypes, deliver educational sessions on the fly, convert the crowd and talk them into abandoning C++, COBOL, what have you, and move to the new mistress, Java, all from my lowly little laptop. Not having to rely on any tool, other than a simple text editor, I could do high grade, high output work and get paid handsomely.

Then, one day, ouch! Along came IBM and started pushing this completely idiotic product called Visual Age on unsuspecting Java developers. Oh, how I hated the guts of that product! My company forced me to complete weeks and weeks of advanced Visual Age IBM courses, and that was one of the most painful experiences in my software development career. Right there, I knew they are pushing this ‘giant solution in search of a problem’ on people who really, really don’t need it.

Today, if I want to walk into a corporation and offer my Java skills, I need a fancy, state-of-the-art EDI, such as Eclipse (another incarnation of Visual Age, with a lipstick on it) with countless plug-ins and all the bullshit that I don’t really care for. That situation is actually laughable, if you ask me.

Stefan continues with a reasonable observation: “Interestingly, complexity is not something tied to a specific platform. It happens everywhere with all kinds of code.”

True, very true. Everybody knows that complexity is a fact of life, and thus it is unavoidable.

What I was referring to, however, was unnecessary complexity. And my grievance with Java is that it is, as a language and as a platform, unnecessarily complex today.

Here, however, is where we get to the essence of the problem – Stefan writes: “Yes there are a lot of choices, but we are talking about people here who are smart enough to pick Ruby on Rails, so why can’t those people also do a bit of research in the Java world and pick a good framework there.”

You see, Stefan, this is exactly the name of my pain. ‘Do a bit of research’. I’ve spent exorbitant amounts of time in the past six-seven years doing endless research around Java community, relentlessly chasing the elusive dream of the possibility of developing apps in a simple, straightforward way. I’ve tried all the latest and the greatest Java hype, followed religiously every Javaworld and The Server Side and Artima article, but only to in the end throw my hands up in the air in utter desperation. After all was said and done, at the end of the day, all I have is an enormously bloated, to the point of bursting at the seams, Eclipse platform. I am currently nursing countless frameworks, plug-ins, SOA and EMF bullshit is coming out of my ears. My 60 GB laptop with 1 GB main memory cannot take it any more.

All these wonderful frameworks and plug-ins are supposed to turn me into this Super Developer who can crank up incredibly powerful and complex solutions in a matter of minutes. But guess what – instead of being this mythical uber-developer, I can barely get off the ground under the weight of enormous amount of idiotic infrastructure code.

I feel extremely uncomfortable with this situation. I don’t think we, as a community of Java developers, are standing on the solid, firm ground. I think the ground on which we stand is bound to give in under our feet, sooner or later. This is why I’m trying to defect to the Ruby camp, where life is simpler, the air is fresher, and the movement is truly, not nominally, agile.

So, my bottom line answer to your question above is: if Java were truly capable of offering agility, simplicity and power, then the search for a suitable framework wouldn’t have to turn into a multi-month, open-ended wild geese chase. The agility, simplicity and conciseness would be right there, at our fingertips.

Stefan continues with: “Alex, that you cannot be Agile with Java does not mean that it is not possible at all. Many people are extremely agile with Java development. People even wrote books about it. You obviously don’t know what is going on it that world. Is that why you make a straw man argument?”

I am not disputing that it is possible to be agile with Java. Anything’s possible, given sufficient resources. I’m only complaining that, in order to get to be agile with Java, one is forced to invest inordinate, almost astronomical amounts of effort. And that, not surprisingly, comes with the astronomical price tag.

And that is precisely why I say that business computing is in cahoots today, as every move implies astronomical costs. The businesses simply cannot afford to operate in that fashion, regardless of how much fun the developers may be having in their cubicles.

Towards the end, Stefan throws all caution in the air, not being able to resist bragging a little: “Heh. Dude. Who cares that my EJB3 application needs 12MB of jar files. I can now convert a complete persistence model to working code by just writing plain old java objects and adding an annotation here and there to specify the relationship parameters.”

Stefan is obviously proud of the latest and greatest Java crutch – annotations. I’m not going to rain on anyone’s parade here, so I’ll just leave it at that. But it is a vivid example of what usually tends to go wrong with the Java tunnel vision.

I’ve enjoyed Stefan’s parting words: “I invite you to take a second look at Java. Talk to people. See how they are getting more productive. Look around what is going on. You might discover some Really Good Stuff.”

Thanks for the invitation, Stefan. Java is my bread-and-butter, that’s how I make my living these days, so willy-nilly, I am forced to take a second look at it and to talk to people. They are indeed attempting to get more productive by being forced to constantly toss out some old paradigms that were new six months ago and adopt some shiny new paradigm pushed on them by some group of raving developers (like, the aforementioned annotations), that will become old and rusty six months down the road. Last year it was aspect-oriented bullshit. This year, it is annotations. Next year – anyone’s guess.

Let me tell you – people are getting mighty tired of that particular brand of a rat race.

I finally broke down a few days ago and decided to make Firefox my default browser. I’ve tried to move away from Safari a few times in the past but never made it longer than a day. Safari is a better OS X application in some very fundamental ways but I’m encouraged by the Firefox/Mac activity slated for Firefox 1.5 and beyond.

My reason for switching and my reason for holding off for so long are the same: my browser contains significant customization and personalization. I’ve built up a nice little toolset on top of Safari that includes bookmarklets, extensions, and URL searches, not to mention all of my cookies, passwords, and history. Once you’ve established a normal workflow around these customizations, it becomes hard to go without them. It’s not unlike trying to switch from Emacs (with a ~5,000 line .emacs file crufted over years of use) to vi.

But, as I said, the ability to customize the browser has become so important to me that I’m willing to take the hit to productivity and Macishness now to get ramped up on a platform with far greater support for customization. In other words, Greasemonkey. I’ve only just begun diving in but my experience thus far is promising. The ability to automate tasks and extend basic web content with simple scripts seems so insanely obvious that I’m having a hard time understanding how this wasn’t a built-in feature of the 3.0 era browsers. Greasemonkey may be the first signs of a long awaited convergence between Emacs and The Browser :) Now if I could just get an M-x prompt, we’d be in business (YubNub anyone?)

As with Emacs, I will now begin building an embarassingly unorganized collection of hacks. I spent an hour or so combing through the massive repository of user scripts just to get my feet wet. Notes and pointers to stuff I found useful as follows…

Auto Login Everywhere

The AllowPasswordRemembering and AutoLoginJ scripts can be used to get almost reliable automatic login anywhere that Firefox puts up the “Remember this password” dialog. When you hit a site that Firefox has login information for, the AutoLoginJ script goes ahead and auto-submits the form. I’ve tested it across a couple of different sites and it seems to work about half the time. My banking site, O’Reilly Connection, and Vonage seem to work well but none of Yahoo, del.icio.us, or flickr.com worked.

Video URLs

I hate multimedia on the web – especially video. There’s a special place in hell for whoever is responsible for the tiny embedded players with format/bandwidth selection and shite controls that are so common. I promise the experience would be dramatically increased if all video was reduced to a thumbnailed image linking to an actual real video file.

The Unembed user script gets me half way there by inserting a link to the video file under the video display. My first hack will be to have it replace the <embed> with an <a>.

If you’re looking for somewhere to test this script, I highly recommend This Spartan Life.

A Whole New Google

There is an immense set of useful Google specific scripts. I’m using Google Access Keys and Google Search Keys for quick keyboard navigation of search results. The Google Image Rewriter makes image search results link directly to the image instead of that weird framed thing.

Holy Crap, I can use Slashdot

The Slashdot Single Page View script provides thread expansion without requiring a page refresh, xSidebars removes the butt-cheeks, and Add Mirrors puts nice little Coral, MirrorDot, and Google Cache links next to every single link on the page. There’s more Slashdot specific scripts I haven’t tried yet.

Misc.

Other scripts I’ve found useful:

• BetterDir beautifies Apache directory listings. It’s really not that big of a deal but it shows the types of interaction a user script can have with a page.

• CookieMonster puts a little thing in the bottom left of the page that shows raw cookie data for the current page upon hover.

• Nofollow display strikes links having a rel="nofollow" attribute. Mildly interesting sometimes.

• SourcePlease removes some of the insanity of SourceForge file listings by linking directly to a specific mirror’s copy of a file, removing the need to wade through five pages of ads.

• TextareaResize sounds cool but doesn’t seem to work.

• SearchTermHighlighter gives you Google cache style highlighting of pages entered through a major search engine. The highlight is non-intrusive.

Come on, I know you guys have some gem recommendations – what am I missing?

Kevin Barnes’ Freedom Languages is one of the best high level takes on the language divide I’ve read in a long time. He nails the major differences between the situation previously referred to as “dynamically vs. statically typed languages”. But he goes much further, placing languages into either “Freedom” or “Safety” categories and then listing the relative merits of each. I for one will be adopting this update in terminology.

As an illustration of how dead-on Kevin is in his analysis, here’s a link annotated paragraph on what you can expect from the “Freedom Language” advocate:

The advocates of freedom languages tend to talk first about the speed and efficiency of the individual programmer. They discuss the expressive power of different constructs and focus on all the powerful features that the safety languages lack. They point out complex patterns and show off twenty-line systems that do the same thing. They talk more about the ease or purity of things than the safety of things. They are dismissive of static-type-safety and compile-time validation in general.

Also, be sure to check out James Robertson’s commentary on Kevin’s post as well.

If I were to state that software industry is in crisis, I would be guilty of stating the obvious. The field of software development has been in crisis ever since the early ‘90s, if not even earlier than that.

But what’s important is to realize that nowadays, this crisis has reached major proportions. I know I sound like yet another ‘the sky is falling’ FUD protagonist, but there simply isn’t any other way to look at the problem. A significant number of mid sized and large corporations worldwide have been ‘once bitten, twice shy’ when it comes to making any significant investment in their software practices. Thus, the status quo situation is growing stale at a breakneck pace.

So why am I bringing this up? Simply because I am sensing an impeding tension building up between the proposed practices of software industrialization as opposed to staying with the tried-and-true practices of crafting software by hand.

Software industrialization (you can read more about it here) proponents insist that, in order to get out of the software development slump, we must make a transition from the hand crafted way of developing software to a full blown industrial assembly line model.

Fine. Automate as much as possible, rendering the whole process much more affordable. What could be wrong with that?

But not so fast. At the same time, while we’re getting caught in endless deliberation over whether Microsoft or Java or Oracle or SAP or LAMP or…? should be the worldwide software ‘industrializator’, another light at the end of the tunnel is becoming more and more visible. This light is what I would call the ‘smart servant’ approach.

So, instead of the proposed ‘dumb workforce’ mass industrialization model, an alternative model is being proposed whereby a handful of smart servants should be deployed to accomplish the workload of huge army of faceless, nameless drones.

In other words, instead of generating faceless, nameless reams of cheap software code, the alternative approach offers the possibility of hand crafting very high quality software code that will bring additional power to its users.

The ‘smart servant’ principle claims that it is better to employ a smart person or device, who will be able to accomplish complex tasks upon hearing just a handful of simple commands, than to employ an army of dumb devices who will have to be fed large lists of aggregated complex instructions, to be followed blindly.

The question, of course, is why should a smart servant be a better model?

Before I offer my views on why this should be so, I’d like to pause for a minute, and see if anyone would like to tackle this little challenge?

And so, without further ado, I’d like to now open the floor for a brief discussion on this topic…