lesscode.org

I thought it unfortunate that the controversy kicked up by a few (now retracted) passages in Brian Fioca’s article about WASP detracted from a proper discussion of the article’s real flaw. I addressed the controversial, less interesting sentiments in comment over there, but wanted to take a few moments for a longer commentary. It took a while, but here we go.

The true failing in that article is a terminal case of corporate-management world view. Let me quote a little out of order to highlight this:

Sure, you can write spagetti code in PHP 5, but if you have a well designed framework that keeps PHP code outside of your HTML and in tightly structured classes, you’re more likely to end up with code that looks and works and feels like Java.

The implication appears to be that anyone in the real world, if only they’re using Java, writes well structured and maintainable code.

I think the real world has a few surprises in store for people who think that way. As the saying goes, real programmers can write Fortran in any language.

Deemed “hacker” languages, scripting languages can be quick to write, but since they do not have many of the advanced features of compiled languages like C++ and Java, they have been prone to lax design practices, leading to code that isn’t efficient, stable, or maintainable enough for large solutions. With the correct mindset and help from structured frameworks like WASP, this no longer has to be the case.

This claim is a crock. PHP has a very low barrier to entry; consequently, a lot of eye-wateringly bad programmers are writing code in it; hence, a lot of code written in PHP is awful. This is arguably a good thing: a lot of people who have small needs are empowered to meet them on their own despite having little skill, which would never happen if Java were the only option available.

To conclude that if these programmers did learn Java, they would suddenly turn out maintainable code, or that if a good programmer changed camps from Java to PHP, the quality of his code would suddenly degrade, is fallacious.

Again, you can write pretzel logic in any language.

Until recently, platforms like Java were more likely to provide a stable, proven foundation to design and build well designed code, however by their nature they introduce a level of complexity that takes extra time to overcome.

The entire “more likely” clause is a myth, perpetuated by a middle management population that has absolutely no clue whatsoever what makes code good.

Using scripting languages like PHP tended to produce code in a faster time frame, but it was often impossible to maintain the architectural integrity necessary for building maintainable, extendable applications.

This is true – in an extremely local sense.

The overall design of large Java systems will tend to be more convoluted than that of almost any system written in a dynamic language, simply because the contortions forced by Java require larger overall designs.

Java buries the intention of code under a thick layer of implementation details and typing bureaucracy. Programmers are drilled to execute the requisite gruntwork in repeatable, predictable ways. Moreover, doing so conditions them to design their own system to cater to these reflexes. A maintainance programmer who comes in later will therefore easily be able to make changes to the mortar of an application. Unfortunately, that does not ensure good high-level designs nor does it translate to easy refactoring of convoluted architecture.

Dynamic languages abstract away huge swathes of gruntwork and require less structure. This means small systems have small implementations, which directly translates to better maintainability. The price is a loss of exemplary structure and of conditioning, which requires better innate architectural skills and stronger discipline when building larger designs; neither are strong points of the average programmer. However, designs in such languages aren’t frequently large, because the language does not require large architecture for modestly sized systems – which the majority are.

I have seen as many examples of truly terrible designs in Java as I have seen examples of completely unreadable code in Perl. Maintenance problems may manifest in apparently different ways, but their cost does not degrade appreciably when they afflict a system written in a “more maintainable” language. The only objectively measurable difference is in the amount of required code.

Why would you want to stick with PHP even in situations when you can use Python, Ruby, or anything else?

Simple. Because it’s cheaper to hire programmers that can write code in PHP. And if you have a good technical lead and a well implemented framework, your cheap programmers are going to pick it up and run and write good (and maybe even less) code.

(Brian writes this in a response in the comments, not in the article.)

This is a fear-based middle-management hiring strategy. Good developers know that “cheap programmers” will never hit the high notes. You can hire 100,000 code monkeys and they will never produce the same quality as 5 stellar programmers.

All these quotations are pervaded by a disregard for the fact that programmers are craftsmen; for the fact that that the language, framework, and whatever else they use in the course of their work is a tool. Good tools matter, but the stated opinions read as if the programming language is the principal factor in the quality of the result, when the truth is that it’s the programmers who ultimately matter.

If you hire good programmers, maintainability won’t intrinsically be affected by whether your in-house software development has standardised on Java, PHP, Perl, Ruby, Lisp, Smalltalk, Haskell, or Self. Good programmers can appreciate and can create good design regardless of language, and they can absorb a new language easily enough. Or as Joel Spolsky says:

I have never met anyone who can do Scheme, Haskell, and C pointers who can’t pick up Java in two days, and create better Java code than people with five years of experience in Java, but try explaining that to the average HR drone.

But whether your system is written in Java or an exotic programming language, every single programmer you hire will have to get acquainted with its architecture – which is hence the major factor in its maintainability. Comparatively, the chosen language is barely more than a neglibility, assuming it has a good impendance match with the type of application being written.

In thinking about this issue, I am in fact tempted to recommend that you hire programmers who are unfamiliar with the language your system is written in – because if they still struggle with the basics of a new language after a month, you might want to reconsider their future with your company. But that may be too bold a claim. Even if, in lesser form, it has been made before.

The bottom line is that you need people capable of producing a clean, flexible and only-as-complex-as-necessary architecture for the problem they were tasked with solving. And you won’t get that from an army of code monkeys, even if you hire 1,000,000 of them.

Apologies for cross-posting again, but I thought this little piece on radical simplicity could be of interest to lesscoders. The reason I didn’t post it here is simply due to the fact that it contains a bit of an illicit language, which I wasn’t sure would be tolerated inhere. So anyway, if interested, read all about it here:

Dave Chapelle Endorses Radical Simplicity

Stumbled upon this at InfoWorld (it’s the Alan Cooper interview) :

One of my assertions is that code is not an asset. A lot of companies say their job is to maximize the revenue from our code base. That reflects a kind of an Old World, industrial-age thinking. What’s interesting is the open-source movement is kind of our first proof that code in fact is not an asset. What is your company’s asset is the experience and knowledge that the people who have built your code have gathered during the construction of that code. But the code itself doesn’t have a lot of value. In fact, the code kind of anchors you down. In the traditional manufacturing business 30 years ago, if you had a warehouse full of spare parts and completed products, that was an asset that you put on your books. Today if you’ve got a warehouse full of manufactured goods, that’s a huge liability. In the world of software, it’s your relationships with your customers and vendors and the knowledge of your people that are valuable. This is part of the movement from a product-centered world to a service-centered world that we’re going to. I think the open-source movement is sort of like a moon mission. They went to the moon to see if we could go to the moon, not to set up a burger stand there. People who will come after will set up burger stands there. Linux may or may not be commercially viable. But I think that the people who follow in their footsteps will find that software is service. The software itself is not of value.

Sounds kind of extreme to me, but in actuality says a lot about the value of lesscode.

Now, I’ve been a Cooper-head since 1996. I’ve spent a lot of my brain cycles over the years honing my interaction design skills. And I firmly believe that Alan is absolutely right in his assertion that only desirable products and services make any sense.

As we already know (and agree), lesscode is all about agility. Since software code is not an asset, but rather a liability, the more we can reduce the deadwood, the better off we are. A side-by-side comparison of the code written in RoR with the code doing the same thing in Java or .NET reveals striking discrepancy in size. It is definitely possible to deliver high level of functionality, interactivity and sophistication by utilizing only a portion of code that would normally be used if we stick to the old school (morecode, or more LOC). And that’s a desirable thing.

This is a continuation of my last post, which talked a bit about the platform selection process I’ve been involved with from a fairly high level. I concluded that choosing between .NET, Java, and LAMP/friends for me came down to making two important decisions: where do I fall on the safety/freedom scale and in what manner would I like to proceed with my platform of choice? The first decision makes the second (or vise versa) by dictating whether you will be proceeding primarily by trying to make the chosen platform more safe or by trying to make the chosen platform more free.

While we haven’t decided to throw out our .NET based application (we have no reason compelling enough to justify a rewrite at present), I think it’s safe to say we’ll be moving forward with a different platform. The road here has been kind of weird so I’d like to talk about my activity over the past few months with regards to assembling a basic platform.

I’m a Python guy and have a tremendous interest in seeing Python grow. It’s my language of choice and it’s where I’m most productive. I also enjoy the community a great deal and am constantly humbled by the amount of competence some of these people are toting around. But there are a few gaps that I felt needed to be bridged (or at least needed a plan and visible progress) before I felt comfortable making a recommendation for Python as our general purpose language for building future web applications.

There’s pretty much constant banter in the Python community on which of the following two items is most important: refine the language or restructure / enhance the standard library. For someone in my situation, neither of these issues even blip the radar. Here’s what I see as the gaps in Python as a language for basic web development on top of some variation of LAMP. I consider these to be essential pieces of a language like Python’s overall feature-set (e.g. they should be considered batteries and should be included):

A Good Project Documentation Utility

pydoc is amazing from the terminal but is sorely lacking in its web interface. There are also considerable issues with the basic model of runtime introspection, such as not being able to determine (easily or reliably) whether a certain module/class attribute belongs to a certain module or was imported from somewhere else.

Beyond basic pydoc functionality, I’d like the ability to incorporate external documentation files (in text, Restructured Text, or HTML formats) into the doc build process such that building the documentation results in a static local copy of the project web-site. Some basic templating would go a long way here as well.

The Python Doc-SIG has enjoyed very little activity lately that I’m aware of outside of progress on restructured text. David Goodger and the rest of the folks working on the Docutils project have a plan for an enhanced source-based documentation utility with enhanced docstring formatting rules but progress seems to have slowed or halted.

A standard, Python-based build system

Something simple and task based like make, Ant, or Rake. I think distutils is a fine model but needs to be considered from a more generic build/make style perspective. Ideally this would have its own SIG and reach general acceptance in the community on the level of distutils.

Some of the things I’d like to be able to distill down into a single build script for a project include:

• Initializing databases with SQL files or CSV dumps.
• Build and package all versions of distributables with options for generating signed checksum files or directory indexes.
• Push distributables to other environments and deploy.
• Publish documentation built with the imaginary documentation utility to the project website.
• Send out announcements of new releases.

Normal make type stuff really but simpler, cross platform, and removing the need for some to learn a new syntax (as much as you can actually “learn” make). If modules were easily pluggable that would be great to but I’d like a ton of varied functionality out of the box.

I know of no serious activity around this. There are a few decent build tools that are Python-based (SCons) but they seem oriented toward building non-Python projects.

A standard, Python-based distribution mechanism

This one is seeing some real progress with setuptools, Eggs, and EasyInstall thanks in no small part to Phillip J. Eby, Ian Bicking, and a bunch of other people working as part of the Distutils SIG.

I’ve had a chance to watch progress on this closely and contribute a little and it has just reaffirmed everything I love about the Python community when it decides to agree on something and dig in. In fact, this isn’t really a concern because these technologies are already enjoying wide-spread adoption and have reached a basic level of stability very quickly. That wasn’t the case even two months ago when I was doing this whole platform evaluation/assembly thing.

A Basically Good, Basically Complete Web Framework with a Huge Community

I know, I know, we’ve talked about this forever, during which time I’ve taken every possible position. The “Basically Good” part is met and exceeded with every Python web framework I’ve ever worked under, no doubt. The “Basically Complete” part means I want a full stack with project management capabilities. I don’t care if it’s pieced together from other components or built from the ground up. I’ve tried three of these: Django, Subway, and Paste/Webkit/SQLObject and I’ve liked them all fine enough for me to get work done.

I want a Huge Community.

I do very specific types of work over and over building web applications. I’d estimate that 80% of my code is very specific to building the same old normal web app and the remaining 20% of code is spread out in support libraries, patches, other general application development, or some new and cool style of web programming :) Given that ratio, I’ve made it a priority to find ways of maximizing my productivity through that 80% of code because it provides the biggest bang for the buck.

I’m fairly certain there are a large number of developers working on those same very specific types of tasks with 80% of their code and that if I would just talk to them, we would be able to simplify and enhance the basic model further, faster.

In some ways, the size of the community around the web framework I use is more important than the size of the community around language fundamentals and lower level libraries.

Again, I’m a couple of months behind in following everything closely and it looks like Django is functioning really well as a community. Ian’s Paste has some interesting qualities, though, and I wish Django would push more discussion into the Web-SIG. This is my problem; I sit here and do this and it takes up a piece of my 80% instead of relieving it. Now you’re doing it!

Some Assembly Required

Having established a basic LAMP environment and identified a few pieces I’m missing, I do what I’m supposed to do: I start building it as part of the community:

• buildutils - The build tool.
• pudge - The documentation tool.
• lesscode.org - A weblog where I will attempt to convince people that they should be working on what I’m working on at various levels :)

I also contributed various patches and testing to setuptools when it was in very early stages of development and continued evaluating progress on the web frameworks. If these components could reach maturity and something shakes out on the web framework front, I feel I could just kick ass and take names. Hard problems will still be hard but I would have a simple, manageable, cost effective, community backed, agile, and free-as-in-Kevin system for web applications.

And then I began evaluating Rails for a series of Django comparison articles. I knew that Rails had the Basically Good, Basically Complete Web Framework with a Huge Community - Python will have one too. But I lost it when I found that Ruby has a standard build tool in Rake, a standard documentation tool that meets my needs in RDoc, and that Gems were everywhere. I had assumed that these would be no further developed than their analogs in the Python world.

The long and short of it is that my evaluation turned into what looks to be a long term relationship. I’m committed to my responsibilities on the Python projects I have going right now and I have a ton of existing applications and utilities written in Python that aren’t going anywhere but Rails ganked 80% of my future code somehow. What’s cool is that because I’m dealing with freedom languages, I don’t feel like I have to make a single choice here because they work equally well in my environment (and most any environment). What’s odd is that it wasn’t Rails that finally pushed me over so much as Ruby and its mysterious perfectly placed support libraries and utilities.

This is a two-parter in two parts.

A few months ago I became partial owner of a small technology company. A long time friend and colleague had started in on a set of web applications for managing (US) health benefits (wait - don’t go anywhere yet, this will get on topic in a second) and succeeded in convincing me that this was an area of business IT that could benefit excessively from being part of the web. About 60% of this particular application’s functionality was there when I signed on and was built using Microsoft’s .NET, SQL Server, and IIS. My partner had little experience with the technologies I tend to recommend on this site and so the decision was made to move forward with the safety of .NET while researching potential alternatives for future work.

My days have been spent refining and growing the .NET app while my nights were spent researching and assembling a platform that would allow us to work in a much more agile environment. I have a fair bit of experience working with LAMP (where “P” is for Python and “M” is for PostgreSQL). That experience served as a starting point but what I’m after is much more than a simple running environment. I’d like to pull together the various tools necessary to develop, test, manage, maintain, and deploy applications built on the LAMP foundation. That means everything from Subversion to Mailman, Trac to Wordpress, and on down into the guts of a hacking environment like build tools (make, etc.), automated documentation utilities, continuous integration and testing, etc.

That’s the briefest possible history to bring us up to present time where I’d like to talk about two important observations I’ve made over the past few months of being in this situation. I’ll be talking primarily about Microsoft but everything here is applicable to the Java platform and, in some cases, is more so because I’ve made a few assumptions about what is available for .NET based on Java experience (for instance, I’ve barely used Nant or NUnit but assume they’re roughly on par with Ant and JUnit).

The first observation (and the one I will be exploring for the rest of this post) comes from being in extremely close, and somewhat handicapped, competition with the Microsoft platform. Please remember that I am partial owner of this company, have a working application written in .NET, and feel extreme amounts of pressure to create revenue quickly while keeping cost (in both time and money) at a minimum. This situation has forced me to use extreme prejudice when evaluating tools and technologies. In some ways I’ve inherited the responsibility of championing Microsoft’s platform (if only to myself) as my situation requires that I provide real evidence that shows not only that gains can be had by using something different but that these gains are significant enough to justify the cost of that change. In general, the situation just plain sucks - I have to work much harder than if we were to secure a couple million in R&D budget for the year or were juiced up on venture cap. I’m not saying that because I wouldn’t give 100% of my capacity in those situations but it’s hard to imagine manufacturing the level of discipline created when your house is sunk into something.

Back to Microsoft’s platform: they have all the major pieces in one place, are fairly organized, and are extremely accessible. However, time and again I’m finding that I can beat them in almost any given area with lightweight F/OSS technology but I can’t beat the platform as a whole. It’s just all there. The sense of safety this creates cannot be ignored (I’ve tried).

I want to be absolutely clear about that last point because I’d hate to be misrepresented: in a great number of cases, I’ve found individual F/OSS technologies to be superior to their Microsoft counterparts but the lack of an overall platform whole is distressing. I don’t care about indemnity, paid support, certification, finding people to hire, TCO (blech!), etc. because I already understand the F/OSS answer to those questions and they are compelling. I do care about being able to answer the phone, talk to a customer, and feel confident in what and when I can deliver. Microsoft either gives me that ability or gives me the illusion of having that ability - it’s assuring and troubling at the same time.

All this is to say that Kevin Barnes’ Freedom Languages really hit a chord with me. I have very little doubt in the area of language: Python or Ruby just completely destroy any of the mainstream .NET languages and Java to boot. What makes me feel safe is the consistency of the rest of the tool-chain, the massive amount of component libraries, and the breadth of the core platform.

I keep going back to Kevin’s idea of measuring things on the axes of Freedom vs. Safety, where “freedom” isn’t so much GNU-freedom-type-freedom but rather a sort of aggregate measure of the qualities we find most compelling in software design and architecture. Weighing things on this scale helps explain some of the intuition I’ve been feeling over the past months. If we measure the three major platforms competing for business mind-share, I think you find first that Microsoft is extremely safe while offering the least freedom. Next we have Java, which is pretty safe too but the established F/OSS communities (e.g. Apache, Codehaus, etc.) and even the vendor’s (Sun, IBM, BEA, etc.) positions provide a bit more freedom. We then arrive at LAMP and friends which tend to sacrifice safety but provide unmatched freedom and thus, when measured along the web’s core axioms, quality.

Given those basic metrics, choosing a platform comes down to making two key decisions: where do you fall on the safety/freedom scale and in what manner would you like to proceed? Your remaining time (e.g. forever) will be spent doing one of two things: making the platform more safe or making the platform more free. You have infinitely greater control over one of these situations.

We played the fence but I believe we’re arriving at a turning point. I’ll talk about that in part two because the areas explored there are different enough to warrant a separate URL - mainly my experiences in making the LAMP platform more safe and complete but with twists and turns that will eat your brane.

Take Me To Part Two - Assembling The Platform