lesscode.org

“I laugh the laugh of a third world designer.” A gem from James Governor.

This post is a bit all-over-the-place, sorry.

I’ve stumbled across Code Snippets at least 5 times in the past couple of days. It’s basically del.icio.us for small pieces of code. Each snippet gets a title, description, actual code snippet, a set of tags, and most importantly a URL. The result of this situation is obviously great search engine indexability because, as I said, I’ve happened upon it at least 5 times now through basic Google usage.

What’s interesting is that, as far as I can tell, they’ve placed NO limitations on what type of snippets can be posted. There’s a quick bash two-liner for Automatically adding a bunch of stuff to CVS next to a Generic XHTML Template next to a Python snippet for generating midi tones on a Series 60 cell.

Considering this from more abstract level, you might call this a demonstration of a few bits of theory laid out by Doc Searls and David Weinberger in World of Ends. Here we have two ends (Google and Code Snippets) that work well together due to a common level of understanding of what’s desirable in the larger system they both operate in. The value of each end seems to increase with each new end that it touches. I think this basically follows Metcalfe’s Law, which states that “the value of a network equals approximately the square of the number of users of the system (n2).” Only “users” in this context can mean “other systems”. In this case, Code Snippets enhances Google and Google enhances Code Snippets. You might also say that Code Snippets gets more value from Google than Google gets from Code Snippets and that the actual value each obtain is close to that predicted by Metcalfe (or not).

Anyway, the enhanced searchability this style of organization facilitates got me thinking about the quality of metadata at del.icio.us proper… Did you know that Joshua disallows access to / for all robots? and that Google is not spidering the amazing set of collobrative metadata available there? Having the Googlebot run through del.icio.us on a regular basis would be insanely expensive for del.icio.us - Technorati did it for about two weeks and then were cut off if I remember correctly.

But why shouldn’t Google/Yahoo/whoever purchase the bandwidth and other resources necessary to run their spiders over del.icio.us? If you did pay, spidering probably wouldn’t be the best method of getting at the meat on del.icio.us. If I were a search engine, I would try to convince Joshua to lease me a basically persistent stream from this RSS 1.0 Feed. You should be able to put something together that improved the quality of your search dramatically with the quality of data in that stream.

Finally, it’s interesting to note that “/rss” is the only resource robots are allowed to access:

http://del.icio.us/robots.txt:

User-agent: *
Disallow: /
Allow: /rss

If del.icio.us were so inclined, I think it’s reasonable to believe that they could start picking up revenue by leasing high quality access to that URL.

Tim Bray warms up to Ruby, and had this to say about the value of static typing:

“Call me hidebound and conservative, but I think that ‘optimizing code’ and ‘helping IDEs’ (and it’s a whole lot more than just ‘tooltip help’) are awfully damn important. In particular, as James Strachan has often argued, the combination of a good modern IDE and a statically typed language mean that you hardly ever have to type out a full method or variable name, and even though you might have to write more lines of Java than you would in Ruby, you might get the code written just as fast.”

Yes you might get the code written just as fast. And in truth you can really bomb through Java code in a tool like IDEA. But after you ship, you will have more code to deal with, you will have more tools in the chain, you will have more technology to deal with. All that extra stuff and surface area is not neccessarily a good thing. And there’s no guarantee the system logic will be as flexible as the one written in some other language. “But my refactoring IDE will help me deal with all that” - yes, and this is what we might call a self-fulfiling prophecy ;)

On the other hand the most common non-idiosyncratic compliant I hear about Python is not being able to know what a function takes or returns without jumping out to the function source. This sounds trite, but it appears to really drive people to distraction - I suspect it has something to do with breaking flow.

But if this is the sole argument left against dynamic typing (and it appears to be, along with the human resources), then the dynamic camp have come a long way . Komodo and Eclipse PyDev shows what you can do in terms of autocomplete; not nearly as good as the features provided by IDEA and Eclipse java, but which might be good enough.

It is becoming increasingly difficult for me to envision why would anyone not consider lesscode much more desirable than morecode? Software code is like the headcount: you typically only hire as many people as you need in order to successfully run your business. Anyone that might suggest to you that it is much better to double or triple your payroll, ‘just in case’, is simply hosing you.

But I am getting mighty mystified by the fact that so many developers tend to lean towards and support the morecode way of doing software development. Why would anyone in their right mind embrace enthusiastically the proliferation of code? I see that developers are now cheering about the introduction of annotations (yet another extra conceptual space where code proliferation is bound to occur). Why would people celebrate something that is only going to make their lives more complicated?

Then it dawned on me – we’re in the midst of vendor wars. It is a trivial observation, but software vendors are manipulating the situation in the attempts to - dare I say it? - rule the world!

From a typical software vendor perspective, morecode is not necessarily a bad situation. Pushing a platform/vision on unsuspecting population of developers, whereby forcing them to engage in the practice of morecode is not something over which software vendors are necessarily losing their sleep. At the end of the day, it will be us, developers in the trenches, who will have to wrestle with the morecode migraines.

I’ve always found it curious how easy it is to recruit foot soldiers for a particular vendor. There never seems to be shortage of Microsoft or Java or Oracle etc. apologists, who are canvassing the tradeshows, propagating the vendor of their choice. They appear as if they’re working for their sweetheart vendor, but what’s fascinating is that all these Herculean efforts are voluntary. No one pays them to do all the free marketing and advertising for Microsoft, Java, Oracle, IBM…

Why would someone sacrifice so much of their lives to serve some giant vendor? I must honestly say I really don’t understand the motivation behind that. However, it neatly explains why so many developers are happily embracing the morecode gospel.

This is a continuation of my last post, which talked a bit about the platform selection process I’ve been involved with from a fairly high level. I concluded that choosing between .NET, Java, and LAMP/friends for me came down to making two important decisions: where do I fall on the safety/freedom scale and in what manner would I like to proceed with my platform of choice? The first decision makes the second (or vise versa) by dictating whether you will be proceeding primarily by trying to make the chosen platform more safe or by trying to make the chosen platform more free.

While we haven’t decided to throw out our .NET based application (we have no reason compelling enough to justify a rewrite at present), I think it’s safe to say we’ll be moving forward with a different platform. The road here has been kind of weird so I’d like to talk about my activity over the past few months with regards to assembling a basic platform.

I’m a Python guy and have a tremendous interest in seeing Python grow. It’s my language of choice and it’s where I’m most productive. I also enjoy the community a great deal and am constantly humbled by the amount of competence some of these people are toting around. But there are a few gaps that I felt needed to be bridged (or at least needed a plan and visible progress) before I felt comfortable making a recommendation for Python as our general purpose language for building future web applications.

There’s pretty much constant banter in the Python community on which of the following two items is most important: refine the language or restructure / enhance the standard library. For someone in my situation, neither of these issues even blip the radar. Here’s what I see as the gaps in Python as a language for basic web development on top of some variation of LAMP. I consider these to be essential pieces of a language like Python’s overall feature-set (e.g. they should be considered batteries and should be included):

A Good Project Documentation Utility

pydoc is amazing from the terminal but is sorely lacking in its web interface. There are also considerable issues with the basic model of runtime introspection, such as not being able to determine (easily or reliably) whether a certain module/class attribute belongs to a certain module or was imported from somewhere else.

Beyond basic pydoc functionality, I’d like the ability to incorporate external documentation files (in text, Restructured Text, or HTML formats) into the doc build process such that building the documentation results in a static local copy of the project web-site. Some basic templating would go a long way here as well.

The Python Doc-SIG has enjoyed very little activity lately that I’m aware of outside of progress on restructured text. David Goodger and the rest of the folks working on the Docutils project have a plan for an enhanced source-based documentation utility with enhanced docstring formatting rules but progress seems to have slowed or halted.

A standard, Python-based build system

Something simple and task based like make, Ant, or Rake. I think distutils is a fine model but needs to be considered from a more generic build/make style perspective. Ideally this would have its own SIG and reach general acceptance in the community on the level of distutils.

Some of the things I’d like to be able to distill down into a single build script for a project include:

• Initializing databases with SQL files or CSV dumps.
• Build and package all versions of distributables with options for generating signed checksum files or directory indexes.
• Push distributables to other environments and deploy.
• Publish documentation built with the imaginary documentation utility to the project website.
• Send out announcements of new releases.

Normal make type stuff really but simpler, cross platform, and removing the need for some to learn a new syntax (as much as you can actually “learn” make). If modules were easily pluggable that would be great to but I’d like a ton of varied functionality out of the box.

I know of no serious activity around this. There are a few decent build tools that are Python-based (SCons) but they seem oriented toward building non-Python projects.

A standard, Python-based distribution mechanism

This one is seeing some real progress with setuptools, Eggs, and EasyInstall thanks in no small part to Phillip J. Eby, Ian Bicking, and a bunch of other people working as part of the Distutils SIG.

I’ve had a chance to watch progress on this closely and contribute a little and it has just reaffirmed everything I love about the Python community when it decides to agree on something and dig in. In fact, this isn’t really a concern because these technologies are already enjoying wide-spread adoption and have reached a basic level of stability very quickly. That wasn’t the case even two months ago when I was doing this whole platform evaluation/assembly thing.

A Basically Good, Basically Complete Web Framework with a Huge Community

I know, I know, we’ve talked about this forever, during which time I’ve taken every possible position. The “Basically Good” part is met and exceeded with every Python web framework I’ve ever worked under, no doubt. The “Basically Complete” part means I want a full stack with project management capabilities. I don’t care if it’s pieced together from other components or built from the ground up. I’ve tried three of these: Django, Subway, and Paste/Webkit/SQLObject and I’ve liked them all fine enough for me to get work done.

I want a Huge Community.

I do very specific types of work over and over building web applications. I’d estimate that 80% of my code is very specific to building the same old normal web app and the remaining 20% of code is spread out in support libraries, patches, other general application development, or some new and cool style of web programming :) Given that ratio, I’ve made it a priority to find ways of maximizing my productivity through that 80% of code because it provides the biggest bang for the buck.

I’m fairly certain there are a large number of developers working on those same very specific types of tasks with 80% of their code and that if I would just talk to them, we would be able to simplify and enhance the basic model further, faster.

In some ways, the size of the community around the web framework I use is more important than the size of the community around language fundamentals and lower level libraries.

Again, I’m a couple of months behind in following everything closely and it looks like Django is functioning really well as a community. Ian’s Paste has some interesting qualities, though, and I wish Django would push more discussion into the Web-SIG. This is my problem; I sit here and do this and it takes up a piece of my 80% instead of relieving it. Now you’re doing it!

Some Assembly Required

Having established a basic LAMP environment and identified a few pieces I’m missing, I do what I’m supposed to do: I start building it as part of the community:

• buildutils - The build tool.
• pudge - The documentation tool.
• lesscode.org - A weblog where I will attempt to convince people that they should be working on what I’m working on at various levels :)

I also contributed various patches and testing to setuptools when it was in very early stages of development and continued evaluating progress on the web frameworks. If these components could reach maturity and something shakes out on the web framework front, I feel I could just kick ass and take names. Hard problems will still be hard but I would have a simple, manageable, cost effective, community backed, agile, and free-as-in-Kevin system for web applications.

And then I began evaluating Rails for a series of Django comparison articles. I knew that Rails had the Basically Good, Basically Complete Web Framework with a Huge Community - Python will have one too. But I lost it when I found that Ruby has a standard build tool in Rake, a standard documentation tool that meets my needs in RDoc, and that Gems were everywhere. I had assumed that these would be no further developed than their analogs in the Python world.

The long and short of it is that my evaluation turned into what looks to be a long term relationship. I’m committed to my responsibilities on the Python projects I have going right now and I have a ton of existing applications and utilities written in Python that aren’t going anywhere but Rails ganked 80% of my future code somehow. What’s cool is that because I’m dealing with freedom languages, I don’t feel like I have to make a single choice here because they work equally well in my environment (and most any environment). What’s odd is that it wasn’t Rails that finally pushed me over so much as Ruby and its mysterious perfectly placed support libraries and utilities.