Lesscode using Domain Specific Languages (DSL)
By Mitch Barnett under Talk on 28. September 2005Having spent some time in the VS2005 environment, I can say the following about DSL’s:
A domain-specific language (DSL) is a language designed to be useful for a specific task in a fixed problem domain, in contrast to a general-purpose language. DSLs are gaining popularity in the field of software engineering to enhance productivity, maintainability, and reusability of software artifacts, and enable expression and validation of concepts at the level of abstraction of the problem domain.
Using domain-specific languages, one can build customized modeling tools and essentially define a new modeling language and implement it very simply. For example, a specialized language may be used to describe a user interface, a business process, a database, or the flow of information, and then used to generate code from those descriptions.
I built a (small) DSL for modeling application integration scenarios, which is always an issue in the IT business world. First, I defined all of the specific domain model terms used in application integrations scenarios such as XML messages, source – destination applications, XSL maps, business units, protocols, business rules, etc. Then I described the designer definitions that make up the visualization tool. Once the meta data was defined in a supplied Visual Studio template, you build the solution and another instance of Visual Studio fires up with your visual designer implemented.
You then use the visual designer you just created to draw/model the application integration scenario and when you run this solution, it code generates the solution.
Of course, I have left out a lot of detail and it was not easy the first couple of times. I have left out a set of code generators, which take a domain model definition and a designer definition as input, and produce code that implements both of the components as output. The code generators also validate the domain model and designer definition, and raise errors and warnings accordingly. But eventually, you get the hand of it.
Think of it this way, DSL is a tool for building tools :-) For example, anyone that has used the Class Designer in Visual Studio is using a DSL outputted visual designer, specifically designed for building classes.
The process has been quite the learning experience for me and has proven to be very enlightening. I am an old guy, been writing code for 15 years. Quite frankly, I don’t give a hoot about which programming language I use cause I see them all as being the same, some better than others, but still hand crafting code. I don’t want to hand craft code anymore – every time I get involved in a software development project I a) have done it before and b) oh man, this is going to take 6 months of grinding it out. In other words, it’s gonna hurt.
I see DSL’s as a evolution in our software industry to use higher level abstractions in the way of visual designers to so that I can spend time “designing” the solution and have most of the infrastructure code generated for me, that I otherwise would have to grind it out.
If anyone is interested in more information about DSL’s and don’t mind reading it from Microsoft, download the DSL Toolkit and have a read of the documents at: Microsoft Domain-Specific Language (DSL) Tools
Ravi Mohan:
Excellent Post.
Most writing about DSL are very vague and not based on actual experience as yours is. That alone makes this a refreshing read. I have been using the same approach to build aMachine Learning Framework, and while optimizing the copiler is a little hairy the effort pays off handsomely.I am also a fairly “old guy” (10 years) and whle I belive that you need to be able to roll your own interprters/code generators before depening on tools to do so, I agree with your idea of DSLs as the next step.
Good writing.Keep this up!
comment at 29. September 2005
Aristotle Pagaltzis:
Strange. To me, DSLs are the antithet of code generation. To me, DSLs mean that the language is geared to the problem space so extensively that I can hand-write the solution to a task, directly, in the DSL. In other words, DSLs fall firmly on the “language maven” end of the IDE divide.
Either you have completely misunderstood something or I did.
comment at 29. September 2005
akash:
Do you know about Logix. A Meta language created in Python to create Domain Specific Languages. We had a presentation by the creator once.
http://www.livelogix.net/logix/
Thought you might like to check it out
comment at 29. September 2005
Alex Bunardzic:
I’m afraid some of you may have misunderstood Mitch’s post. Of course, I don’t want to speak on his behalf, but being coworkers we get to spend time together talking about these things. And I think I happen to know where he is coming from, which is away from the code and towards the design. In other words, the focus is upwards, towards the human comunity, not downwards, towards the machinery.
I hope I got this right, as I don’t want to dillute Mitch’s message. If not, at least that’s my own personal orientation. And from this perspective, I agree with Mitch that it’s better to focus on the domain than on the infrastructure.
comment at 29. September 2005
Alex Schroeder:
Wasn’t Lisp all about writing applications as an implementation of a domain specific language? In this respect I was quite surprised by the turn taken in the article. I was expecting something about building an application bottom-up, writing code that gives you more more and more data- and control-structures, building the vocabulary to finally allow you to solve the problem on high-level terms. If I take away the GUI part of the article, it could be about Lisp, I guess.
comment at 29. September 2005
Ryan Tomayko:
I don’t subscribe to the GUI as DSL theory either. In fact, I think there’s a lot of places (especially in visual studio) where GUIs make bad substitutes for stronger language features. Aristotle hit it on the head for me when he said:
When I think of DSL, I think of Lisp, Rake, Rails, SQLObject, etc. not GUI wizards and code generators.
comment at 29. September 2005
Nico Mommaerts:
Using visual tools and code generation seems to be the Microsoft way of using DSL’s. There are 2 kinds of DSL’s: embedded and layered. A classical example of this is Ant vs Rake. Ant was built by designing a new language (although using XML to avoid having to write a compiler/interpreter), Rake uses an existing language (Ruby). Anybody who has used Ant knows how frustrating and limiting it can be, you can’t fall back on another language, you are tied to the existing tasks or write new ones. The same is true for visual DSL tools, although you have a highly specialized language at your disposal, if you want to do anything not foreseen by the tool, you are stuck or have to bend into all kinds of uncomfortable corners.
A colleague of mine suggested reading Thinking Forth as a way of learning about DSL’s.
My (few) links about DSL’s are here.
comment at 29. September 2005
Mitch Barnett:
Wow! Interesting responses. I certainly never figured myself as a language maven. Maybe some context around why I wrote what I did about DSL’s.
My job is to solve business problems using IT where the requirements are generally very poor and there is little money or time to solve the business problem at hand. Just part of the reality in my little business world I live in from a Systems Integrator perspective. So to combat that, I try and find anything (beg, borrow or steal) that assists me in meeting unrealistic requirements and schedules. This means a lot of incremental and iterative development to close the aforementioned gaps.
I look at DSL’s as just another tool in the toolbox to help me solve the business problem at hand. As Alex has pointed out, my job is to design a solution to solve a business problem without having to write the same (or similar) plumbing or infrastructure code every time So spending more time in the problem domain helps me do this. Using DSL’s along with writing code, using 3rd party components and any other piece of design and/or code that I can reuse helps me solve the business problem.
Don’t get me wrong. I live to code. But business constraints of the real world force me to do (some) unnatural acts. The application integration DSL I put together, albeit small, greatly assists me in meeting those business constraints, hence the reason why I use it – in addition to traditional coding (interface development), etc. But when I can make a domain model/designer change in my DSL and have a solution delivered to my paying clients in fraction of the time it takes to traditionally code everything by hand, my clients really appreciate that and keep me employed. I am just being pragmatic about the whole thing.
I also believe that, as in everything software, everyone has a different meaning and motivation to use the languages and tools we use. Hopefully, with this little explanation some of that has come to light.
comment at 29. September 2005
Aristotle Pagaltzis:
Mitch: I see all that. The point is, using a wizards to generate code (even if the generator works on top of a model of the domain) still contradicts my understanding of DSLs.
In my mind, the entire point of a DSL is to be so abstract and high-level that you can write the code by hand but without writing any plumbing and infrastructure.
The canonical example of a language whose mentality is that you should do any task by creating a DSL of this type for the problem at hand and then using that DSL to solve it is Lisp, as Alex Schroeder points out. The main tool towards that end are macros—which are far more powerful in Lisp than in any of the mainstream languages.
(FWIW, such macros are coming to Perl6.)
comment at 29. September 2005
Mitch Barnett:
Aristotle - we definitely have a disconnect here. Who said anything about wizards? Let me state categorically that the DSL I am talking about is NOT wizard driven. I ask that you take 10 minutes of your time to look at the link I provided to the documentation in the first post to look at how MS is doing DSL’s - we are talking about a “high fidelity” modeling environment - a visual designer that you drop domain specific objects onto a canvas to draw/model the solution, using a DSL visual designer. Not wizards. This is something completely different than what you think it is or at least from your post reply’s what I think you think it is :-)
comment at 29. September 2005
Aristotle Pagaltzis:
Sorry, sloppy terminology. I tend to lump all the IDE-style stuff together, my bad. Please re-read my comment with “GUI designers” substituted for “wizards;” the points remain the same ones.
comment at 30. September 2005
Mitch Barnett:
Re: Aristotle and Alex S. - Sorry guys, I think you are missing the point entirely - DSL is NOT a programming language. It is a higher level abstraction using a “meta model”, in this case a visual designer to “describe” what you want to build and then the “framework” generates code, and in MS case, it is either C# or VB.NET. I am an old Smalltalk guy and I can tell you, this ain’t LISP - again, a DSL is NOT a programming language and that’s the whole point I am trying to make – obviously poorly.
Remember UML and class diagram’s where you could “draw” a class diagram hierarchy and Rational’s tool will build the code framework (ok stubs) for you? Well, a DSL is a much more refined and higher level abstraction that allows you to “design” a specific language to solve a problem domain issue of interest - without hand crafting code. That’s the point of a DSL. Who care’s “how” it is done. All I can tell you is that it would take me 10X longer to handcraft the code, compared to what I “designed” in the DSL to generate the code (more or less what I would have hand crafted in C# anyway) for me. In other words, a DSL is simply a productivity tool that allows me to describe or define or model the solution, using meta data. That’s it. Simply another tool in the toolbox. Nothing more, nothing less.
comment at 30. September 2005
Ryan Tomayko:
Mitch said:
I think the point was made aptly, there’s just some disagreement about your definition. For instance, Fowler describes DSLs as follows:
He then points to Eric Raymond’s Minilanguages and Paul Graham’s Programming Bottom Up, both of which describe using highly extensible languages (like Lisp) to craft new language features that serve a specific problem domain.
For more background, see Domain-Specific Languages: An annotated Bibliography, which cites some 78 historical DSL texts. The definition of “DSL” put forth in that paper is as follows:
To sum up, I don’t know what Microsoft is selling over there but it isn’t what the larger community has come to know as “Domain Specific Languages”. Perhaps a better term would be “Domain Specific Designer”.
comment at 30. September 2005
Mitch Barnett:
Point taken Ryan (and Aristotle and Alex S.). I had a look at the DSL definitions you supplied (and from others in this post) and understand why there might be different interpretations of the meaning of DSL. Of course Microsoft, being who they are, always seems to do things “their way”. Here is Microsoft’s view on the DSL tools they supply with Visual Studio”
“Using the Microsoft DSL tools you can create your own designer, integrated into Visual Studio, for a visual domain-specific language. The tools help you define the domain-specific language and generate the code of a graphical designer for you. The resulting designer uses the same underlying modeling technology that is used by the Class Designer and Distributed System Designers in Visual Studio 2005.”
In the end, I believe it complies with what a Domain Specific Language is, in its purest form, it’s just that Microsoft has a different way on “how” it achieves the end goal.
Personally, I have been a fan of visualization tools, particularly from an architectural design perspective, because it raises the level of abstraction, much like how architectural blueprints are common practice in the “traditional” building design/construction world, using visualization tools like AutoCAD. This is something we are still struggling with in the software development world – we don’t have a “standard” architectural blueprint that every programmer understands, from a semantic meaning perspective, and can build software that meets the specifications of the blueprint.
This is what I mean by software industrialization – I certainly don’t mean making it a mechanized process, it will never become that way, but at the same time, every physical buildings architecture has been derived from an “architectural blueprint “where everyone involved in the construction process knows how to read it and more importantly, knows what it means. From where I sit, we (still) don’t have anything like that is the software development world. To me DSL’s are one step closer to this inevitable realization.
comment at 30. September 2005
Alex Schroeder:
I think the problem is that most developers have never seen both powerful visual programming environments and useful non-trivial building blocks. If you have found both, more power to you!
comment at 05. October 2005
Steven Kelly:
Using “Designer” tends to put the focus on the tool; I think the equivalent term for the actually language should be “Domain-Specific Modeling Language”. I’ve also heard people talk about “textual DSLs” (which is fine) vs. “graphical DSLs”, but I’m not a fan of the latter term. In all the real industrial cases I can think of, the graphical format of the DSL does not show all the information contained in the model. To see the fine details of an object, you have to double-click its property dialog open or look at its property sheet. That’s an important advance over textual DSLs, which show all the detail. With a DSM Language the language designer chooses what is important enough to show all the time, and what can be hidden as details. He can also allow the modeler to make that decision for some elements (like most UML tools allow the modeler to show or hide the Attributes and Operations compartments of a UML Class symbol).
There was some discussion earlier about the difference between wizards, earlier code generators and DSM with full code generation. The main difference is whether you have to maintain what is generated, at the level of the generated code. Wizards and old-style code generators generally meant you had to do that: they produced lots of code quickly, but it was unfamiliar to you and you needed to maintain and probably extend it. With DSM, the code generated is the same as the code you wrote by hand before, and you don’t need to read or edit it. (See this blog entry for more detail.)
And of course so as not to be accused of bias towards Microsoft, I ought to point out that they’re not the only people doing this :-). The DSM Forum is the industry organization for all companies involved, and the web pages are a good starting point for information and links to further reading, real world cases and related events. In addition to Microsoft’s pre-release DSL tools there are also others that have a longer history in this area, including ourselves at MetaCase with MetaEdit+, which has been used commercially for over ten years so is a tad more mature (if you’ll excuse the lack of modesty), and the Vanderbilt University GME research tool, which has also been around for years and is great for those who want to work on coding the next generation of such meta-tools.
If you want to play around with this to learn more, I think you’ll easily get furthest in a short time if you start with the MetaEdit+ 31-day evaluation version: in an hour you can build your own first modeling language. If you prefer MS tools, you can try it out for 90 minutes on-line here - installing it can be something of a long process, so that’s a good alternative.
comment at 29. November 2005