LinQ is often understood in terms of introducing a Domain Specific Language to work with data to C# and .Net in general. The fact is:it is not, and there is a considerable difference between LinQ syntax nature and a DSL. The problem is that DSL definition is blur enough to take anything interesting or cool under it!

Back to the beginning. When faced to doing software for a business that has some moving parts, a good practice is to make a generalization or an abstraction of the fixed part and express what is moving through some sort of configurations. That is in some respect what most of enterprise frameworks do when implementing the fixed part and letting you nicely only express what varies for your context. This way you are almost only concerned about your part since the framework itself is supposed to be steady and well abstracted and as a result implicit in your context. This experience hasn’t been nice enough when XML used to be involved. As someone that defends multi-paradigm design I could argue that XML can play a best fit in some contexts but this is out of the scope of this post. XML charset is not optimal for a lot of cases, so why not creating our own charset, and here comes DSLs.

Of course here I am only talking about the rise of DSLs in the enterprise and not the DSLs that FP been doing for ages, and I’ll get back to this one later.

Now using my favorite meta magical super dynamic language, I’d like to construct my customized embedded(or internal) domain specific language, and I’ll make my best to be English like/business like/ host language unlike to be really specific. And that is where my brains blocks! How can I get grasp of that thing, that ‘lil’ language? It might be a personal problem I have because of the nature of my profession as a consultant: I switch contexts often, and… well I can only hope that it is 1:designed, 2: designed considering practices, rules and well established… did I say patterns?

P for Patterns :

Another way of providing a solution to the problem of generality/variability is use a catalogue of non-formal generalization of recurring solutions to recurrent problems that provide a rich knowledge base of experience about tackling the kind of problem. Being of non-formal nature means that patterns aren’t provided as a framework but rather as a set of vocabulary and description that has its power in being adaptive  and flexible. In my own experience, what I found particularly interesting when working with code that uses patterns is the way they communicate intent. If I find “Strategy” postfix I start searching for the other parts of the patterns and then start building on that. Providing this shared vocabulary helps me getting operational quite fast and help me scan code faster in a more “visual” way.

The good thing also about patterns is that I am still playing under the rules of my preferred hosting language. I know very well doing two very important things with it: combining and abstraction. A programming language is a set primitives together with a way to combine them and a mechanism for abstraction. What about my lil DSL? well it depends… Did I make sure to break any host language logic in there? Did I fight the intuitive nice “.”? Or did I decide that to implement my DSL for scratch? A programming language without means of abstraction results inevitably in long flat copy/paste configurations files and please don’t argue with me about their maintainability. What is the problem? The problem is that I am not a programming language designer, and to create a programming language no matter how simple and small it is, you need to have programming language design skills, so do you?

Abstraction and Communication for People:

Something that really drew my attention in functional programming languages like Haskell is their solid formal abstractions used for expressing computational patterns. These abstractions are packaged into a framework, and have a clear identity defined by a set of rules and are useable whenever semantics match. Take monads for example. It doesn’t matter if you are working with state, a database, the web, the screen or a robitic arm: you just use the do notation. The do notation in Haskell is a convenient way of working with a sound abstraction that is THE MONAD. And all what you need to do for your framework is to verify that you satisfy the Monad laws and then you can safely use that syntax for it. This sharing of well identified semantics of the abstraction help developers reuse a well established mental model. LinQ syntax is the same, the dot “.” operator is the same, F#’s  “|>” operator is the same and examples are a lot. And I tend to prefer these kind of abstractions as they don’t sacrifice readability for developers for “readability” for users. I guess they have both the communicative property of Design Patterns yet they are formal and are offered as a concrete framework. Then of course you can name them DSLs, if you insist :) .

Now, if you are from the functional programming camp, and you have been doing embedded domain specific language using Lisp or Haskell for long time, then I apologize for wasting your time. I know you guys know best about doing abstractions, and you have lessons to tell if we’d only listen. And for those interested in some examples in EDSLs from the functional world, Paul Hudak goes through the design of a language for reactive animation in his book, Lennart Augustsson discussed EDSLs in Haskell on his blog.

Conclusion

Though an attractive option of abstraction, DSLs trade off readability for developers for “potential” readability of users. Haskell monads, F#’s “|>” operator and LinQ are examples of sound formal abstraction that can be offered as frameworks (code). These abstractions can save developers time being in a form of a recurring pattern that can be used in different contexts that match semantically, thus reusing developers’ understanding of the abstraction’s mental model.

http://www.infoq.com/interviews/LINQ-Erik-Meijer

Originally posted on http://www.infoq.com/news/2008/12/external-dsl-advantages-pitfalls

Given the growing interest in Domain Specific Languages (DSLs), Michael Feathers provides some reflections on external DSLs, their advantages and pitfalls as well as possible success and failure factors. He stresses that while external DSLs “give you a freer hand » and allow you creating « a complete syntax and tune it directly to a particular domain”, they represent an important commitment for companies using them. And this “commitment doesn’t end when the initial implementation is done”. 

Because of the specificity of external DSLs, they may induce significant maintenance costs since there should be in-house expertise for adapting DSL to occurring changes. For the same reason, the use of external DSLs complicates hiring and, more specifically, hiring of domain experts:

Let’s face it, if you are looking for a job what do you want to have on your resume?  A language used by precisely one company or one that is known or used throughout the industry? […]

One very public example is Erlang.  People are paying a lot of attention to Erlang today, but we shouldn’t forget that it was dropped by Ericsson in part because they were concerned with their ability to hire and train developers.  Many people know the story of Erlang, but what they don’t realize is just how common this scenario has been in the industry.  I get to see a lot of legacy code in my work, and a good amount of it is in homegrown languages that companies have trouble supporting.

So, Michael Feathers argues that, with regard to external DSLs, “success is a function of far more than the technology”. Their potential can only be successfully exploited if company objectively assesses the needed commitment and can assume it “even under adverse business conditions”. In addition to that, Feathers outlines a couple of ideas about how to reduce potential costs of external DSLs:

1. If you are creating an external DSL, consider open sourcing it or, at least, consider forming some sort of an industry group around it.  This partially externalizes the cost of maintenance and avoids hiring trouble. 
2. If you can’t or won’t open your DSL(s) to the outside world, make sure that your developers are not expected to use them exclusively.  If you don’t, you can expect to pay higher wages for developers of equivalent skill than you would otherwise.  In negotiations, they will factor in the cost of working with little known technologies.

Considering embedded DSLs could be another alternative since, according to the author, they are easy to construct and can yield many of the same advantages as external DSLs. 

Having worked with several Object-Relational mapping frameworks in the last few years, I got to a point where I couldn’t justify their complexity in my project. We often talk about the mismatch between the database and the object worlds, and that is where ORMs are often stated and referenced for “bridging the gap”!

Well I prefer to call it lifting the gap, or highering the gap, to have it now between DAOs and the rest of the code than having it between database and code.But I wouldn’t call this in any way reducing the gap.

First thing to mention here, is that data structures are not evil. We often deal with XML data structure but we absolutely refuse to deal with relational data structures. The problem might be related to time, XML came with a whole arsenal of managed APIs that represent its model in a way that makes it more comfortable for you to manipulate these documents. All together with transformation APIs that made it a lot easier to fill your object model out of an XML. Relational data structure on the other hand did not have the luxury of standard APIs and representation models in mainstream languages, and that made dealing with a database request result so arbitrary (Datasets existed in .Net but they have been cursed heavily and has been forced out of the cool toolbox, do not know yet why!) . 

In some way, I guess that the “mismatch” between Relational data structures and objects exists much similarly between XML and objects, however we didn’t think for so long of a doing a mapping between XML and object OXM? (no we are starting to do so, writing some XML to map XML to objects!).

With the experience I had with mapping technologies, I can say that they do not solve the problem for me. They merely add yet another useless level of abstraction where your domain types look nothing like relational data structures, but yet nothing like your target domain model! They have gone a half way, and now its your turn to poison your application with a lot of absurd ORM code trying to stretch this bridge to reduce the gap…

Geeks love talking about intrusivty of an ORMs and compare level of intrusivty in different terms. Intrusivty is much more than the lack of an interface or methods to implement in your domain objects. I guess I am ok to deal with that kind of intrusivity. However all ORMs I tried do not give me the opportunity to express my domain model in the way I would like to. I have to deal with notions of Lazy Loading, Attached/Detached entities, Entity sets, ids, back fields and a lot of other things that I don’t want to appear in my model and that make the resulting code so ugly and complex that I don’t wish to visit soon.

As Bob Martin said before, we are missing the opportunity to do oop, I would enforce and say we are missing the opportunity to model at all. And I guess we have gone too far with ORMs and it is time to go back to the basics, in the same way we have done after the EJB great abstraction.

It is very contradictory, while we are so excited about DSLs and the goodness they can potentially bring to enterprise development that results in a declarative semanticful style of programming, we can’t help but to engage all of our means in trying to bury SQL (and effective, semanticful, declarative powerful domain specific language) behind a dumb API to “abstract database access”! I guess we got to chose one strategy, either we keep SQL and we  keep marketing DSLs or we hope for a new advanced GPL compiler that will hide all of this in the same way modern languages did with garbage collection, concurrency and synchronization.

Since I am not yet a compiler writer, and I aint got around an effective way of integrating database in mainstream languages (there is actually a very productive in memory database on Haskell that worth trying), I decide to keep the DSL, and to work with them without the need to hide them, but rather to get a better tooling to help converting between difference data structures.

I guess you’ve guessed it by now. For me LinQ is the mid-term solution for the data structure mismatch. No not LinQ to sql, nor Ado.Net entities (they are helpful and we might use them but lets focus on the solution). LinQ is a very productive tool that halps effectively transform anything to anything in few lines of code. I mean that I can still use powerful SQL to get data effeciently and then convert them with one line of LinQ into the destination data structure. No abstraction leak, less complexity: Use the right tool where it applies.

So what I would actually do is to use a framework that will execute my SQL or SQL like query on the database and get me the records in much the same way they are in the database,Much in the same way Active Record gets you records as they are in the database. No useless efforts from the framework to bend the data to make it look untruthfully like the destination model.  I guess I know better my domain model, and with this better tooling I can get it done with no problem.

This actually reminds me of a good definition I read (I can’t recall where) of these design patterns: Resolutions to programming language limitations implemented using the programming language available syntax. When one learns a programming language, he learns to express what he wants to express using a very limited set of expressions of the language. The problem with this is that his imagination in this environment gets boxed! So what are GoF design patterns?

Most of GoF design patterns are inspired from Smaltalk, so what is so special about Smaltalk? Closures! It turns out that Java and C++ didn’t have closures and higher order functions. Some of the Design Patterns came to rescue and provide "alternatives" to these powerful constructs or workarounds  to resolve these limitations. Such an approach requires a kind of object gymnastics and results in an explosion of classes!

I am now on a project where I use C# 3.5 features, and guess what? Suddenly I use much less strategies, much less templates. I write much less code yet being as or even more flexible. Powerful lambda expressions!

Update: DSLs also come from fp paradigm!

http://www.infoq.com/news/2008/03/dsls-are-not-natural-languages

There is a widespread opinion that a good DSL has to be English-like in order to be readable for non-programmers. Dave Thomas advocates against such approach asserting that DSL are not about getting as close as possible to natural languages. Moreover, he argues that having this as a guiding principle of DSL design can be rather detrimental. He also highlights what he believes is important in DSL design and provides some examples of successful DSL that do not necessarily reed like English.

According to Dave, DSL don’t need to be close to English or any other natural language because they targets a very specific category or users – domain experts – who actually don’t speak a natural language

Domain experts […] are speaking jargon, a specialized language that they’ve invented as a shorthand for communicating effectively with their peers. Jargon may use English words, but these words have been warped into having very different meanings—meanings that you only learn through experience in the field.

Hence, DSL should reflect this jargon and express the expertise of domain specialists in a concise way. Make for dependency management, Groovy builders for expressing data in code and Active record declaration for data modeling in Ruby are a few successful examples of such DSL that respond to domain experts needs without necessarily being English-like. Even though some statements in Active record declaration may look like English, e.g. has_many or belongs_to, they actually are not: “they are jargon from the world of modeling” and “they have a specific meaning in that context.”

Another important point raised by Dave is that, in his opinion, “domain experts” should not be understood as business users but rather as people who are writing specs. These people are programmers. They do not really need an English-like language. Dave actually believes that the notion of fluent interface is often misunderstood: “the fluency here is programmer fluency, not English fluency. It’s writing succinct, expressive code”.

Dave Thomas argues that not only isn’t it necessary trying to get closer to a natural language, but it can also be detrimental. Natural languages are imprecise. This makes their power in the real world but this cannot apply to programming. This is why, “whenever we try to create a DSL that looks like a natural language, we fall short”. However hard one tries, syntax tends to remain “very unEnglish like”. And this gap is rather confusing:

There’s a major cognitive dissonance—I have to take ideas expressed in a natural language (the problem), then map them into an artificial language (the AppleScript programming model), but then write something that is a kind of faux natural language.

To illustrate the possible confusion, Dave gives an example of piece code from a test written using the test/spec framework and analyses one expression:

@result.should.be.a.kind.of String

It reads like English. But it isn’t. The words are separated by periods, except the last two, where we have a space. As a programmer, I know why. But as a user, I worry about it. In the first example, we write @result.should.be.a.kind_of. Why not kind.of? If I want to test that floats are roughly equal, I’d have said @result.should.be.close value. Why not close.to value?

Trivial details, but it means that I can’t just write tests using my knowledge of English—I have to look things up. And if I have to do that, why not just use a language/API that is closer to the domain of specifications and testing?

It is true that English-like DSL may be more readable, but Dave argues that “the attempt to create a natural language feel in the DSL leads to all sorts of leaks in the abstraction”. It might add to readability of code but it would “be taking away from its writability” and “adding uncertainty and ambiguity”:

The second you find yourself writing

  def a
     self
  end

so that you can use "a" as a connector in

add.a.diary.entry.for("Lunch").for(August.10.at(3.pm))

you know you’ve crossed a line. This is not longer a DSL. It’s broken English.

One of commentators, Has, also believes that trying to make a language readable to non-programmers one risks to end up with a "read-only language”. He takes the example of AppleScript. To improve its readability, it was necessary to remove “most of the usual symbolic cues that describe a language’s semantics”. As a result, “the syntax effectively obfuscates, not clarifies, the language semantics”. If “it’s very easy to read an AppleScript and understand _what_ it does, it’s damnably hard to figure out exactly _how_ it does it”.

Has highlights another issue that may result from using an English-like DSL: users might assume that “because it _looks_ like English, it will also _behave_ like it” and “form all sorts of very strong associations and conclusions about its nature, which then have to be undone the long, hard way”. Hence, according to Has, English-like appearance “accidentally encourages unrealistic user assumptions”

If DSL readability and expressiveness are of interest for you, find more examples and comments on Dave’s blog post.

http://www.infoq.com/news/2008/01/narrative-vs-symbolic-code

Achieving readability and expressiveness by writing English-like code is one of the trends on the rise in today’s industry. This is particularly true for DSL and BDD communities, but many developers working in Java, C# or Ruby also try to make their code look as close to English as possible.

In his article “The Narrative Urge”, Michael Feathers challenges the idea that “code becomes better as it approaches English”. Expressiveness and readability help improving maintainability and bridging the gap between programmers and business, as it was highlighted by Martin Fowler and Neal Ford in their presentation about DSLs. Michael Feathers does not question that fact. He emphasizes, however, that “natural language should not be the only direction we explore when we look for expressivity”.

One should not neglect the fact that there are different ways of understanding the text. Feathers compares narrative and symbolic modes:

In the narrative mode, we are approach text like prose and read through it the way we would read through English or any other natural language – it tickles the verbal centers of our brain. The symbolic mode is more visual.

Hence, expressiveness can also be yielded by the symbolic approach and Feathers argues that in some circumstances it may be more appropriate than the English-esque one. He outlines a number of trade-offs that might impact the choice between these approaches.

Feathers considers narrative mode to be more appropriate for user facing tools rather than for “code written by and for developers”. More importantly, however, the choice depends on the domain specificities. First of all, according to Feathers, it may depend on how “settled definitions and nomenclature” even though criteria are not that clear:

On the one hand, if you are working in a domain like dates or scientific units, with settled definitions and nomenclature, it can pay to use a natural language style in your API. When definitions are settled, there is less of a chance that the machinery of a DSL will have to be refactored as much over time. On the other hand, once definitions are settled and well understood, you can eliminate a lot of cruft by moving toward a symbolic approach.

Feathers also highlights that in some domains, it is not easy to give a verbal expression for a task. In this case, symbolic alternative can “make code comprehension a breeze” being “more precise than what we are used to with natural language”. Feathers further develops this idea in his other post about trade-offs of narrative vs. symbolic programming . He gives the example of the symbolic approach in an embedded DSL for music composition:

funkGroove

        = let p1 = perc LowTom qn

              p2 = perc AcousticSnare en

          in Tempo 3 (Instr Percussion (cut 8 (repeatM

                  ((p1 :+: qnr :+: p2 :+: qnr :+: p2 :+:

                    p1 :+: p1 :+: qnr :+: p2 :+: enr)

                   :=: roll en (perc ClosedHiHat 2))

             )))

According to Feathers, the symbolic mode works well for this specific domain because “people don’t talk about how one note follows another as much as they listen for notes or see them on a page.” More generally speaking, he argues that the symbolic approach is more appropriate for programs that are about structure rather than about semantics for which narrative mode more suited. 

Hence, a certain number of elements should be taken into consideration before chosing English-like code as a means for improving readability and expressiveness. What does your experience tell you about the expressive power of symbolic approach and different trade-offs presented by Michael Feathers?