LinQ is often understood in terms of introducing a Domain Specific Language to work with data to C# and .Net in general. The fact is:it is not, and there is a considerable difference between LinQ syntax nature and a DSL. The problem is that DSL definition is blur enough to take anything interesting or cool under it!

Back to the beginning. When faced to doing software for a business that has some moving parts, a good practice is to make a generalization or an abstraction of the fixed part and express what is moving through some sort of configurations. That is in some respect what most of enterprise frameworks do when implementing the fixed part and letting you nicely only express what varies for your context. This way you are almost only concerned about your part since the framework itself is supposed to be steady and well abstracted and as a result implicit in your context. This experience hasn’t been nice enough when XML used to be involved. As someone that defends multi-paradigm design I could argue that XML can play a best fit in some contexts but this is out of the scope of this post. XML charset is not optimal for a lot of cases, so why not creating our own charset, and here comes DSLs.

Of course here I am only talking about the rise of DSLs in the enterprise and not the DSLs that FP been doing for ages, and I’ll get back to this one later.

Now using my favorite meta magical super dynamic language, I’d like to construct my customized embedded(or internal) domain specific language, and I’ll make my best to be English like/business like/ host language unlike to be really specific. And that is where my brains blocks! How can I get grasp of that thing, that ‘lil’ language? It might be a personal problem I have because of the nature of my profession as a consultant: I switch contexts often, and… well I can only hope that it is 1:designed, 2: designed considering practices, rules and well established… did I say patterns?

P for Patterns :

Another way of providing a solution to the problem of generality/variability is use a catalogue of non-formal generalization of recurring solutions to recurrent problems that provide a rich knowledge base of experience about tackling the kind of problem. Being of non-formal nature means that patterns aren’t provided as a framework but rather as a set of vocabulary and description that has its power in being adaptive  and flexible. In my own experience, what I found particularly interesting when working with code that uses patterns is the way they communicate intent. If I find “Strategy” postfix I start searching for the other parts of the patterns and then start building on that. Providing this shared vocabulary helps me getting operational quite fast and help me scan code faster in a more “visual” way.

The good thing also about patterns is that I am still playing under the rules of my preferred hosting language. I know very well doing two very important things with it: combining and abstraction. A programming language is a set primitives together with a way to combine them and a mechanism for abstraction. What about my lil DSL? well it depends… Did I make sure to break any host language logic in there? Did I fight the intuitive nice “.”? Or did I decide that to implement my DSL for scratch? A programming language without means of abstraction results inevitably in long flat copy/paste configurations files and please don’t argue with me about their maintainability. What is the problem? The problem is that I am not a programming language designer, and to create a programming language no matter how simple and small it is, you need to have programming language design skills, so do you?

Abstraction and Communication for People:

Something that really drew my attention in functional programming languages like Haskell is their solid formal abstractions used for expressing computational patterns. These abstractions are packaged into a framework, and have a clear identity defined by a set of rules and are useable whenever semantics match. Take monads for example. It doesn’t matter if you are working with state, a database, the web, the screen or a robitic arm: you just use the do notation. The do notation in Haskell is a convenient way of working with a sound abstraction that is THE MONAD. And all what you need to do for your framework is to verify that you satisfy the Monad laws and then you can safely use that syntax for it. This sharing of well identified semantics of the abstraction help developers reuse a well established mental model. LinQ syntax is the same, the dot “.” operator is the same, F#’s  “|>” operator is the same and examples are a lot. And I tend to prefer these kind of abstractions as they don’t sacrifice readability for developers for “readability” for users. I guess they have both the communicative property of Design Patterns yet they are formal and are offered as a concrete framework. Then of course you can name them DSLs, if you insist :) .

Now, if you are from the functional programming camp, and you have been doing embedded domain specific language using Lisp or Haskell for long time, then I apologize for wasting your time. I know you guys know best about doing abstractions, and you have lessons to tell if we’d only listen. And for those interested in some examples in EDSLs from the functional world, Paul Hudak goes through the design of a language for reactive animation in his book, Lennart Augustsson discussed EDSLs in Haskell on his blog.

Conclusion

Though an attractive option of abstraction, DSLs trade off readability for developers for “potential” readability of users. Haskell monads, F#’s “|>” operator and LinQ are examples of sound formal abstraction that can be offered as frameworks (code). These abstractions can save developers time being in a form of a recurring pattern that can be used in different contexts that match semantically, thus reusing developers’ understanding of the abstraction’s mental model.

One thing that annoys me though about most programming language books is how raw they often are.

  Probably constrained by time, authors and book teams seem to invest less on editorial stuff and teaching methodology and more on explaining details of technical and behind the scenes stuff. This often results in more manual-like books where you have a great wealth of material that you are not able to exploit anyway because of, well, lack of time. I am not sure I can invest time in reading 500+ manual of a new, or at least non mainstream, language that I might decide not to use by the end. 

I am not saying here that technical and behind the scene stuff does not matter, quite the opposite. I just think that manual, important it is, should not be the introduction to the programming language. A rather 100 to 200 pages brief book that is carefully edited to introduce paradigm, concepts, and strengths of the programming language is a more attractive choice , accompanied with a wordy but well organized reference book with examples and more in-depth explanation of language features. I guess this is a more pragmatic approach to programming language learning in the presence of WWW and Google search.

PS: I personally find Paul Hudak’s Haskell book a very good example of an enjoyable, brief, nicely and carefully edited programming language book. Also LUA’s programming book is nicely done.

Originally posted on http://www.infoq.com/news/2008/07/stream-oriented-ui

The mainstream approach to user interface has recently been questioned by Bryce Harrington who believes that “the prevailing UI paradigm today […] built around the notion of document authoring” is not appropriate for what most people use computers for. He argues in effect that knowledge workers more often deal with streams of information rather than traditional documents:

Really, what I mostly do today is stream management. And I suspect this is true for the vast majority of people. I don’t deal with writing documents, but with changes to documents. I put comments onto things. I slap patches onto things. I tweak the states of things. Once in a rare while I may author a completely new thingee, but even there I usually end up working with it as a stream of changes that I build up over time (and usually in collaboration with a few other people who stream changes to me).

Harrington believes that, with regard to this, many tools offered by today’s interfaces are irrelevant whereas they are not at all instrumental in helping users staying atop all the streams. What actually handles streams are cron jobs, “a stodgy old *server* tool” that non-technical people are not acknowledgeable about. To remedy to this, Harrington advocates for changing the underlying paradigm of UI:

Since the purpose of our desktop UI is to make our work easier and more efficient, then if today’s knowledge workers are, like me, more stream-oriented than document-oriented, then doesn’t it stand to reason that we ought to re-think our UI design to optimize it for making stream management easier and more efficient? How would such optimization be done? How would such a UI look and feel? What kinds of toolkits would be needed?

This post triggered a lot of reactions on the author’s blog and on reddit even though it was pointed out by many commentators that this idea is not really new. Many mentioned Lifestream project led in the mid-nineties at Yale University. Based on the conviction that today desktop paradigm is not the optimal way to organize information, Eric Freeman under the direction of David Gelernter developed novel software architecture:

Lifestreams is built on a simple storage metaphor — a time-ordered stream of documents combined with several powerful operators — that replaces many conventional computer constructs (such as named files, directories, and explicit storage) and in the process provides a unified framework that subsumes many separate desktop applications to accomplish and handle personal communication, scheduling, and search and retrieval tasks.

In the aftermath of this project, Steve G. Steinberg has analyzed the advantages of the approach based on temporal dimension that is ignored by most UI and absent from many alternative methods of organizing information rather focused on spatial, semantic or network aspects. First of all, Steinberg stresses that “unlike spatial and networked schemes, which require users to come up with their own, highly arbitrary classifications, and unlike semantic schemes that place the burden on the computer, chronological ordering is clearly defined and unarguable.” Moreover, this kind of UI facilitates information search because “instead of following links or guessing keywords, we can simply scroll back in time, using our memory for hints”, which in turn allows to rebuilt the context of the researched information. Finally, as highlighted by Steinberg, “chronological ordering underlies many types of information”, e.g. files, emails, URL of visited web pages, and this makes Lifestreams “incredibly general and flexible”.

Even though commercialization of the project’s results was not successful, many recent products mentioned by commentators are based on similar concepts: Mac OS X, services like LifeInLine and LifeBlob, Sugar interface by the One Laptop Per Child Foundation, Miro interface or upcoming Microsoft’s Live Mesh. It looks like the momentum is growing for a shift towards stream-oriented approach to UI. Other technologies and tools that may be instrumental, i.e. dataflow languages or functional reactive programming, are summed up in Bryce Harrington’s follow up to his own post and on a wiki set up by Rudd-O.

http://www.infoq.com/news/2008/03/dsls-are-not-natural-languages

There is a widespread opinion that a good DSL has to be English-like in order to be readable for non-programmers. Dave Thomas advocates against such approach asserting that DSL are not about getting as close as possible to natural languages. Moreover, he argues that having this as a guiding principle of DSL design can be rather detrimental. He also highlights what he believes is important in DSL design and provides some examples of successful DSL that do not necessarily reed like English.

According to Dave, DSL don’t need to be close to English or any other natural language because they targets a very specific category or users – domain experts – who actually don’t speak a natural language

Domain experts […] are speaking jargon, a specialized language that they’ve invented as a shorthand for communicating effectively with their peers. Jargon may use English words, but these words have been warped into having very different meanings—meanings that you only learn through experience in the field.

Hence, DSL should reflect this jargon and express the expertise of domain specialists in a concise way. Make for dependency management, Groovy builders for expressing data in code and Active record declaration for data modeling in Ruby are a few successful examples of such DSL that respond to domain experts needs without necessarily being English-like. Even though some statements in Active record declaration may look like English, e.g. has_many or belongs_to, they actually are not: “they are jargon from the world of modeling” and “they have a specific meaning in that context.”

Another important point raised by Dave is that, in his opinion, “domain experts” should not be understood as business users but rather as people who are writing specs. These people are programmers. They do not really need an English-like language. Dave actually believes that the notion of fluent interface is often misunderstood: “the fluency here is programmer fluency, not English fluency. It’s writing succinct, expressive code”.

Dave Thomas argues that not only isn’t it necessary trying to get closer to a natural language, but it can also be detrimental. Natural languages are imprecise. This makes their power in the real world but this cannot apply to programming. This is why, “whenever we try to create a DSL that looks like a natural language, we fall short”. However hard one tries, syntax tends to remain “very unEnglish like”. And this gap is rather confusing:

There’s a major cognitive dissonance—I have to take ideas expressed in a natural language (the problem), then map them into an artificial language (the AppleScript programming model), but then write something that is a kind of faux natural language.

To illustrate the possible confusion, Dave gives an example of piece code from a test written using the test/spec framework and analyses one expression:

@result.should.be.a.kind.of String

It reads like English. But it isn’t. The words are separated by periods, except the last two, where we have a space. As a programmer, I know why. But as a user, I worry about it. In the first example, we write @result.should.be.a.kind_of. Why not kind.of? If I want to test that floats are roughly equal, I’d have said @result.should.be.close value. Why not close.to value?

Trivial details, but it means that I can’t just write tests using my knowledge of English—I have to look things up. And if I have to do that, why not just use a language/API that is closer to the domain of specifications and testing?

It is true that English-like DSL may be more readable, but Dave argues that “the attempt to create a natural language feel in the DSL leads to all sorts of leaks in the abstraction”. It might add to readability of code but it would “be taking away from its writability” and “adding uncertainty and ambiguity”:

The second you find yourself writing

  def a
     self
  end

so that you can use "a" as a connector in

add.a.diary.entry.for("Lunch").for(August.10.at(3.pm))

you know you’ve crossed a line. This is not longer a DSL. It’s broken English.

One of commentators, Has, also believes that trying to make a language readable to non-programmers one risks to end up with a "read-only language”. He takes the example of AppleScript. To improve its readability, it was necessary to remove “most of the usual symbolic cues that describe a language’s semantics”. As a result, “the syntax effectively obfuscates, not clarifies, the language semantics”. If “it’s very easy to read an AppleScript and understand _what_ it does, it’s damnably hard to figure out exactly _how_ it does it”.

Has highlights another issue that may result from using an English-like DSL: users might assume that “because it _looks_ like English, it will also _behave_ like it” and “form all sorts of very strong associations and conclusions about its nature, which then have to be undone the long, hard way”. Hence, according to Has, English-like appearance “accidentally encourages unrealistic user assumptions”

If DSL readability and expressiveness are of interest for you, find more examples and comments on Dave’s blog post.

http://www.infoq.com/interviews/coplien-martin-tdd

Thanks Cope, thanks Bob.

Many efforts in today’s software community are targeted at bridging the gap between software professionals and business people. Several blogspace authors look at the issue from a slightly different perspective, highlighting the gap between developers and the software they are building.

According to Jeff Attwood, Amazon’s experience of regularly involving its developers with costumer service is a valuable means for improving quality and usability of software.  He believes indeed that “all too often, software developers are merely tourists in their own codebase” because they lack a basic understanding of software users, their problems and concerns. This is what he has earlier referred to as  “ivory tower software development”:

In the absence of any other compelling evidence, developers assume everyone else is a developer. [...] The more isolated the developers, the worse the resulting end product. It doesn’t help that most teams have a layer of business analysts who feel it is their job it to shield developers from users [...] It’s dangerous to create an environment where developers have no idea who the users are.

However, today’s industry is characterized, according to Abhijit Nadgouda, by hierarchy, existence of multiple layers and information shielding among those layers. He highlights that this simplifies management and makes business safer, but has rather negative implications on software quality:

We create a hierarchy in our projects, and every level shields the lower levels from some information. How many members of the software development team know value of the software they are working on, or its importance to the client’s business? How many of them even know about pieces other the one they are working on?

[...]

There seems to be a disconnect between better business and better software development. Which is why, I believe, most of us are doing good business, but we as an industry still stink at software development.

In his attempt to identify the reasons why “we still stink at software development”, Reg Braithwaite also raises this issue arguing that it might be wrong to divide up work on a project in a way “that the people with the least experience are protected from damaging the code”.

Architecture based on such approach tends to simplify developers work through abstaction. If this is pushed to the extreme, developers tasks are taken out of functional context to become purely technical, which can potentially create a gap between developers and the software they are working on.

What is your opinion? Is such a protectionist architecture an impediment for software usability? Can architecture with developers ignorant of the full picture be effective? Does it deliver sofware and value? 

James O. Coplien has recently argued that today’s industry is based on buzzwords and checklists. The use of some techniques and methodologies, TDD for instance, has become “a religious issue”. This prevents from inspecting possible tradeoffs and focusing on finding solutions that would be the most appropriate and the most cost-effective .

“For some reason, we have invented and are following religions […].” This is how James O. Coplien describes today’s industry which, he believes, is based on buzzwords and checklists rather than on thinking, inspection and efforts to find solutions that would be the most appropriate and the most cost-effective for a given project:

How much thought did you put into the tradeoffs of the last technique you brought into your organization: Ajax, TDD, On-Site Customer, or other buzzwords? Did you research its track record? Or did you go to the buzzword yellow pages?

James uses the example of the debate around TDD which has occurred in the blog space in the wake of Jaoo 2007 conference. When the discussion focuses on what approach to testing is the best (i.e. TDD or acceptance testing), it essentially addresses the wrong question. It obfuscates the real objective which is “to deliver what the customer wanted while optimizing quality” and to create value for the client. Testing is what we have on today’s checklist for quality but it is far from covering all the issues that actually affect quality:

Quality suffers if you misunderstand what the customer wanted, or if the code has internal interactions that are difficult to foresee, or if local code violates array boundaries or uses untethered pointers. It also suffers if the interface makes it possible for the user to commit errors, or if it takes too many keystrokes to enter a small amount of common information.

Focusing on quality requires considering every weapon in the arsenal:

That means using Use Cases (which are an Agile way of engaging customers […]) instead of XP-style user stories (so you understand feature interactions up-front), doing pair programming or desk-check code inspections, doing design by contract, driving the process with usability experts and doing good usability testing, and a host of other things.

Coplien emphasizes that it is impossible to do everything and that it is “a matter of cost effectiveness”. But being focused on techniques and driven by buzzwords we get stuck in practices that might not be optimal. The use of some techniques and methodologies, e.g. TDD, has become “a religious issue”, according to James:

We are told that you have to do TDD to be a professional […] and yet are given no reason to believe this, no substantiation, and no evidence. Just believe.

Coplien argues, for instance, that “integration and system testing have long been demonstrated to be the least efficient way of finding bugs”, that TDD “deteriorates the architecture”, and that “acceptance testing is orders of magnitude less efficient than good old-fashioned code inspections, extremely expensive, and comes too late to keep the design clean.” But he stresses how difficult it is to engage a challenging discussion on such “religious” topics because any criticism is met with a lot of emotion.

People have tied their Agile success to their introduction of TDD practices into their work. It’s the one thing they can do as individuals rather than as an enterprise.

Coplien advocates for focusing on quality rather than testing and, more generally speaking, on thinking rather than checklists. He highlights however that “sorting these things out requires a system view”, which often times lacks in today’s industry. He believes that “New Age Agile movements consistently move away from” systems engineering. According to Coplien, “one root of the problem lies in modern education, which has increased its focus on technique and reduced its focus on thinking.” Today’s students “fewer and fewer understand software history” and “have stronger memories […]on exactly when to use --v and when to use v-- than they recall anything about logical design.”

Close links which exist between academia and industry are also a part of the problem. Universities tend to adapt curricula to industry’s needs in order to facilitate students’ placement. Hence, it becomes difficult for academics to challenge predominate views, probably “as difficult as it was for Newton” to prevail “with new insights on the working of the universe”:

In fighting the broad misunderstandings that industry has chosen to adopt from selected interpretations of Agile, SOA, and programming language buzzwords, we face no less a religion, and no less threat to our cozy industry-financed academic positions, than he did.