LinQ is often understood in terms of introducing a Domain Specific Language to work with data to C# and .Net in general. The fact is:it is not, and there is a considerable difference between LinQ syntax nature and a DSL. The problem is that DSL definition is blur enough to take anything interesting or cool under it!

Back to the beginning. When faced to doing software for a business that has some moving parts, a good practice is to make a generalization or an abstraction of the fixed part and express what is moving through some sort of configurations. That is in some respect what most of enterprise frameworks do when implementing the fixed part and letting you nicely only express what varies for your context. This way you are almost only concerned about your part since the framework itself is supposed to be steady and well abstracted and as a result implicit in your context. This experience hasn’t been nice enough when XML used to be involved. As someone that defends multi-paradigm design I could argue that XML can play a best fit in some contexts but this is out of the scope of this post. XML charset is not optimal for a lot of cases, so why not creating our own charset, and here comes DSLs.

Of course here I am only talking about the rise of DSLs in the enterprise and not the DSLs that FP been doing for ages, and I’ll get back to this one later.

Now using my favorite meta magical super dynamic language, I’d like to construct my customized embedded(or internal) domain specific language, and I’ll make my best to be English like/business like/ host language unlike to be really specific. And that is where my brains blocks! How can I get grasp of that thing, that ‘lil’ language? It might be a personal problem I have because of the nature of my profession as a consultant: I switch contexts often, and… well I can only hope that it is 1:designed, 2: designed considering practices, rules and well established… did I say patterns?

P for Patterns :

Another way of providing a solution to the problem of generality/variability is use a catalogue of non-formal generalization of recurring solutions to recurrent problems that provide a rich knowledge base of experience about tackling the kind of problem. Being of non-formal nature means that patterns aren’t provided as a framework but rather as a set of vocabulary and description that has its power in being adaptive  and flexible. In my own experience, what I found particularly interesting when working with code that uses patterns is the way they communicate intent. If I find “Strategy” postfix I start searching for the other parts of the patterns and then start building on that. Providing this shared vocabulary helps me getting operational quite fast and help me scan code faster in a more “visual” way.

The good thing also about patterns is that I am still playing under the rules of my preferred hosting language. I know very well doing two very important things with it: combining and abstraction. A programming language is a set primitives together with a way to combine them and a mechanism for abstraction. What about my lil DSL? well it depends… Did I make sure to break any host language logic in there? Did I fight the intuitive nice “.”? Or did I decide that to implement my DSL for scratch? A programming language without means of abstraction results inevitably in long flat copy/paste configurations files and please don’t argue with me about their maintainability. What is the problem? The problem is that I am not a programming language designer, and to create a programming language no matter how simple and small it is, you need to have programming language design skills, so do you?

Abstraction and Communication for People:

Something that really drew my attention in functional programming languages like Haskell is their solid formal abstractions used for expressing computational patterns. These abstractions are packaged into a framework, and have a clear identity defined by a set of rules and are useable whenever semantics match. Take monads for example. It doesn’t matter if you are working with state, a database, the web, the screen or a robitic arm: you just use the do notation. The do notation in Haskell is a convenient way of working with a sound abstraction that is THE MONAD. And all what you need to do for your framework is to verify that you satisfy the Monad laws and then you can safely use that syntax for it. This sharing of well identified semantics of the abstraction help developers reuse a well established mental model. LinQ syntax is the same, the dot “.” operator is the same, F#’s  “|>” operator is the same and examples are a lot. And I tend to prefer these kind of abstractions as they don’t sacrifice readability for developers for “readability” for users. I guess they have both the communicative property of Design Patterns yet they are formal and are offered as a concrete framework. Then of course you can name them DSLs, if you insist :) .

Now, if you are from the functional programming camp, and you have been doing embedded domain specific language using Lisp or Haskell for long time, then I apologize for wasting your time. I know you guys know best about doing abstractions, and you have lessons to tell if we’d only listen. And for those interested in some examples in EDSLs from the functional world, Paul Hudak goes through the design of a language for reactive animation in his book, Lennart Augustsson discussed EDSLs in Haskell on his blog.

Conclusion

Though an attractive option of abstraction, DSLs trade off readability for developers for “potential” readability of users. Haskell monads, F#’s “|>” operator and LinQ are examples of sound formal abstraction that can be offered as frameworks (code). These abstractions can save developers time being in a form of a recurring pattern that can be used in different contexts that match semantically, thus reusing developers’ understanding of the abstraction’s mental model.

One thing that annoys me though about most programming language books is how raw they often are.

  Probably constrained by time, authors and book teams seem to invest less on editorial stuff and teaching methodology and more on explaining details of technical and behind the scenes stuff. This often results in more manual-like books where you have a great wealth of material that you are not able to exploit anyway because of, well, lack of time. I am not sure I can invest time in reading 500+ manual of a new, or at least non mainstream, language that I might decide not to use by the end. 

I am not saying here that technical and behind the scene stuff does not matter, quite the opposite. I just think that manual, important it is, should not be the introduction to the programming language. A rather 100 to 200 pages brief book that is carefully edited to introduce paradigm, concepts, and strengths of the programming language is a more attractive choice , accompanied with a wordy but well organized reference book with examples and more in-depth explanation of language features. I guess this is a more pragmatic approach to programming language learning in the presence of WWW and Google search.

PS: I personally find Paul Hudak’s Haskell book a very good example of an enjoyable, brief, nicely and carefully edited programming language book. Also LUA’s programming book is nicely done.

The first time I got introduced to mutation, I had a bad feeling. How can x:=x+1 be logical at all?

It felt so unnatural, scary, or maybe like a hack. Then, because of society constrains, I got to forget the bad feeling about that. Well, my mind started to tolerate with counter-logic logic. And that is how I became an enterprise developer. I am not sure how proud I am with this title anyway. I feel that tolerating and accepting the counter-logic logic is one, and most important one, of the prerequisite to this title.

Nowadays, I am working on a project that has to process a huge number of data to do calculations on it and produce results that make sense to the user. Performance is a constraint. And I decided this time to do it the right way, the way that I believe is right. The logical way.

I decided to the calculations with no mutable state. I wanted immutability to be my guarantee to safety. Mutability breaks logic, and so produces bugs. But I couldn’t imagine how hard this choice can be. You tend all the time to feel that it is easier to introduce a mutable object, mutable flags, temporary mutable containers inside loops. Imperative programming is so tempting. Just like drugs. It feels so strange and weird in the first time, but then, it becomes a habit, a bad one, and you can’t help but to break into it. Mutation and imperative programming is the simplest thing that work in some way.

Today, I am happy I almost finished the whole calculation logic I had to model ( and it is quite a complex one). And I am proud that I could resist introducing mutability in the algorithm. Yet, I did some experiments along the way to introduce just a bit of state, kind of just what I need to introduce faster one kind of calculation. This very small mutation just breaks it all. It just breaks the whole modularity of the algorithm, and just makes the whole calculation much less generic, and more importantly less maintainable and evolutive. Such a small mutation, like introducing a mutable flag, or a global variable, just breaks the whole logic of my calculation, and drives me far away from a good modular calculation, with a good design based on plain old principles of separation of concerns. It just makes impossible to go further with the implementation! It is like a cancer of code…

Now, after the discipline and the resistance to a mutable state, we are so happy with the resulting code that yet needs polishing, makes it very easy to parameter more calculations and so add more desired functionality to the program. And just for fun, we compared it to the same calculation done with imperative programming. It is much more concise, intention concealing, and as a result, surprisingly we paid no performance cost for a much more declarative code.

I owe a lot to Erik Meijer, Paul Hudak, Linq, and Haskell :)

http://www.infoq.com/news/2008/02/idioms-as-signals

Making code more readable and more explicit? Reg Braithwaite believes that correctly choosing programming idioms can be instrumental for achieving that goal. He stresses that there are often several ways to get to the desired outcome and the choice should not be random. One should prefer the tool that signals in a most obvious way the intent of the piece of code that is being built. Syntax choices, for instance, can bear useful information about code’s intent:

Consider blocks in Ruby. Some people use do … end when the block needs multiple lines and { … } when it fits on one line. A popular (AFAIK) and superior idea is to use do..end and {…} to disambiguate between blocks that are executed chiefly for their side effects and blocks that are executed for their return values:

foo.map { |x|
# I care about the result
}
foo.each do |x|
# I care about the side effects
end

The computer doesn’t care, of course, but it signals an extra piece of information, the fact that you are chiefly interested in side effects in one case and chiefly interested in the result in another case.

Braithwaite believes that this approach is not limited to syntax idioms but can scale up to paradigms. In some languages, e.g. Ruby, “there are several entirely different philosophies the language supports.” This can be used as a means to declare intent by borrowing idioms from paradigms that correspond the best to what one is trying to achieve.

By making functional things look functional, you clearly signal that you are performing a calculation strictly for the result. Using imperative syntax like for and while should be reserved for the times you need to mutate variables or cause other side effects. […] Likewise, OO is a great paradigm—when you are modelling entities in the real world or when you need long-term persistent state. Languages like Ruby force you to use objects behind the scenes, your code shouldn’t look OO unless you are trying to signal your colleagues that there are entities involved.

Even though it is obvious that, sometimes, the choice of paradigm may be imposed by performance requirements, using this kind of conventions, when possible, allows providing additional information to programmers thus making code more readable and more expressive.

Some argue that using undocumented and unvalidated conventions would only result in more confusion. Reg argues, however, that any project involves programming conventions of some kind, e.g “using Bang! method names to indicate state changes (sort vs. sort!)”, and that nothing prevents teams from documenting established conventions, if necessary.

What do you think about using idioms as signals and means to declare intent?

Recently, I’ve been playing a lot with Meta programming System of JetBrains. I feel that it has a lot in common with my idea of Paradigm Oriented Programming. In the way that it defines the language by links, adjectives and things (i.e. dimensions) that form together an abstraction as a language for a certain domain.

The package comes with Java already implemented, that can be quite handy, as it makes it easy for us to embed languages and concepts inside Java. But there is something that bothers me here; does Java (or even C#) work as an IL, a basic flexible language that other designed DSL will be interpreted to? Actually I strongly doubt that. First of all Java syntax comes already with a lot of constrains, static typing, curly brackets and a lot of other imitations.

Another thing is that Java itself is already an implementation of an abstraction. That is OOP. And fitting everything into OOP will force our DSL to respect that style of programming and to be constrained by it. Java is so static, and neither flexible not dynamic enough to have DSLs built on it.

Fortunately JetBrains’s MPS is not exclusive for java. It rather can be used with any other programming language. Because what it does is to build a coherent structure of text with its grammar in a syntax tree. Then it allows text to be generated out of properties of the syntax.

I thought a lot about a language that can be flexible enough, so that we can build abstractions that can be easily interpreted and implemented into that language’s syntax. I guess Lua is a good candidate. I believe that the way it is built permits a lot of dynamism for implementing meta-systems and making them co-exist in a paradigm oriented programming (See OOP implementation in Lua).

I still see some improvements still can be added to it. But for now I see it a perfect fit as an intermediate language for my POP.

Looking at paintings, they are often composed of a figure and a ground. The figure is usually what is more visible for the eye. Often the ground is less important as recognizing the shape is thought to be satisfied by recognizing the figure.

Understanding the figure can be simpler due to its more concrete and explicit nature, but the vision can not be complete without the ground. I even guess that thinking about the ground as a result of drawing the figure is quite naive. I guess that the ground defines rules of the environment, and is present implicitly in the definition of the shape. The fact that these rules are implicit makes them harder to capture. And easily ignored.

If we think about Object Oriented Programming as a figure, then what is the ground? The ground should be the environment that allowed us to define this meta-System.

Thinking about OOP as a meta system, allows us to capture the meta-meta-system. How this meta-meta-system can be defined?

Well, If we look at objects, they are special things that fall in a category ,things that fall in this category have special kind of relationship  called inheritance, this relationship is governed by rules of polymorphism. Also we can add adjectives to tag some special behavior on the methods and properties of these things.

So far, I got things that fall in a category, governed by rules applied to relationships . moreover methods and properties of these things can have adjectives that can add rules on their use.

Now, with such a meta-meta-system I can define another meta-system than OOP, say a hierarchical one, or maybe a system of connectors. These several meta-systems can co-exist exactly as several categories of things co-exist in real world.

 A question that raises evidently here, is How do they co-exist? How do they inter-connect? The simple answer is : messages… The problem of the integration of different types of paradigm had always existed, and had always been pushed out to frameworks to handle it. I believe that these frameworks of integration are themselves not well integrated in the language. Maybe including them in the language can make more natural?

Anyway my whole idea is a paradigm oriented programming language, where we define dimensions, that we can use to define meta-systems. I see the idea is pretty faisable, might be more than a (things, category, relationship, rules, adjectives) even if I could imagine a lot of meta-systems built only on this 5 dimensioned programming language…

All what I am about in this blog post is ideas, far from being mature.It is just a stream of ideas that I felt to persist somewhere. And here I did :)

I just streamed some fresh ideas I got on my Domain Specific Wiki about a term I call  “Thing Oriented Programming”.
Later I ll try to gather them in a more readable essay that will be available both here and there.