LinQ is often understood in terms of introducing a Domain Specific Language to work with data to C# and .Net in general. The fact is:it is not, and there is a considerable difference between LinQ syntax nature and a DSL. The problem is that DSL definition is blur enough to take anything interesting or cool under it!

Back to the beginning. When faced to doing software for a business that has some moving parts, a good practice is to make a generalization or an abstraction of the fixed part and express what is moving through some sort of configurations. That is in some respect what most of enterprise frameworks do when implementing the fixed part and letting you nicely only express what varies for your context. This way you are almost only concerned about your part since the framework itself is supposed to be steady and well abstracted and as a result implicit in your context. This experience hasn’t been nice enough when XML used to be involved. As someone that defends multi-paradigm design I could argue that XML can play a best fit in some contexts but this is out of the scope of this post. XML charset is not optimal for a lot of cases, so why not creating our own charset, and here comes DSLs.

Of course here I am only talking about the rise of DSLs in the enterprise and not the DSLs that FP been doing for ages, and I’ll get back to this one later.

Now using my favorite meta magical super dynamic language, I’d like to construct my customized embedded(or internal) domain specific language, and I’ll make my best to be English like/business like/ host language unlike to be really specific. And that is where my brains blocks! How can I get grasp of that thing, that ‘lil’ language? It might be a personal problem I have because of the nature of my profession as a consultant: I switch contexts often, and… well I can only hope that it is 1:designed, 2: designed considering practices, rules and well established… did I say patterns?

P for Patterns :

Another way of providing a solution to the problem of generality/variability is use a catalogue of non-formal generalization of recurring solutions to recurrent problems that provide a rich knowledge base of experience about tackling the kind of problem. Being of non-formal nature means that patterns aren’t provided as a framework but rather as a set of vocabulary and description that has its power in being adaptive  and flexible. In my own experience, what I found particularly interesting when working with code that uses patterns is the way they communicate intent. If I find “Strategy” postfix I start searching for the other parts of the patterns and then start building on that. Providing this shared vocabulary helps me getting operational quite fast and help me scan code faster in a more “visual” way.

The good thing also about patterns is that I am still playing under the rules of my preferred hosting language. I know very well doing two very important things with it: combining and abstraction. A programming language is a set primitives together with a way to combine them and a mechanism for abstraction. What about my lil DSL? well it depends… Did I make sure to break any host language logic in there? Did I fight the intuitive nice “.”? Or did I decide that to implement my DSL for scratch? A programming language without means of abstraction results inevitably in long flat copy/paste configurations files and please don’t argue with me about their maintainability. What is the problem? The problem is that I am not a programming language designer, and to create a programming language no matter how simple and small it is, you need to have programming language design skills, so do you?

Abstraction and Communication for People:

Something that really drew my attention in functional programming languages like Haskell is their solid formal abstractions used for expressing computational patterns. These abstractions are packaged into a framework, and have a clear identity defined by a set of rules and are useable whenever semantics match. Take monads for example. It doesn’t matter if you are working with state, a database, the web, the screen or a robitic arm: you just use the do notation. The do notation in Haskell is a convenient way of working with a sound abstraction that is THE MONAD. And all what you need to do for your framework is to verify that you satisfy the Monad laws and then you can safely use that syntax for it. This sharing of well identified semantics of the abstraction help developers reuse a well established mental model. LinQ syntax is the same, the dot “.” operator is the same, F#’s  “|>” operator is the same and examples are a lot. And I tend to prefer these kind of abstractions as they don’t sacrifice readability for developers for “readability” for users. I guess they have both the communicative property of Design Patterns yet they are formal and are offered as a concrete framework. Then of course you can name them DSLs, if you insist :) .

Now, if you are from the functional programming camp, and you have been doing embedded domain specific language using Lisp or Haskell for long time, then I apologize for wasting your time. I know you guys know best about doing abstractions, and you have lessons to tell if we’d only listen. And for those interested in some examples in EDSLs from the functional world, Paul Hudak goes through the design of a language for reactive animation in his book, Lennart Augustsson discussed EDSLs in Haskell on his blog.

Conclusion

Though an attractive option of abstraction, DSLs trade off readability for developers for “potential” readability of users. Haskell monads, F#’s “|>” operator and LinQ are examples of sound formal abstraction that can be offered as frameworks (code). These abstractions can save developers time being in a form of a recurring pattern that can be used in different contexts that match semantically, thus reusing developers’ understanding of the abstraction’s mental model.

Originally posted on http://www.infoq.com/news/2008/11/oop-vs-fp-4-code-structure

Programming languages that offer more power and flexibility have been lately gaining momentum. Johnatan Tang highlights, however, the flexibility vs. productivity tradeoff due, amongst other things, to the fact that traditional object orientation makes organizing programs easier and more straightforward.

He argues that in languages with single dispatch, “given a parameter or other value, you know exactly what you can do with it”. Whereas in statically typed languages code is more revealing about values involved, in structurally typed languages it is more general, but still gives an immediate idea what operations are available on a given value. And for both, statically and structurally typed languages, it is pretty clear what to do if you want to perform an operation that is not supported by a given object:

You write a method that performs that operation. This is problematic if you don’t control that class, which is why Ruby and C# have moved to open classes. […]

And it’s usually obvious where this method should go: on the class that you need to manipulate.

In multi-dispatch languages, it is much less obvious where a method should be written and “which parameter of the new method might take [a given] value”. This provides more flexibility in arranging code but this implies that more decisions should be taken about it and more efforts should be provided to keep these decisions in mind:

And then every time someone uses the method, they need to remember where it was defined and add the appropriate import statement. If they forget, they might get behavior they don’t expect, as the appropriate method for the given generic function won’t even have been loaded.

Establishing conventions can be considered a solution to this issue. However, Tang believes that “conventions that aren’t enforced by the language tend not to be followed”. Moreover, having tried to define some patterns for modules arrangements, he also asserts that “this is inherently problem-domain-specific: there’s no one right organizing principle that applies to everyone’s programs.”

Hence, according to Johnatan Tang there is no obvious solution to this tradeoff of flexibility vs. productivity in organizing programs. What about your favorite languages? What solutions could you find? And, given your experience, is there any convenient compromise between the two?

Having worked with several Object-Relational mapping frameworks in the last few years, I got to a point where I couldn’t justify their complexity in my project. We often talk about the mismatch between the database and the object worlds, and that is where ORMs are often stated and referenced for “bridging the gap”!

Well I prefer to call it lifting the gap, or highering the gap, to have it now between DAOs and the rest of the code than having it between database and code.But I wouldn’t call this in any way reducing the gap.

First thing to mention here, is that data structures are not evil. We often deal with XML data structure but we absolutely refuse to deal with relational data structures. The problem might be related to time, XML came with a whole arsenal of managed APIs that represent its model in a way that makes it more comfortable for you to manipulate these documents. All together with transformation APIs that made it a lot easier to fill your object model out of an XML. Relational data structure on the other hand did not have the luxury of standard APIs and representation models in mainstream languages, and that made dealing with a database request result so arbitrary (Datasets existed in .Net but they have been cursed heavily and has been forced out of the cool toolbox, do not know yet why!) . 

In some way, I guess that the “mismatch” between Relational data structures and objects exists much similarly between XML and objects, however we didn’t think for so long of a doing a mapping between XML and object OXM? (no we are starting to do so, writing some XML to map XML to objects!).

With the experience I had with mapping technologies, I can say that they do not solve the problem for me. They merely add yet another useless level of abstraction where your domain types look nothing like relational data structures, but yet nothing like your target domain model! They have gone a half way, and now its your turn to poison your application with a lot of absurd ORM code trying to stretch this bridge to reduce the gap…

Geeks love talking about intrusivty of an ORMs and compare level of intrusivty in different terms. Intrusivty is much more than the lack of an interface or methods to implement in your domain objects. I guess I am ok to deal with that kind of intrusivity. However all ORMs I tried do not give me the opportunity to express my domain model in the way I would like to. I have to deal with notions of Lazy Loading, Attached/Detached entities, Entity sets, ids, back fields and a lot of other things that I don’t want to appear in my model and that make the resulting code so ugly and complex that I don’t wish to visit soon.

As Bob Martin said before, we are missing the opportunity to do oop, I would enforce and say we are missing the opportunity to model at all. And I guess we have gone too far with ORMs and it is time to go back to the basics, in the same way we have done after the EJB great abstraction.

It is very contradictory, while we are so excited about DSLs and the goodness they can potentially bring to enterprise development that results in a declarative semanticful style of programming, we can’t help but to engage all of our means in trying to bury SQL (and effective, semanticful, declarative powerful domain specific language) behind a dumb API to “abstract database access”! I guess we got to chose one strategy, either we keep SQL and we  keep marketing DSLs or we hope for a new advanced GPL compiler that will hide all of this in the same way modern languages did with garbage collection, concurrency and synchronization.

Since I am not yet a compiler writer, and I aint got around an effective way of integrating database in mainstream languages (there is actually a very productive in memory database on Haskell that worth trying), I decide to keep the DSL, and to work with them without the need to hide them, but rather to get a better tooling to help converting between difference data structures.

I guess you’ve guessed it by now. For me LinQ is the mid-term solution for the data structure mismatch. No not LinQ to sql, nor Ado.Net entities (they are helpful and we might use them but lets focus on the solution). LinQ is a very productive tool that halps effectively transform anything to anything in few lines of code. I mean that I can still use powerful SQL to get data effeciently and then convert them with one line of LinQ into the destination data structure. No abstraction leak, less complexity: Use the right tool where it applies.

So what I would actually do is to use a framework that will execute my SQL or SQL like query on the database and get me the records in much the same way they are in the database,Much in the same way Active Record gets you records as they are in the database. No useless efforts from the framework to bend the data to make it look untruthfully like the destination model.  I guess I know better my domain model, and with this better tooling I can get it done with no problem.

posted originally on http://www.infoq.com/news/2008/10/measure-programmers-productivity

In the field of software development, just as in any other area, managers need to appreciate performance of their programmers and the progress of their projects. However, defining appropriate metrics to this end appears to be a tricky task.

Measuring source lines of code (SLOC) is one of commonly used approaches that presents however a number of important limitations highlighted recently by Shahar Yair and Steve McConnell. First of all, the amount of lines of code does not allow effectively measuring the progress of a project because it focuses on activity rather than results. LOC do not have any value as such: the value of the final product depends on its performance and quality, not on the amount of code. Hence, focusing on the latter is actually a very limited approach to productivity.

SLOC doesn’t tell anything about the complexity of the issue at hand or about the quality of the final product in terms of maintainability, flexibility, extensibility, etc. With regard to quality, it can actually be negatively correlated. Refactoring as well as some design patterns result in reducing LOC while improving code’s quality. Larger code base may mean more noise, more unnecessary complexity, less readability and flexibility.

What is particularly risky about adopting such a reductive view of programmers’ performance is that it creates wrong incentives. Rather than optimizing their work in terms of final product, developers may be encouraged to favor the quantity of code to the detriment of quality and even intentionally write more verbose code. “What gets measured gets done”, recalls Steve McConnell.

He points out that some of these issues can be solved by using function points as measuring metric. The program size is then determined by the number of inputs, outputs, queries and files. Nevertheless, this approach also has its downsides. McConnell mentions some practical issues like the necessity of having a certified function point counter and the difficulties with mapping each function point back to individual programmers. A certified function point specialist Daniel Yokomizo highlights in his comment other limitations of such approach: lack of tools to measure function points’ complexity and to take into consideration things such as code sharing, frameworks, libraries, etc., that affect the time needed to create a feature.

Even though many commentators involved in the discussion about measuring approaches agree on their limitations, they do not necessarily dismiss the need for measuring programmers’ performance. Many insist on the fact that SLOC, for instance, can be used as a baseline for a more complex analysis combining different factors. This goes along the lines of four principles outlined by McConnell that should guide any analysis of programmers’ productivity:

1. Don’t expect any single dimensional measure of productivity to give you a very good picture of individual productivity […]

2. Don’t expect any measures […] to support fine-grain discrimination in productivity among individuals. [They] give you questions to ask but they don’t give you the answers […]

3. Remember that trends are usually more important than single-point measures […]

4. Ask why you need to measure individual productivity at all […]

In which context measuring programmers’ productivity is actually meaningful? What criteria can be used for it? How can they be combined? Many questions are still open for discussion and if your experience has brought you some answers, don’t hesitate to share.