Recently, I’ve been working on a Web Api system where, thanks to support of @jeanlaurent http://morlhon.net/blog/, I used Scala applying DCI architecture in a real world project. This post is about reporting benefits of using this approach. Other posts will follow that will be more focused on the use of Scala and Functional Programming in that project. Code included is a bit simplified and parts of the system that are not of interest are omitted.

The concerned part of the system is an API for logging a Web API usage and using it for showing interesting statistics to users and for implementing a Web 2.0 business model authorization.

The story starts with an API key passed as a parameter in the requested URI something like product/1234?api_key=AFFSEFFVFE2344

I create an object that represents the API user containing the key: (did I tell I like one liners?)

This is a class of an object that holds the apiKey, now I need a method that I will call to log the use of product/123 for user represented by the apiKey AFFSEFFVFE2344. What could be quite convenient and OOP-iesh is to call a record method on the apiKey instance, something like apiUser.record(…) . The problem with this, as explained in the original DCI article, is that adding these methods to the domain objects adds noise in other contexts for which recording is not relevant such as authorization and statistics, as we will see later. I would rather use here the DCI’s role concept (or context). Doing so I would say (apiUser in recording role).record(…) . This pseudo code tends to be very close to mixins implemented in Scala language and will look like following in real world Scala code:

Here the ApiUser object doesn’t contain a record method by default and so it can be shared in different contexts. In other words we kept our domain objects pure yet not scarifying OOP convenience. Now the Recorder is a trait that minds only business in the context of recording. The definition looks like the following:

Another view of the system is the end user’s statistics. Statistics should be represented in a meaningful way to the user and the user can’t care less about how the recording is done. So, it is quite desirable not to have the noise of recording when talking about the statistics view context. Here is how the call looks like in our example in Scala:

Again what we are doing here looks merely like saying: I want to use the ApiUser in a statistics context and I choose to call method getRequestsPerMonthForLast12Months. I guess the convenience brought by using this approach is obvious. This is how the Statistics trait looks like:

Note that as in the Recording trait, the ApiUser object is not passed as an explicit parameter but rather as an implicit receiver parameter (something like this for an object). To access it we use Self that refers to it.

The third view of the system is what allows us to provide an authorization mechanism based on user’s activities for a business plan they choose (be it a quota per day, per month or other interesting combinations). It could be quite convenient to have the call looking like apiUser.isAuthorized . We get pretty close with DCI without polluting our domain object with a specific context logic:

Like the previous, the Authorizer trait looks straightforward :

Now that we looked at the three views of our system introducing three examples of “Context” from DCI pattern we can go beyond that to discover how we use nested contexts (or roles).

Looking at the Authorizer trait, the ApiUser referred to by Self, doesn’t tell us much about how many requests the user did today or this month. And for authorizing user one needs a richer ApiUser that can give the necessary information that live themselves in a file or a database. But once again, we don’t want to pollute our domain object with Data Access code that varies on a different axis than the domain object itself.  Introducing a nested context/role that adds the Data Access logic would pretty convenient here:

Here Self is a richer ApiUser that carries Requests Data Access logic defined in the RequestsDataAccessor trait:

And the implementation of the Authorizer trait defined in terms of ApiUser with RequestsDataAccessor:

In the same way we implement the Statistics and Recorder traits using ApiUser with RequestsDataAccessor:

If you are not familiar with generics ( getRequestsPer[Day] ) you can ignore them since they are not central to the subject of this post and simply use their readability to understand the purpose of the method. The use of generics in this project will be the subject of a future post.

Encapsulation Test!

Now in the first version of the system, the ApiKey was passed in clear in the service URI. This is not very secured as it is quite easy to get the key and use it for using the API. We might prefer to encrypt the key in some way. This fortunately changes nothing for any of our traits implementations since the decryption of the key is the responsiblity of the ApiUSer object and all what our traits need is to get the concerned ApiKey from that object which can be easily done with pattern matching. Same thing if we want to replace database logging with file system in RequestsDataAccessor. Modularity and separation of concerns are apparent in this DCI architecture implementation.

Conclusion

This is a refreshing approach to OOP. The Domain Context Interaction pattern allowed us to restore interesting properties of real OOP approach yet not sacrificing modularity and separation of concerns. This simple view of “an object in several contexts” helps quite much in thinking explicitly about and understanding context boundaries. Note that an object(instance) is not defined by a single class but each instance curries the necessary context. This is essential to OOP which is NOT COP (for Class Oriented Programming ;) )

Feel free to ask questions and/or comment about this implementation. Refer to original papers for more in-depth explanation of the pattern.

Thanks to JetBrains’ IDEA and the Scala plug-in that helped me programming in Scala.

http://www.infoq.com/news/2009/05/dci-coplien-reenskau

http://www.infoq.com/news/2009/01/rwh-book-interview

For those that didn’t happen to be there, here is a link to my tinny presentation I did in VTDays last year. There is a huge room for improvement, yet funny :)

http://storage02.brainsonic.com/customers/valtech/20081021/AR7/files/index_popup.html

One thing that annoys me though about most programming language books is how raw they often are.

  Probably constrained by time, authors and book teams seem to invest less on editorial stuff and teaching methodology and more on explaining details of technical and behind the scenes stuff. This often results in more manual-like books where you have a great wealth of material that you are not able to exploit anyway because of, well, lack of time. I am not sure I can invest time in reading 500+ manual of a new, or at least non mainstream, language that I might decide not to use by the end. 

I am not saying here that technical and behind the scene stuff does not matter, quite the opposite. I just think that manual, important it is, should not be the introduction to the programming language. A rather 100 to 200 pages brief book that is carefully edited to introduce paradigm, concepts, and strengths of the programming language is a more attractive choice , accompanied with a wordy but well organized reference book with examples and more in-depth explanation of language features. I guess this is a more pragmatic approach to programming language learning in the presence of WWW and Google search.

PS: I personally find Paul Hudak’s Haskell book a very good example of an enjoyable, brief, nicely and carefully edited programming language book. Also LUA’s programming book is nicely done.

Having worked with several Object-Relational mapping frameworks in the last few years, I got to a point where I couldn’t justify their complexity in my project. We often talk about the mismatch between the database and the object worlds, and that is where ORMs are often stated and referenced for “bridging the gap”!

Well I prefer to call it lifting the gap, or highering the gap, to have it now between DAOs and the rest of the code than having it between database and code.But I wouldn’t call this in any way reducing the gap.

First thing to mention here, is that data structures are not evil. We often deal with XML data structure but we absolutely refuse to deal with relational data structures. The problem might be related to time, XML came with a whole arsenal of managed APIs that represent its model in a way that makes it more comfortable for you to manipulate these documents. All together with transformation APIs that made it a lot easier to fill your object model out of an XML. Relational data structure on the other hand did not have the luxury of standard APIs and representation models in mainstream languages, and that made dealing with a database request result so arbitrary (Datasets existed in .Net but they have been cursed heavily and has been forced out of the cool toolbox, do not know yet why!) . 

In some way, I guess that the “mismatch” between Relational data structures and objects exists much similarly between XML and objects, however we didn’t think for so long of a doing a mapping between XML and object OXM? (no we are starting to do so, writing some XML to map XML to objects!).

With the experience I had with mapping technologies, I can say that they do not solve the problem for me. They merely add yet another useless level of abstraction where your domain types look nothing like relational data structures, but yet nothing like your target domain model! They have gone a half way, and now its your turn to poison your application with a lot of absurd ORM code trying to stretch this bridge to reduce the gap…

Geeks love talking about intrusivty of an ORMs and compare level of intrusivty in different terms. Intrusivty is much more than the lack of an interface or methods to implement in your domain objects. I guess I am ok to deal with that kind of intrusivity. However all ORMs I tried do not give me the opportunity to express my domain model in the way I would like to. I have to deal with notions of Lazy Loading, Attached/Detached entities, Entity sets, ids, back fields and a lot of other things that I don’t want to appear in my model and that make the resulting code so ugly and complex that I don’t wish to visit soon.

As Bob Martin said before, we are missing the opportunity to do oop, I would enforce and say we are missing the opportunity to model at all. And I guess we have gone too far with ORMs and it is time to go back to the basics, in the same way we have done after the EJB great abstraction.

It is very contradictory, while we are so excited about DSLs and the goodness they can potentially bring to enterprise development that results in a declarative semanticful style of programming, we can’t help but to engage all of our means in trying to bury SQL (and effective, semanticful, declarative powerful domain specific language) behind a dumb API to “abstract database access”! I guess we got to chose one strategy, either we keep SQL and we  keep marketing DSLs or we hope for a new advanced GPL compiler that will hide all of this in the same way modern languages did with garbage collection, concurrency and synchronization.

Since I am not yet a compiler writer, and I aint got around an effective way of integrating database in mainstream languages (there is actually a very productive in memory database on Haskell that worth trying), I decide to keep the DSL, and to work with them without the need to hide them, but rather to get a better tooling to help converting between difference data structures.

I guess you’ve guessed it by now. For me LinQ is the mid-term solution for the data structure mismatch. No not LinQ to sql, nor Ado.Net entities (they are helpful and we might use them but lets focus on the solution). LinQ is a very productive tool that halps effectively transform anything to anything in few lines of code. I mean that I can still use powerful SQL to get data effeciently and then convert them with one line of LinQ into the destination data structure. No abstraction leak, less complexity: Use the right tool where it applies.

So what I would actually do is to use a framework that will execute my SQL or SQL like query on the database and get me the records in much the same way they are in the database,Much in the same way Active Record gets you records as they are in the database. No useless efforts from the framework to bend the data to make it look untruthfully like the destination model.  I guess I know better my domain model, and with this better tooling I can get it done with no problem.

In my trip through Haskell programming I keep discovering how clean and enjoyable the language is. By the end I am someone that loves doing abstractions and I guess I’ve found my world.

I discussed with some colleagues lately how a programming language creates a whole culture around it. This culture inherits from language properties and shapes the language programming style.

Take for example Java; there are a lot of different ways to program in java. The java syntax does not force you to program OOP. It is just more convenient to do so. Same thing goes with other programming languages. However it is not only about paradigm, it is really about a whole mindset.

While programming a web site with Haskell, I was searching for a function that parses a string to return me an int. As I’ve been for a long time a C#/Java programmer I expected to find a simple function like what I used to use in these language. A function that simply takes a string and produces and int. After a fair search I didn’t find such a “simple” function. I’ve rather found a function with slightly more complicated/richer signature (type). This function takes a string and returns a list of int-string pairs. This got me thinking, there should be a reason for not offering the simple function I was searching for…

I quickly started understanding why libraries implementers chose to offer this function with this signature and I did so simply by imagining its use. Imagine that I pass a string that is not the expected type (in our case int) to this reads (stands for read string) function, what would happen? Well in the simple version of C# and Java, the parsing function well throw an exception when passed a string that can not be parsed. In some way, the signature of the function promises me to get always the expected type (in our case an int) as a return of using it but it actually lies! Throwing an unexpected exception comes as a surprise as the signature (the function type) looked extremely kind but behaved irresponsibly unexpectedly when I trusted it. But what about the Haskell version?

Well the type of the Haskell function tells me “I will try to parse the string passed, and each time I find a fit with your expected type I will return a pair with the value in the expected type and a remaining string that represents what remains”. So here the function type tells me that it will try to parse and that it will might face several possibilities. There is no lie here, no magic, no false promise, parsing strings can produce several possibilities depending on the parsing method, and that what the function tells me.

I like the no surprise / safe  programming culture in Haskell. I guess there are a lot of lessons to be learnt from this style of programming.

PS: Some might call this safe style “defensive programming” to make it look rigid, I guess there is no point in calling it so, and I would rather call it communicative programming, no surprise programming, or intention revealing programming. I like DbC!

How many languages are you using on the same project? If you go counting you will see that they are many. I mean XML, Java, XSLT, HTML, CSS… etc. But the reason why you are using almost all of them is that they happen to be mainstream and, oftentimes, they are the only language choice for a needed framework. You are actually almost obliged to use them. The choice is done for you. Style? CSS. Configuration? Often XML. Web interface description? Html. However, if you want to adopt true polyglot programming, you will have to face inevitable decision of language choice.

Read my article on Paradigm based Polyglot Programming at InfoQ.com

The big milestone of C# 3.0 got me thinking, what can we do more to improve the language. After a lot of interesting discussions an debates I got with people that care about the subject, I realized that it can be really interesting to remove some unimportant complexity of the language. This can be done by abstracting and generalizing some concepts. An example that I thought of for this blog post is Methods.

Do we need Methods?

Why do we need methods at all? For me, in C# 3.0, a method is just a special case of of a property that returns a function as value. What I mean is that this:

public string SayHello(){

return “Hello”;
}

is identical to

public Func<string> SayHello{
get{
return ( ) => “Hello”;
}
}

In functional programming languages, functions are values exactly like int and string. They can be assigned to variables, passed as a parameter and return from other functions. This fits very well with the definition of a method I’ve just given.   So what I am doing here is eliminating the need for the special case of method and using properties of function type instead. The syntax is acceptable (actually I can even argue that it is more readable). Moreover, adopting this approach buys us much more than removing one existing duplication in the language.

Benefits of generalizing Methods to Properties of function value type

The first thing I would argue about is readability,  I guess it is more readable to describe methods as lambdas, but this can be considered being subjective, so I will rather focus on other practical benefits of the generalization:

Passing some methods implementations in the constructor:

Almost one year ago, I asked the C# team whether it can be possible to implement anonymous classes in C#. Anonymous classes ( quite different from Anonymous Types) is a neat feature that allows you to instantiate an instance of a Class that implements a desired contract. With anonymous classes you do not actually need to create the class, you just specify the contract you need to satisfy then you start implementing the logic.

Using our generalization we can do something similar (at least semantically).  For Instance we can do:

new HelloWordContract( HelloWorldGreeter= (s=>  Console.WriteLine(”Hello World “+ s)) );

Modifying an implementation at runtime only for the instance:

Manipulating functions exactly like values opens up for a very powerful mechanism of function passing. One very powerful functionality in Ruby is that you can change an instance’s method implementation at runtime. And that will only mutate the instance’s implementation and won’t touch to the class’ definitions. Using our generalization it becomes really easy to implement an extension method that changes the implementation (which is the property value) of a method for an instance.

public static ModifyImplementation<TObject,TFunc>(this TObject object, Func<TObject,TFunc> newFunctionValue){…} //here I pass the instance value in case needed by the function

Conclusion

Improving programming languages is not only about adding features, but it is sometimes about deleting some.  I illustrated briefly in this post how we could, with the simple method generalization, benefit from a very powerful use of functions in C#. To support better this approach we still need a better type inference for functions types (delegates) and function types with more parameters ( something like Func<T1,T2,T3,…………………….Tn,R>.