Recently, I’ve been working on a Web Api system where, thanks to support of @jeanlaurent http://morlhon.net/blog/, I used Scala applying DCI architecture in a real world project. This post is about reporting benefits of using this approach. Other posts will follow that will be more focused on the use of Scala and Functional Programming in that project. Code included is a bit simplified and parts of the system that are not of interest are omitted.

The concerned part of the system is an API for logging a Web API usage and using it for showing interesting statistics to users and for implementing a Web 2.0 business model authorization.

The story starts with an API key passed as a parameter in the requested URI something like product/1234?api_key=AFFSEFFVFE2344

I create an object that represents the API user containing the key: (did I tell I like one liners?)

This is a class of an object that holds the apiKey, now I need a method that I will call to log the use of product/123 for user represented by the apiKey AFFSEFFVFE2344. What could be quite convenient and OOP-iesh is to call a record method on the apiKey instance, something like apiUser.record(…) . The problem with this, as explained in the original DCI article, is that adding these methods to the domain objects adds noise in other contexts for which recording is not relevant such as authorization and statistics, as we will see later. I would rather use here the DCI’s role concept (or context). Doing so I would say (apiUser in recording role).record(…) . This pseudo code tends to be very close to mixins implemented in Scala language and will look like following in real world Scala code:

Here the ApiUser object doesn’t contain a record method by default and so it can be shared in different contexts. In other words we kept our domain objects pure yet not scarifying OOP convenience. Now the Recorder is a trait that minds only business in the context of recording. The definition looks like the following:

Another view of the system is the end user’s statistics. Statistics should be represented in a meaningful way to the user and the user can’t care less about how the recording is done. So, it is quite desirable not to have the noise of recording when talking about the statistics view context. Here is how the call looks like in our example in Scala:

Again what we are doing here looks merely like saying: I want to use the ApiUser in a statistics context and I choose to call method getRequestsPerMonthForLast12Months. I guess the convenience brought by using this approach is obvious. This is how the Statistics trait looks like:

Note that as in the Recording trait, the ApiUser object is not passed as an explicit parameter but rather as an implicit receiver parameter (something like this for an object). To access it we use Self that refers to it.

The third view of the system is what allows us to provide an authorization mechanism based on user’s activities for a business plan they choose (be it a quota per day, per month or other interesting combinations). It could be quite convenient to have the call looking like apiUser.isAuthorized . We get pretty close with DCI without polluting our domain object with a specific context logic:

Like the previous, the Authorizer trait looks straightforward :

Now that we looked at the three views of our system introducing three examples of “Context” from DCI pattern we can go beyond that to discover how we use nested contexts (or roles).

Looking at the Authorizer trait, the ApiUser referred to by Self, doesn’t tell us much about how many requests the user did today or this month. And for authorizing user one needs a richer ApiUser that can give the necessary information that live themselves in a file or a database. But once again, we don’t want to pollute our domain object with Data Access code that varies on a different axis than the domain object itself.  Introducing a nested context/role that adds the Data Access logic would pretty convenient here:

Here Self is a richer ApiUser that carries Requests Data Access logic defined in the RequestsDataAccessor trait:

And the implementation of the Authorizer trait defined in terms of ApiUser with RequestsDataAccessor:

In the same way we implement the Statistics and Recorder traits using ApiUser with RequestsDataAccessor:

If you are not familiar with generics ( getRequestsPer[Day] ) you can ignore them since they are not central to the subject of this post and simply use their readability to understand the purpose of the method. The use of generics in this project will be the subject of a future post.

Encapsulation Test!

Now in the first version of the system, the ApiKey was passed in clear in the service URI. This is not very secured as it is quite easy to get the key and use it for using the API. We might prefer to encrypt the key in some way. This fortunately changes nothing for any of our traits implementations since the decryption of the key is the responsiblity of the ApiUSer object and all what our traits need is to get the concerned ApiKey from that object which can be easily done with pattern matching. Same thing if we want to replace database logging with file system in RequestsDataAccessor. Modularity and separation of concerns are apparent in this DCI architecture implementation.

Conclusion

This is a refreshing approach to OOP. The Domain Context Interaction pattern allowed us to restore interesting properties of real OOP approach yet not sacrificing modularity and separation of concerns. This simple view of “an object in several contexts” helps quite much in thinking explicitly about and understanding context boundaries. Note that an object(instance) is not defined by a single class but each instance curries the necessary context. This is essential to OOP which is NOT COP (for Class Oriented Programming ;) )

Feel free to ask questions and/or comment about this implementation. Refer to original papers for more in-depth explanation of the pattern.

Thanks to JetBrains’ IDEA and the Scala plug-in that helped me programming in Scala.

Clojure multimethods are just another level of that same logic, in fact they are a realization of the last sentence I just said. They are dispatch based upon an arbitrary function of the arguments. You define a multimethod and you say “Here is a function of the arguments I’d like you to use” You could look at the first argument, you could look at the 5th, you could look at all of them, you could look inside them, some member of an argument, it could look at the types or not or the values. Now, you could look at relationships between arguments, you have dispatch based upon an arbitrary function of the arguments and you have a vastly wider set of polymorphic possibilities than you had before and it’s quite powerful. In particular, it allows you to do Runtime dispatch on Runtime attributes. You don’t usually represent something like being hungry as part of something’s type, it’s some attribute that it acquires while the program is running or being outdated or things like that. Now you can access those things and you can do things polymorphically based upon that and take a lot of switch statements out of your code.

Should I call it first class Pattern Matching? or Pattern Matching in disguise?

http://www.infoq.com/interviews/F-Sharp-Don-Syme

originally posted on http://www.infoq.com/news/2009/01/pl-predictions-2009

In the beginning of the last year, Ehud Lamm launched a thread on Lamba the Ultimate inviting other bloggers to submit their predictions for 2008 in fields relative to programming languages (PL).

Concurrency was one of the first topics to be defined as an upcoming theme of the year even though it was argued that it “won’t go anywhere because the current paradigms […] and architectures […] can’t deal with it very well”. Many equally ambivalent predictions concerned functional programming languages. Haskel was supposed to “rock the blogosphere” without being widely used as such, but rather becoming an inspiration for new features in more mainstream programming languages. More generally speaking, some expected that “no functional language [would] become significantly popular”, while several other bloggers had far more optimistic prognoses for F# and Scala that were to “enjoy large uptakes”, at least through the development of "multi-language" projects“ in some combination of F#/C# or Scala/Java”.  As far as other languages were concerned, Java was supposed to become “more entrenched”. And the same was to happen to Ruby that would be undermined by the downtrend of Rails. On the contrary, C++ with its new ISO standard for 2009 was expected to “become the language-of-the-year in 2008” and Javascript to gain more momentum.

As an echo to this tread, James Iry asked bloggers at the end of 2008 to share their opinion on “what was noteworthy about 2008 as far as programming languages were concerned”.

Trying to assess the validity of last year predictions, Key Schluehr believes that concurrency was far from being the theme of the year. And, if there were one, it was, in his opinion, “cloud computing”, even though he considers that “this had little to with computing at all”. He also asserts that, just like Morris Johns expected, no functional language became significantly popular, which many other bloggers disagree with.

James Iry argues indeed that even though no functional language is at Java or even Ruby level of popularity, “the fact that languages like these popped up so brightly on the mainstream radar last year is not just significant, but huge.” Eli Ford highlights that “F# got its own CTP in September, and is now slated for a supported release as part of Visual Studio 2010, alongside C# and VB”. And Sean McDirmid argues that “as far as specific languages go 2008 was a good year for Scala”. “Clojure”, that was not at all mentioned in the last year discussion, is considered to be the discovery of the year by Chris Rathman who believes that it is a good example of how “to go about integrating existing concepts into a programming language”.

Along with Scala, Sean McDirmid mentions Objective C “as the real hot language of 2008 thanks to the iPhone SDK” and believes that 2008 was also the year of C vengeance with various forms of it being “used to program GPU hardware (HLSL, CUDA, OpenCL…)”.

On the other hand, several bloggers highlight that last year was not that good for Java. Sean McDirmid asserts that “JavaFX was late and didn’t make the splash it needed to make”. If it is true that bloggers express some concerns about Java’s future, Daniel Weinreb stresses that “it’s used in so many places now that we’re very unlikely to see it disappear” and, according to James Iry, “it is still and will remain for quite awhile one of the a handful of "safe" choices for an IT manager”. Others question however the capacity of Sun to survive the current crisis and speculate about the future of JVM expecting IBM or Google to step in.

The discussion moves from 2008 to 2009 and a number of new predictions are made. In the field of functional programming, James Iry expects great things from both Clojure and Scala teams while Falcon argues that “2009 will be a year of clojure rather than scala” and expects F# to be finally brought “to the attention of mainstream .NET developers.” Ross Smith, however, reiterates the prediction of last year that functional programming will enter the mainstream rather “by being incorporated into pre-existing procedural and OO languages”. He also believes that “the new C++ standard will become official”, “concurrency, including GPGPU applications, will continue its rise in importance”, “Python will start to bleed users specifically because of [its] lack of good support for concurrency” while “JavaScript’s star will continue to rise”.

Xscott goes along the same lines about JavaScript predicting that it “will eventually become the popular server and application scripting language – mostly because of it’s various JIT compiling implementations”, whereas Kay Schluehr believes that it “will not expand on its niche”. On the other hand, he thinks that “one of the great-future-of-programming hopes like Perl 6, Rubinius or PyPy [will] finally [reach] a state where programmers other than core developers start to show interest.”

Kaveh Shahbazian believes that “one thing [that] is going to happen [in 2009] is finding new approaches to employ scripting” and names Lua as a successful example. And, last but not least, Sean McDirmid predicts that “no progress will be made on the static/dynamic debate.” :)

One thing that annoys me though about most programming language books is how raw they often are.

  Probably constrained by time, authors and book teams seem to invest less on editorial stuff and teaching methodology and more on explaining details of technical and behind the scenes stuff. This often results in more manual-like books where you have a great wealth of material that you are not able to exploit anyway because of, well, lack of time. I am not sure I can invest time in reading 500+ manual of a new, or at least non mainstream, language that I might decide not to use by the end. 

I am not saying here that technical and behind the scene stuff does not matter, quite the opposite. I just think that manual, important it is, should not be the introduction to the programming language. A rather 100 to 200 pages brief book that is carefully edited to introduce paradigm, concepts, and strengths of the programming language is a more attractive choice , accompanied with a wordy but well organized reference book with examples and more in-depth explanation of language features. I guess this is a more pragmatic approach to programming language learning in the presence of WWW and Google search.

PS: I personally find Paul Hudak’s Haskell book a very good example of an enjoyable, brief, nicely and carefully edited programming language book. Also LUA’s programming book is nicely done.

Originally posted on http://www.infoq.com/news/2008/11/oop-vs-fp-4-code-structure

Programming languages that offer more power and flexibility have been lately gaining momentum. Johnatan Tang highlights, however, the flexibility vs. productivity tradeoff due, amongst other things, to the fact that traditional object orientation makes organizing programs easier and more straightforward.

He argues that in languages with single dispatch, “given a parameter or other value, you know exactly what you can do with it”. Whereas in statically typed languages code is more revealing about values involved, in structurally typed languages it is more general, but still gives an immediate idea what operations are available on a given value. And for both, statically and structurally typed languages, it is pretty clear what to do if you want to perform an operation that is not supported by a given object:

You write a method that performs that operation. This is problematic if you don’t control that class, which is why Ruby and C# have moved to open classes. […]

And it’s usually obvious where this method should go: on the class that you need to manipulate.

In multi-dispatch languages, it is much less obvious where a method should be written and “which parameter of the new method might take [a given] value”. This provides more flexibility in arranging code but this implies that more decisions should be taken about it and more efforts should be provided to keep these decisions in mind:

And then every time someone uses the method, they need to remember where it was defined and add the appropriate import statement. If they forget, they might get behavior they don’t expect, as the appropriate method for the given generic function won’t even have been loaded.

Establishing conventions can be considered a solution to this issue. However, Tang believes that “conventions that aren’t enforced by the language tend not to be followed”. Moreover, having tried to define some patterns for modules arrangements, he also asserts that “this is inherently problem-domain-specific: there’s no one right organizing principle that applies to everyone’s programs.”

Hence, according to Johnatan Tang there is no obvious solution to this tradeoff of flexibility vs. productivity in organizing programs. What about your favorite languages? What solutions could you find? And, given your experience, is there any convenient compromise between the two?

Having worked with several Object-Relational mapping frameworks in the last few years, I got to a point where I couldn’t justify their complexity in my project. We often talk about the mismatch between the database and the object worlds, and that is where ORMs are often stated and referenced for “bridging the gap”!

Well I prefer to call it lifting the gap, or highering the gap, to have it now between DAOs and the rest of the code than having it between database and code.But I wouldn’t call this in any way reducing the gap.

First thing to mention here, is that data structures are not evil. We often deal with XML data structure but we absolutely refuse to deal with relational data structures. The problem might be related to time, XML came with a whole arsenal of managed APIs that represent its model in a way that makes it more comfortable for you to manipulate these documents. All together with transformation APIs that made it a lot easier to fill your object model out of an XML. Relational data structure on the other hand did not have the luxury of standard APIs and representation models in mainstream languages, and that made dealing with a database request result so arbitrary (Datasets existed in .Net but they have been cursed heavily and has been forced out of the cool toolbox, do not know yet why!) . 

In some way, I guess that the “mismatch” between Relational data structures and objects exists much similarly between XML and objects, however we didn’t think for so long of a doing a mapping between XML and object OXM? (no we are starting to do so, writing some XML to map XML to objects!).

With the experience I had with mapping technologies, I can say that they do not solve the problem for me. They merely add yet another useless level of abstraction where your domain types look nothing like relational data structures, but yet nothing like your target domain model! They have gone a half way, and now its your turn to poison your application with a lot of absurd ORM code trying to stretch this bridge to reduce the gap…

Geeks love talking about intrusivty of an ORMs and compare level of intrusivty in different terms. Intrusivty is much more than the lack of an interface or methods to implement in your domain objects. I guess I am ok to deal with that kind of intrusivity. However all ORMs I tried do not give me the opportunity to express my domain model in the way I would like to. I have to deal with notions of Lazy Loading, Attached/Detached entities, Entity sets, ids, back fields and a lot of other things that I don’t want to appear in my model and that make the resulting code so ugly and complex that I don’t wish to visit soon.

As Bob Martin said before, we are missing the opportunity to do oop, I would enforce and say we are missing the opportunity to model at all. And I guess we have gone too far with ORMs and it is time to go back to the basics, in the same way we have done after the EJB great abstraction.

It is very contradictory, while we are so excited about DSLs and the goodness they can potentially bring to enterprise development that results in a declarative semanticful style of programming, we can’t help but to engage all of our means in trying to bury SQL (and effective, semanticful, declarative powerful domain specific language) behind a dumb API to “abstract database access”! I guess we got to chose one strategy, either we keep SQL and we  keep marketing DSLs or we hope for a new advanced GPL compiler that will hide all of this in the same way modern languages did with garbage collection, concurrency and synchronization.

Since I am not yet a compiler writer, and I aint got around an effective way of integrating database in mainstream languages (there is actually a very productive in memory database on Haskell that worth trying), I decide to keep the DSL, and to work with them without the need to hide them, but rather to get a better tooling to help converting between difference data structures.

I guess you’ve guessed it by now. For me LinQ is the mid-term solution for the data structure mismatch. No not LinQ to sql, nor Ado.Net entities (they are helpful and we might use them but lets focus on the solution). LinQ is a very productive tool that halps effectively transform anything to anything in few lines of code. I mean that I can still use powerful SQL to get data effeciently and then convert them with one line of LinQ into the destination data structure. No abstraction leak, less complexity: Use the right tool where it applies.

So what I would actually do is to use a framework that will execute my SQL or SQL like query on the database and get me the records in much the same way they are in the database,Much in the same way Active Record gets you records as they are in the database. No useless efforts from the framework to bend the data to make it look untruthfully like the destination model.  I guess I know better my domain model, and with this better tooling I can get it done with no problem.

Originally posted at http://www.infoq.com/news/2008/10/more-languages-on-erlang-vm

Some time ago a blog post on Antares Trader explored the idea of implementing a ruby-like language on the top of Erlang virtual machine – BEAM – in order to combine the advantages offered by Ruby syntax and those of Erlang’s concurrency model:

Ruby makes OO programming easy, enjoyable and intuitive [...] but its concurrency model is out of the dark ages” [...]

Erlang makes concurrency easy and natural, but its syntax is poor, static, and high on ceremony

[...] if Ruby can run on top of the Java Virtual Machine and take advantage of all the power of that environment while still being an expressive and efficient language, why not have a modified version of ruby on top of BEAM, the Erlang virtual machine.

While the author outlines how a Ruby like language on top of Erlang could look like, commentators highlight possible challenges and necessary precautions and point out to the existing initiatives of implementing languages on BEAM, such as Reia (a Python/Ruby like language) and LFE (Lisp Flavoured Erlang).

Debasish Ghosh, who wrote several posts about Erlang virtual machine, seems to believe momentum is growing for increasing number of languages variants being implemented on top of BEAM. In his post „Erlang VM : now hosting multiple languages”, he also mentions Reia and LFE whereas one comentator refers to some Haskel based initiatives. Debasish suggests that this growth of Erlang ecosystem is part of a more general trend observed by Ted Leung who argues that „we are going to see not only flourishing new virtual machines, but also lots of languages atop existing virtual machines“.

How many languages are you using on the same project? If you go counting you will see that they are many. I mean XML, Java, XSLT, HTML, CSS… etc. But the reason why you are using almost all of them is that they happen to be mainstream and, oftentimes, they are the only language choice for a needed framework. You are actually almost obliged to use them. The choice is done for you. Style? CSS. Configuration? Often XML. Web interface description? Html. However, if you want to adopt true polyglot programming, you will have to face inevitable decision of language choice.

Read my article on Paradigm based Polyglot Programming at InfoQ.com