# A Tale of Three Loops

This one has been cooking for a very long time. Like many professional programmers I have often wondered what is it about programming that is just hard. Too hard in fact.

My intuition has led me in the direction of turing completeness: as soon as a language becomes Turing complete it also gathers to itself a level of complexity and difficulty that results in crossed eyes. Still, it has been difficult to pin point exactly what is going on.

### A Simple Loop

If you are a hard boiled programmer, then you will write a loop that looks a bit like:
``` int total = 0; for(Integer ix:table) total += ix; ```
Simple, but full of pitfalls. For one thing we have a lot of extra detail in this code that represents additional commitment:

• We have had to fix on the type of the number being totaled.
• We have had to know about Java’s boxed v.s. unboxed types.
• We have had to sequentialize the process of adding up the numbers.

While one loop is not going to hurt anyone; real code is stuffed with them. There have been occasions (not many thankfully) where I have written loops nested to a depth of 7 or 8. Such code really does become impossible to follow; let alone to write.

### A Functional Loop

In a functional programming language, there are two ways to accomplish the task. The apprentice’s approach might be to write a recursion:
``` total(nil) is 0; total(cons(E,L)) is total(L)+E; ```
While workman-like, for many instances a smarter way is to use a fold:
``` fold((+),0,L) ```
Apart from being more concise; the fold is higher-level: it abstracts away the machinery of the loop itself and it is also independent of the representation of the collection of numbers.

What is really interesting in relation to my original thesis is that the fold expression is closer to a problem-solving representation of the task.

However, ask any functional programmer about their use of `fold` and you will likely encounter a fairly procedural interpretation of how it works and how it should be used. (Something about how it successively applies the `+` function to each element of the list accumulating the answer as it goes.)

I.e., fold is better than for; but is not good enough.

### A Totalization Query

My third version of this would be familiar to any SQL programmer:

``` total X where X in L ```

I.e., if you want to add up the elements of the list, then say so!

This query — which is based on notation in the Star programming language — declaratively states what is required. Although it’s form is a little too specific, a more realistic variant would be:

``` fold X with (+) where X in L ```

I argue that either of these queries is better than either of the previous solutions because it comes closest to the original description and makes the fewest assumptions about the nature of the collections or the arithmetic.

It is also much closer to a problem oriented way of thinking about the world. I would argue that more people — especially non-programmers — would be able to follow and even to write such a query than either of the earlier formulations.

Why is that?

### The Homunculus

Traditional programming is often taught in terms of programs being sequences of steps that must be followed. What does that imply for the programmer? It means that the programmer has to be able to imagine what it is like to be a computer following instructions.

It is like imagining a little person — a homunculus — in the machine that is listing to your instructions and following them literally. You the programmer have to imagine yourself in the position of the homunculus if you want to write effective programs.

Not everyone finds such feats of imagination easy. It is certainly often tedious to do so.

# The true role of domain specific languages

It is easy to be confused by the term domain specific language. It sounds like a fancy term for jargon. It is often interpreted to mean some form of specialized language. I would like to explore another role for them: as vehicles for policy statements.

In mathematics there are many examples of instances where it is easier to attack a problem by solving a more general, more uniform, problem and then specializing the result to get the desired answer.

It is very similar in programming: most programs take the form of a general mechanism paired with a policy that controls the machine. Taken seriously, you can see this effect down to the smallest example:
```fact(n) where n>0 is n*fact(n-1); fact(0) is 1```
is a general machine for computing factorial; and the expression:`fact(10)` is a policy ‘assertion’ that specifies which particular use of the factorial machine is intended.

One important aspect of policies is that they need to be intelligible to the owner of the machine: unlike the machine itself which only needs to be trusted by the owner.

So, one critical role for a DSL is to be the policy language for the user of a mechanism.

Starting from this light leads to interesting conclusions. In particular, DSLs should be ubiquitous not rare; in particular, DSLs support the role that abstractions play in programming: by layering an appropriate syntax on top of the expression of the abstraction. It also means that programming languages should be designed to make it easy to construct and use DSLs within systems as well as externally.

A simple example: the query notation in Star, as well as in formalisms such as LINQ, may be better viewed as simple DSLs where the user is the programmer. The difference between these and more traditional DSLs is that the DSL expressions are embedded in the program rather than being separated from the code.

I think that embracing the DSL in this way should make it easier for a programming language to survive the evolution of programming itself. With an effective DSL mechanism a language ‘extension’ encoding a new language concept (for example, queries over C# or objects over C) and be done without invalidating the existing language. (The mechanisms in Star go further: it is possible to construct a DSL in Star that either augments the base language or even replaces it. We have used both approaches.)

It also explains why LISP’s macro facilities have allowed it to survive today more-or-less unchanged (nearly 60 years after being invented) whereas languages like Java and C++ have had to undergo painful transitions in order to stay relevant.

# Single Inheritance and Other Modeling Conundrums

Sometimes a restriction in a programming language makes sense and no sense at all — all at the same time.

## Modeling the real world

Think about the Java restrictions on the modeling of classes: a given class can only have one supertype and a given object’s class is fixed for its lifetime.

From a programming language perspective these restrictions make a good deal of sense: all kinds of ambiguities are possible with multiple inheritance and the very idea of allowing an object to be ‘rebased’ fills the compiler writer with horror. (Though SmallTalk allows it.)

The problem is that, in real life, these things do happen. A ‘natural’ domain model is quite likely to come up with situations involving multiple inheritance and dynamic rebasing.

For example, a person can go from being a customer, to an employee, to a manager to being retired. A given person might be both an employee and a customer simultaneously (someone else may not be).

Given a domain that is as flexible as this one if forced to ‘simulate’ it in Java. I.e., one cannot use a Java class called `Customer` to represent a customer; because Java’s idea of class is not rich enough to model the domain.

At the same time, the modeling is not random and a good architect will try to ensure some discipline in the application.

The logical conclusion is that large applications tend to contain a variant of ‘the type system’ where the domain model is represented. Java is used to implement the meta model, not the domain model.

This dynamic type system may or may not be based on a well founded model (such as that of description logic); but in any case the programming language is not helping as much as it should.

## What is a language to do?

On the face of it, it seems that the logical thing is to make a programming language’s type system sufficiently flexible to actually model real world scenarios.

However, there is a difficulty with that: it is not the case that any one modeling system is best suited to all applications. In addition, a modeling system that is well-suited to modeling domain knowledge is not guaranteed to be equally well suited to regular programming tasks.

A better approach is to embrace diversity. A combination of DSLs and libraries enable one to build out a particular modeling system and to support the programmer with direct appropriate syntax.

For example, this pseudo-code example:

```  customer isa person
customer has account
...
person has name
...
C instance of customer
...
if overdrawn(C's account) then
...
```

shows one example of a modeled customer. The ‘actual’ code implied by this fragment might look like:

```  C : object;
...
if overdrawn(findAttribute(C,"account")) then
...
```

The principal point here is that the syntactic sugar offered by a DSL is not mere syntactic sugar: it can help the application programmer to use a language that is appropriate for her needs while at the same time enforcing sanity checks implied by the particular modeling language.

At the same time, there is no implied permanent commitment to one particular way of modeling with the host language.

# Too Many Moving Parts

A common, if somewhat informal, observation about a large code base is that there are “too many moving parts” in it. In my experience, this is especially true for large Java systems but is probably universally true.

## What do we mean by ‘too many moving parts’?

Simply put, there is always a significant semantic gap between a programming language and the program. The larger this gap, the more that has to be expressed in the language, as opposed to simply using it.

For example, consider the problem of traversing a recursive tree structure. In Java, we can iterate over an `Iterable`; structure using a loop, for example, to count elements:

```int count = 0;
for(E el:tree)
{
count++;
}
```

If the `Tree` class did not implement `Iterable` we would be forced to construct an explicit iterator (or worse, write a recursive one-off function):

```int count = 0;
for(Iterator<E> it=tree.iterator(); it.hasNext();)
{
E el = it.next();
count++;
}
```

This version illustrates what happens when a programming language does not quite meet us halfway in our programming task. There is a lot of extra clutter (managing the iterator) that makes it hard to see what is really going on.

In this case, Java’s `for` notation makes it significantly easier to see the program. However, there are many cases where this is not true. For example, you cannot use a similar technique for reading files, searching or removing elements from a tree, etc. etc.

## Language Extensions

One way of reducing clutter is to permit the programmer to extend the language. Of course, the designers of Java set their face against this — there is no macro facility in Java — for a reasonable if misguided reason: to prevent programmers lying to each other.

The Star language does permit language extensions to be introduced by the programmer. This has the effect of encapsulating not only data abstractions but also control abstractions.

For example, to count the elements of a tree in Star, we can do:

```var count:=0;
for E in tree do
count := count+1;
```

The program depends on the programmer implementing the type contract for `_search` and a macro expansion rule:

```# for ?Ptn in ?Exp do ?Act ==>
__search(Exp,procedure(X){ if X matches Ptn then Act})
```

No apologies for the macro definition itself, but the effect is that the language has been lifted into one that fits the requirements more closely. This, in turn, reduces the semantic gap between the language as used by the programmer and the application.

Of course, there is quite a bit more to reducing clutter than macro definitions. However, it should be an important goal of language design to ensure that programmers can express themselves with minimum extraneous concepts.

# Late Binding in Programming Languages

Late binding is key to enhanced productivity in programming languages. I believe that this is the single most important reason why so-called dynamic typed languages are so popular.

This note is part of an ongoing ‘language design’ series which aims to look at some key aspects of programming language design.

## What do we mean by late binding?

Simply put, a programmer should not have to say more than they mean at any particular time.

To see what I mean, consider a function that computes a person’s name from a first and last name. In Star, I can write this:

```fullName(P) is P.firstName()++P.lastName()
```

This constitutes a complete definition of the function: there is no need to declare types; furthermore this function will work with any type that has a first and last name.

Contract this with a typical well-crafted Java solution:

```boolean fullName(Person P){
return P.firstName()+P.lastName();
}
```

Not so different one might argue.

Except that we have had to define a type `Person`; at best this is an interface and not a class. The Java `fullName` method will only work with objects of type `Person`.

In designing the Java version we have to find and/or create a type `Person`. In addition, we must make sure that all classes that we want to compute the full name of implement the `Person` type.

This last aspect can be a real productivity killer. Suppose that we want to be able to compute the full name of many different kinds of objects; then we must arrange them all to implement the `Person` type. This may not even be possible if the classes in question are from a library that you don’t have the source to.

## Late Binding does not mean dynamic typing

One of the common perceptions is that you lose the safety of a type checker if you want to allow late binding. This is not true; at least, it is not true for Star.

Star is a statically typed language which supports a range of type constraints. In the case of the `fullName` function, the constraint is that the type of `P` has the `firstName` and `lastName` attributes. (The details of how this is done are too gory to go into here.)

These constraints can be checked; so, for example, every time the `fullName` function is used on an argument, the checker can verify that the type of the argument is consistent with having a first and last name. This check can be performed at compile-time.

It is even possible (necessary) to go one step further and allow generic functions to use the `fullName` function.

## Other forms of Late Binding

Late binding shows up in other ways. For example, when specifying an imported library of some form or other, it should be possible to declare the requirement for a library in terms of what is needed, rather than the name of the library. I.e., instead of:

```import java.util.List;
import java.util.ArrayList;
```

we should be able to say:

```require List of integer;
```

or some such expression.

The difference is that the former says — in the source code — which package to import whereas the latter merely declares a contract requirement. How the contract is fulfilled is a separate step; one that a smart compiler system may even be able to automate.

(Some language do work this way. Typically LISP language systems organize their modules in terms of requires and provides.)

## Take Away

It can be hard to know what features should go into a programming language. Having a few principles to guide us make the task of designing a language more tractable.

In this case, the message is that programmers should be able to program in terms of their requirements, not the compiler’s. This need not come at the expense of safety or of performance.

# Productivity gains are permanent, Performance losses are temporary

This is the first in a short series of notes about language design. The goal is to identify what is truly important about programming language design.

## It is all about productivity

There are ‘too many moving parts’ in a typical Java program.

Here is a simple straw-man example: iterating over a collection. Before Java 5, the paradigmatic way of iterating over a collection was to use a fragment like:

```for(Iterator it=coll.iterator();it.hasNext();){
SomeType el = (SomeType)it.next();
...
}
```

There is a lot of ‘clutter’ here; but worse, there are a lot of concepts that are not really that important to the problem at hand.

Compare that to the Java 5 version:

```for(SomeType el:coll){
...
}
```

Here we can focus on exactly what is needed: the collection and the iteration.

This kind of example can be repeated fractally up the entire stack of abstractions: not just at this micro level but also at the level of using standard JDK class libraries on through to using features such as Spring/Hibernate or Wicket.

## The elements of productivity

I believe that the list of requirements is not that long:

### Digestibility

A programming language is an artifact, and needs to be understood in order to be effectively used. So, a language with a well crafted structure is going to be easier to learn and use than one with many ill fitting pieces.

Note that this does not mean a small number of pieces (or features). The classic example of this is natural languages (such as English or Spanish). There are at least 500,000 words in the English language. A daunting task for someone wishing to learn the language. But, the grammar of English is relatively simple compared to other languages (such as German or modern Greek). This has helped English to become the dominant second-language globally.

### Semantic Lifting

Structure in software is evident at many levels: in the micro structure of individual fragments of code, to the organization of libraries to the ecosystem of multiple applications.

A powerful tool for managing this complexity is abstraction. Commonly abstraction is thought of as a technique that enables one to ignore inessential details.

But a better concept might be semantic lifting. Semantic lifting is a common technique in Mathematics. For example, vector geometry is layered over cartesian geometry but permits powerful statements to be expressed in a simple way.

A programming language that can support similar techniques for lifting the language — like the iteration example — enables higher productivity.

### Late Binding

Programming languages are often quite fussy in character: requiring all kinds of details to be established quite early in the design process. For example, Java (like most languages) requires that all types have names.

This emphasis on detail makes a significant barrier to developing a program: the programmer is forced to focus on issues that she may not actually be ready or willing to.

A language that supports late binding allows programmers to delay such choices until they are needed.

For example, one reason that type inference is so powerful is that it permits a programmer to program in a ‘type-free’ way while knowing that the compiler will verify the type safety of the program. Establishing types of functions is a detail that can often be left to later. However, ultimately, type inference still ensures that the program is ‘correct’.

### Declarative Semantics

This one is a hard one!

But the fundamental benefits of a declarative semantics arise from the tractability that follows. Having a declarative semantics makes program manipulation of all kinds more straightforward. That, in turn, means that some high-powered transformations — such as those required for scaling on parallel hardware — much easier to accomplish without requiring enormous input from the programmer.

## What does the future language look like?

According to this thesis, a language based on these principles would have a sound semantic foundation, would be easy to understand and would not require more from the programmer than was required by the problem. And it would be easy to deploy on systems consisting of many cores.

What’s not to like? In the subsequent posts I will look at each of these principles in turn in a little greater depth.

# (Software) Architecture = Policy + Mechanism

In the early 1980′s Bob Kowalski made famous an interesting equation: Program = Logic + Control. The idea of that equation was that programming was essentially a combination of logic — i.e., what you wanted done — with algorithm — how you wanted it done.

It is a fairly commonplace fact that any non-trivial program has a similar flavor to it: there is often a substantial amount of machinery that is used to deliver the value in the program; together with some form of policy statement/expression that governs the precise requirements for a particular execution of the program.

The larger the program, the more obvious it is that there is this layering into mechanisms and policies. For example, one could argue that a word processor’s mechanisms are all the pieces need to implement text editing, formatting and so on. If the word processor supports styles, especially named styles, then these styles are a simple form of policy.

At larger scales, when considering networked applications for example, there are often formal languages used to express the different kinds of policy that apply: security policies, management policies and so on.

So, my thesis of the day is that Architecture consists of the specification of the mechanisms together with the specification of the policies that may apply.

Is this useful? Being clear about the `natural divisions’ in a complex structure is the first step in making that structure tractable.