Organizing principles for services

One of the questions that comes up from time to time is how to define your services. This has come up for me in two independent fora: within the OASIS Service Oriented Architecture work and in the context of human provided services, for example at Genietown.

In the work on the SOA Reference Model we decided that “services are the mechanism by which needs and capabilities are brought together”; i.e., its about needs and capabilities to satisfy those needs, and the access mechanism.

However, this still begs the question somewhat. In the domain of human services, where the services are things like “building a home”, “walking the dog”, “taking care of my elderly parents”; it gets even fuzzier. Sometimes a service seems to organized around the person offering the service, for example, an architect, or a doctor. Sometimes the service is organized around a particular kind of product, such as doors or skylights. At other times, the service is organized around a material, such as glazing and paint services. Another kind of organizing principle is location; for example, landscape services involve a whole range of things in the garden.

While it is clear that a service is typically organized around a set of actions, and one or more resources; I do not believe that there is a single ‘syntactic’ condition that can be applied to define a service.

Currently, we do not have a good higher-level handle on how to organize a large suite of resources and actions into well differentiated services. The standard technologist’s answer to all this is to throw the problem ‘upstairs’ and appeal to business function (e.g., accounts, sales, library book management etc.). What that amounts to is asking someone else to come up with the organizing principles for defining a service. It may just be that the business level is the right level to ask these questions.

The Yin and Yang of Actions and Events

Today, in our telcon we invented something. It is not often that we can claim that; but today we did.

There are, it sometimes seems, people who think that everything is an event. There are others (including a lot of philosophers) who think everything is an action. (Well, everything that people are involved with anyway.)

Well, they are wrong.

In human communication, the only way of getting someone to understand something you want them to is by saying something to them: by performing one or more speech actions. In the SOA we interpret this by committing to a view that participants in a service interaction denote the actions to be performed by exchanging messages. I.e., an appropriately formatted message that is effectively communicated between participants counts as an effort to perform an action.

But, not all messages encode actions. Some encode events. For us, an event can be defined as something that happened that someone has an interest in. We can encode a description of an event as a message, in exactly the same way that an action can be so encoded.

So, what is the relationship between events and actions?

If you look at the message that denotes the action, you can call it an event (the event is the performing of the action). On the other hand, traditional speech act theory would encode an event as a speech action: an inform of the change.

Personally, I do not like the smushing together of events and actions in this way: each is a first class idea, not to be subordinated by the other. But they do seem to be very closely connected; almost two sides of a coin — a yin to a yang.

So, we now have two fundamental ‘things’ being communicated: actions and events. Incidentally, an event can be used to communicate a real world effect – one of the cornerstones of the SOA Reference Model. And that is our invention for today.

Cookie cutters and IT fashion

I have had reason to look at the job boards recently. I am struck by something: nearly all the jobs in the IT industry have a sameness to them: there is some list of acronyms that you are supposed to have had x years of experience with (sometimes x is longer than the technology has been available) and apply those acronyms to the job at hand.

There is very little room, it seems to me, for someone who has a deeper experience but who does not fit into the cookie mould.

There are several unfortunate consequences of this (apart from the obvious one): any business that seeks to differentiate themselves from the competition cannot do so effectively if they slavishly follow the deeply rutted road trod by those before them; and some of the hardest problems in the Industry have nothing to do with applying the latest technology to yesterday’s problems.

One thing that I have learned in life (here he puts on his tri-cornered hat) is that there is always someone who will be faster than you, and there will also be people who are slower than you. Instead of trying to win the race, it might be better to think more carefully about what you really need.

It is also true that what you need may not be what is on offer at the local store.

But, it seems to me, the biggest problem in the IT industry is the relationship with its customers: the business community (shorthand for anyone who needs computers to do their work).

The gap between IT and business is famous; famously difficult and famously wide. Yet, you very rarely see ‘able to cross the divide between IT and business’ on a job listing; even on senior ones. That takes something other than acronym soup.

Giving the customers what they want

I do not believe that I am an elitist, but at the same time, I wonder about that phrase. To me, it implies an abdication of responsibility. Which is better: to give the customer what he asks for or to solve the real problem?

Here is what I mean. Occasionally, someone asks me for some tool/gadget/software program that strikes me as not really addressing the problem. This can be for any number of reasons; the customer has an immediate pain point and wants to address the specific requirement, the customer is already fixated on the technology and want that solution, the customer has been told that the answer is SOAP (and what was the question?).

As a professional, that puts me in a dilemma: either I end up arguing with the customer or I hold my nose and give him what he so plainly wants even if I think that it is not the right answer. Given my temperament, it means that I usually end up contradicting the client and thereby losing the deal.

Today I ended up doing that (I think, it may be too early to tell the final outcome). I was confronted with questions about scale, P2P etc. for a project for which I believe the real issues had nothing to do with scale, P2P or whatever technology; but were really about the business context that the solution was planned for.

Personally, I think that if you are going to hire someone for their experience and judgement, you should be prepared to listen to their advice. This is like going to the doctor and demanding that they prescribe you some drug that you had heard of; what you should ideally be doing is presenting them with your symptoms and trusting their judgement. If you don’t trust the doctor, go somewhere else (or go to medical school).

On the other hand, he who pays the piper calls the tune. And it’s a big mistake to be too far ahead of the piper.

Safe and effective software

Someone recently asked me why I was working on the particular topics that I was interested in. I am afraid that in the heat of the moment I had a reasonable but ultimately lame answer (something about reducing friction in the marketplace).

In fact, the true answer is simpler and much more powerful. I want to be part of a ‘professional’ industry, and I believe that we are not really there yet. It is a constant source of amazement to me that there have not been any class action lawsuits against certain high profile software companies.

I like the phrase safe and effective, which describes the basic requirements for medicines of course, but should be equally applicable to software.

What would the benefits of being able to label a system safe and effective? Primarily it means that someone using the system has some assurance that the software will do what it is supposed to do, and that it wont lead you into trouble.

Of course, if you take too many aspirin, or if you misuse a software system, it can hardly fail to cause you grief: so in the end safety and effectiveness is never absolute. But we, as an industry, can do a lot better than we do today.

In the case of services, what does it take to be safe and effective?

I have written before that SOA is about action at a distance. We want to be able to use the Internet to achieve our objectives, to perform tasks. This is fundamental difference between SOA and the Web architecture (for example).

For services to be widely used it will be necessary to be able to show that they are safe and effective too.

Since it of the essence that service is about crossing ownership boundaries, about you using my system, a key piece of figuring out Safety and Effectiveness in relation to services is in being able to answer some simple questions:

1. What is the effect of using the service? (Will it cure my disease?)
2. Do the participants in the service have the appropriate rights in using and delivering the service? (Is the patient an appropriate target for the medicine?)
3. What are the potential side effects of using/offering the service? (What are the complications?)

That is the heart of the problem it seems to me. And, I firmly believe that when we can reliably answer these issues in a systematic way then we will be entitled to call ourselves a profession.

BPM for machines or for people

I have to say that from a Computer Science perspective, I agree with Tom Baeyens, BPMN seems to reverse many of the lessons that we have learned (we have the lumps on the head to prove it). However, I think that this misses the point of BPM. According to Adobe, some 85% of business processes are executed by people. I think that this figure is not likely to change anytime soon.

In effect, this is saying that BPM is a natural descendant of Workflow; except in the modern era where Web services are expected to be plentiful and processes span ownership boundaries.

When it is primarily people that are expected to execute a process, they are significantly less tolerant of constraints such as “don’t do that, it is too hard to get right”.

BPM lives right on the boundary between IT and business. That means that neither ‘side’ gets to dominate the issues and vocabulary. Business Process designers need BPM because they have to get their multitude of processes right. IT architects need BPM because it defines their responsibilities in a relatively unambiguous way. eBusiness process designers need it because ‘getting it right’ for business between partners is too expensive at the moment.

What’s up at OMG

Busy week at OMG as usual. The BPDM team presented the latest news on the status of the BPDM; and it seems to have come a long way in a few months.

There is something of a fracas about the relationship between BPMN and BPDM: is BPMN ‘only’ a notation or does it have some semantics. This whole thing was news to the BPMN team as they (including me) were blithely assuming that we were trying to define a language. For us, the major issues seem to revolve around the execution semantics of a BPMN diagram; for others, it is only a diagram notation and we needn’t worry our little heads about execution. One might guess where that went!

The BPMN effort does seem to be a bit stuck right now. Personally, I think that the issue is that we are trying to have it both ways: have an easily understood execution semantics and allow the business modeler to do whatever and however he/she likes. The image is one of sharp scissors: do we give the modeler sharp scissors in the knowledge that they might cut themselves? In my opinion, there is a difference between having the sharp edge of the scissors on the outside or the inside of the blades. Some of the most basic features of BPMN (particularly the merges) are downright dangerous.

There is also an initiative to develop a UML profile oriented towards modeling SOA systems. Unfortunately I have not had time to look at it; but it should be a good idea.

Rewriting does not work?

I am having a little trouble figuring out the details of graph rewriting semantics for BPMN. The issue is merges (pesky things – should be banned).

In communication, message receive is by far the hardest to get right. It seems that in BPM, merge has a similar role: get merges right and nothing else will be a problem.

Anyway, in traditional graph rewriting, a new ‘activity’ is created by reading-off a new active copy from the original graph. This corresponds to beta substitution in functional languages. The trouble is that, compared to expressions, BPM graphs are decidedly not well formed: there are loops and all sorts without being specially marked.

Well, the issue is, when you copy off a new activity, how do you ensure that the existing activities are properly linked in to the new one. You can see this in the sequence above. (1) is the start point, and the problem shows up by (4). The (inclusive) merge has lots of connections going in, but only some of these are relevant.

You can’t use special markers — that would allow you to copy off bigger chunks of the graph, including subsequent merges from a split — because BPMN does not have them!

Graph rewriting works in functional languages because a well defined chunk of the graph is rewritten at each stage: that chunk is defined by a particular pattern in the graph which is rewritten with a template (equation).

More thinking required I believe. It was such good idea too!

A requirements notation

Recently I have been doing requirements in a few separate places. I got the notion into my head that there was no good notation for describing requirements, particularly non-functional requirements. UML has a diagram for use cases, but no diagram for requirements per se.

So this is the legend of the notation I drew up:

In the Critical Factors Analysis method, your identify three classes of requirements: the goals that you want to achieve, the critical factors and conditions that must be satisfied to achieve the goals and the measurable requirements on the solution that will meet the goal. This methodology is powerful primarily because it can act as a forcing function to ensure completeness of requirements.

I have had difficulty in getting people to understand the difference between goals, CFAs and requirements. I have also had difficulty getting people to distinguish requirements from solutions! (Everyone has their favorite piece of technology that absolutely must be part of the final product.

My take is that requirements are problems to be solved, not solutions. Preferably, these problems are problems that customers have; not problems developers have!

Diagrams like this

can act as very effective indexes into a textual description of the factors that go into a successful project.

Semantics of BPMN

Recently I have been involved with the standardization of BPMN. I find BPMN fascinating because it lies on the precise boundary between the IT world and the business world. Boundaries tend to be uncomfortable places to be around and BPMN is no exception.

My focus has been primarily on the semantics of BPMN. There have been many issues concerning the semantics of BPMN features. From a strictly programming language point of view BPMN seems to throw away most of the hard learned lessons of programming in the last 40 years: it is unstructured, full of gotos and no objects in sight.

But that is to forget the true purpose of BPMN, it is not a programming language. It is not even a scripting language, it is a language to help humans to manage their own activities. Even when BPMN is supported by automation, it is still primarily humans doing the work.

Anyway, the traditional approach to the semantics of languages like BPMN is Petri Nets. The merit of Petri nets is that they are quite simple to understand and quite simple to implement too.

However, BPMN has a bunch of features that are quite difficult to capture in petri nets without getting quite twisted. These are mostly due to the rich variety of merge behaviors supported by BPMN.

A classic one is the pass-through merge: where two streams are merged into one and all the tokens are forwarded. Simple, but combined with parallelism can get you very quickly into trouble:

This diagram, which shows an and-split forking off into three parallel branches, two of which are merged by a pass-through merge and the resulting pair merged back by an and-merge.

Interpreted as a petri net, three tokens originate from the split, two are merged onto the same ‘wire’ and the and-merge merges tokens on its two wires. This will result in a deadlock after the first successful firing of the and-merge because there is still a token left hanging about that is not collected by the and-merge.

The instinctive reaction of an IT person would be to say “don’t do that”; but we are expecting business analysts to use BPMN, not programmers. And, this kind of situation can show up as easily in a large 400+ node diagram as a 3-node diagram.

One ‘fix’ is to allow annotations on the wires that go into an and-merge: specifically how many tokens that wire is supposed to accept. In my view, that is even worse because there may be no way of setting that count reliably, and further, it is low-level hacking of the worst kind: programming by numbers.

Here is another approach: instead of using petri nets, use graph rewriting semantics. In a graph rewriting approach, there are three kinds of nodes and wires: dormant, active and expired. The BPMN diagram becomes the initial graph which is rewritten until the entire graph is expired (or until only all the end-event nodes are active). This is how the pass-through is evaluated using graph rewriting:

The six stages show the evolution of the graph from a single active and-split ending up at the single active and-merge. The green is active, and the gray/purple signifies expired. (Normally you would remove expired entities but I have kept them for illustration.

Essentially, the pass-through node becomes a kind of ‘cloner’, and the dead-lock problem disappears! Neat!

I think that this may well end up to be the way that BPMN’s semantics is defined. Given the experience of the G-machine (used in combinator approaches to functional programming languages) it’s also not a bad way to implement BPMN automation.

Unless, of course, some problem shows up to make it unsuitable after all.