Designing Layered Decisions-based Systems
An Architectural Approach to Systems Change

 
Business rules are a perennial hot topic for a few obvious reasons: businesses are hard to model, as soon as they are modeled they change, and changing the model is expensive and error prone. The promise of rules environments is that systems can change easily and cheaply. In theory, business users close the feedback loop by setting rules directly. Rules are applied at a high level of the system. In this article we will question if this high-level approach is sufficient and outline a more systemic approach to enabling business change.

Any information system is simply a reflection of the business it supports. If the developers building the system are effective requirements analysts the system will meet current objectives. When the business goals change the developers must change the system. Since neither the original system developers nor the organization's best developers are likely tasked with 'sustaining engineering’, updates to the system will be less expertly performed. If the system is not built with business flexibility as a main concern the update is likely to be nasty, brutish, and long. If, on the other hand, the system is built with real business flexibility as an objective the developers will make short work of updates and the business will continue uninterrupted.

So how do we achieve this? The clear choice is business rules, but what do we mean by that? The usual way of thinking about rules is as a set of if-then statements. Often rules environments are packaged with a GUI that enables non-programmers to manipulate flowchart-like structures. Rules engines excel at handling incremental changes in business conditions within a well-known scope. This type of explicit business rule sits high in the decisions food chain, most often in the dynamic top layer of an information system.

Can a high-level layer of business rules alone provide real, lasting flexibility to a complex system? In the most cases the answer is no. How well do rules engines do when a company is redefining its market or changing businesses? And yet change on this scale is not uncommon in the lifetime of an information system. Systems tend to be long lived. The rules that make a certain business operation possible, say catalog and cart eCommerce, will not apply to transactions in the next emerging market 18 months later, say syndication. Even if a company does not expect to change its practices it will be affected by more dynamic partners in its supply chain. Simple rules environments can be layered on to a system to provide flexibility within a set scope. But when change becomes more fundamental a simple high-level rules change clearly does not suffice.

Lets take the eCommerce business of a book seller. The eCommerce group starts by building a system that permits registered users to search a catalog and preview selections from books. After six months the market starts to change and the systems group gets into gear. The product evolves from simple previews to on-line purchases from the catalog. Change continues and customers begin compiling custom 'books' from sources for download in PDF format. Then, just two years after the catalog debut, the market demands custom printing of the custom built books through a third party with next day shipping and the same high level of branded customer support. System functions evolve in every area from inventory and site management to logistics and customer service.

Analogs to these rapid business model permutations are familiar to a wide range of industries. Facing a business change challenge of this magnitude, one choice would be to throw up one's hands and start building a new system. In our example the bookseller's business model changes three times in two years. On that schedule there is no time for ground-up rebuilds. Beyond the time constraint, what company can afford to treat major information systems like disposable cameras? Another alternative is to let the existing system define the business. Clearly that option is unacceptable. Finally, and most commonly, the systems group can re engineer the old system to fit the new business. Frequently this is an over long, painful process. But if the old system were originally constructed with change in mind the reengineering would be minimal and change would be quick. The renewed system would be new only in the functions and rules unique to the new business.

The way to build a change enabled information system is simple: pay attention to the decision points. A decision point is a point in the design or implementation of the system where an architect or engineer makes a limiting decision. The limit enforced at the decision point may have wide impact or be subtle, but like a virtual decision tree, each decision point passed determines subsequent decisions to be made. All decision points set business rules—explicitly or implicitly, flexibly or inflexibly, singly or in combination. The trick is to identify decision points, isolate the most significant ones, and keep them flexible. Identification of decision points means breaking down decisions, recognizing assumptions, and asking more questions. When each decision point is identified the system designers should attempt to isolate the implementation of that decision from other decisions for clarity’s sake and to simplify implementation and change. Flexibility comes when the identified and isolated decision point is implemented as a choice point.

A choice point is just a dynamic decision point. A decision point that is set once and cannot change does not embody a business rule so much as a business condition. A choice is a tool; a condition is just a fact of life. Choice points can be set and reset to apply particular rules. A rule is an effective tool only if it changes relatively easily. In Figure 1 we see choice points represented as blocks in the layers of a runtime architecture. These layers exist to provide separation of concern, interface contracts, and deployment flexibility. As you progress down through its layers the typical information system is more monolithic—fewer decision points are implemented as choice points. In comparison, the layers of a decisions-based system have an increased population of active choice points. Each business affecting decision can be revisited with comparative ease. Each choice point is settable at build, configuration, or runtime. Simple. Compared to conventional software a decisions-based system puts architectural focus on the simplicity of change.

Figure 1. Choice points tend to decrease at each lower layer of a system, decreasing the system's ability to adapt to business needs.

 
Decisions-based system layers

The approach to system layers here is different from a more general architectural view focused on runtime. We identify decision points across the lifecycle. It is important to identify decision points early on in the least flexible parts of the design. Once a decision point is identified a change friendly pattern is generally easy to find and fit. The rigidity of data and objects gives the framework strength, but the beams should be bolted not welded together. Adding the lifecycle dimension into design is appropriate when long term system flexibility and total cost of change outweigh the dynamics of that change. Despite differences of approach using the decisions-based perspective should present no conflict with runtime-based architectural styles.

Assuming a general business system, lets categorize the choice points typically available to decisions-based development. From more dynamic to more static and from late-bound to design-time, the layers are:

Process management The capability to orchestrate one or more processes as a unit interacting with other units and other systems. Example choice point: provide a high-level interface permitting authorized users to schedule events, setup event listeners, and transform events into commands. Process interface The granularity of processes and their accessibility. Example choice point: enable configuration of available processes by changes to deployment descriptors. Process internals The rules set to explicitly control a particular business operation. Example choice point: permit changes in operation sequencing by simply changing a set of properties bound to a rule name. System configuration and administration The default configurations, mappings, scripts, permissions, clusters, etc. Example choice point: bind the XML representation of an object at runtime by mapping well-known names to elements of a schema. System build and deploy The capabilities and defaults built into the production system and the tools that produce the particular build. Example choice point: enable the system to be built for small footprint distributed operation rather than as a monolithic deployment. System behaviors Common behaviors and cross cutting concerns, especially where these are abstracted away from the object model. Example choice point: handle all event generation in an aspect that can be compiled in or not, applied automatically to new code as it enters the code base, and that can be runtime configured per each event generation point identified by method signature. System components The inner services that act on and with the data. Example choice point: hide object persistence behind a general interface permitting new storage systems and new data objects. System data and algorithms The data model and object model and the functions most closely tied to them. Example choice point: choose to identify event objects by property rather than by type to facilitate new events.

Figure 2, below, groups the eight layers into implementation and configuration. As we said, for simplicity's sake lets leave aside the question of what department modifies a system and focus on total flexibility. The layers described above are listed in descending order of malleability. Generally the lower you go the more the cost of a modification. However, the shallower the decisions-based design stack the more expensive it is to cross into a deeper layer. Each layer builds on the layers below it. If the upper configuration layers sit on top of an inflexible foundation, modifications outside the original business scope will be blocked at the lowest rules enabled layer. At this point the inactive decision points in the lower layers become a significant barrier to system, and business, change.

Figure 2. Decision points are categorized across both development lifecycle and the runtime architecture.

Similarly, in general the lower the layer the less effort should be necessary to take a decisions-based approach during design. As an example, consider the relative difficulty of following patterns like factory or facade (see Gamma, et. al.) versus the greater complexity of binding behaviors to pointcuts (see AspectJ docs) versus the still greater complexity of binding actions to conditions in a business rules engine. Notice that the difficulty of changing a decision is not necessarily less at lower levels; rather, the difficulty of structuring a decision point to enable change is less. This expectation of decreasing simplicity fits with the idea that higher levels, building off those below, should be simpler to modify beyond the original scope of business. However, as we have said, the limits of a higher level's flexibility are determined by choices made in the levels below.

The explicit design of a choice point can be thought of as lying at a point in a cube, see Figure 3, below. Points at front lower left (FLL) are simpler, less dynamic and less costly. Points at rear upper right (RUR) are more complex, nonlinear, dynamic and more costly. The former space is analogous to a script and the latter to an expert system. While it is more likely that points at lower levels of a system are in the FLL space this is not necessarily desirable. One thing that is not shown in Figure 3 is degree of flexibility. Total cost of ownership is the question—cost of change and cost of operations are just a couple of the numbers factored in to TCO. Specific system requirements should determine the coordinates of a choice point, rather than the well-known biases of tool venders, Unix geeks or business users.

Figure 3. Implementation approaches to rules categorized along three axes.

 
Architectural Trade-offs

Every architectural approach entails trade-offs—the decisions-based approach is no exception. The main downsides and risks include:

• Additional configuration complexity
• Testing choice points added to testing the choices
• Risk of greater development time
• Risk of additional design complexity
• Risk of poor performance at choice points
• Risk that the choice points will never be used

In the end the situation is a common one: you get what you pay for. Every system capability costs something and which gets bought is just a question of priorities. It sounds simplistic, but the ‘good’ in ‘good software’ is free if total cost of ownership is accurately measured or if all software is required to be good. More to our point, if a company, like our bookseller example above, is looking at developing a critical system with a long expected life in a competitive industry, the decision seems clear—get ready for change.

Despite the trade-offs a decisions-based architectural approach is the correct way to design many projects. With experience the development cost of structuring decision points at all layers across the development lifecycle drops. This design growth happens in parallel to an organization's maturing in its ability to capture and analyze business rules. Fundamentally, a decisions based architectural approach to systems is less business driven and more a matter of careful, defensive systems design. However, regardless of the level of sophistication the business has about business rules, the benefits of a decisions-based system accrue to both the business and the systems team when the industry stands on its head and rules of the business abruptly change.

 
Many thanks to Ryan Scharer for his insightful critique of this paper.