After having examined low-level concepts and implementation specifics of Scalable Enterprise Integration Architectures earlier, this article is exploring the higher-level concepts and how it all (EDA, SOA, REST, Message Queues, Request/Response that is) comes together.
We shall call it ‘Business Orientated Enterprise Integration’ (BOEI).
Baffled or Just NOT Aligned at all?
Those of us who love to play ‘Buzzword Bingo,’ during meetings or conferences, know that the phrase ‘Business and IT Alignment’ scores frequently. I am surprised how many organisations still disregard it after having spoken about it for so long (Maybe we are secretly chasing high scores at Buzzword Bingo still? 😉 )
How do you know whether IT is aligned with business?
Does the organisation have an Enterprise Architecture (EA) practice – or maybe a vision statement from the CEO, with buy-in from senior & middle-management? Or a strategic roadmap that everyone knows about? What about reference architectures, published standards?
If the answer is No to most of those questions above, I doubt you’ve got business and IT alignment, simply because you have no way to execute according to common sets of standards and rules.
TOGAF talks about things like Architecture Repository, Enterprise Continuum and references to industry or organisational standards, patterns, guidelines and rules. Establishing those allow organisations to walk in the same direction as one entity, by reviewing initiatives according to those standards. And Yes, I am optimist enough to assume that people from the business end of town are communicating with the guys with the thick glasses in the dark rooms to discuss those matters together.
Divide & Conquer: Defining your Business Domain
Business-Oriented Enterprise Integration works for organisations of all sizes. Once you reach a certain size or your business is organised naturally in distinct units you might want to consider dividing the whole into more manageable chunks, or Business Domains, as depicted in the picture.
The same is true for M&A scenarios where you either want to sell parts of your business or integrate acquired businesses more easily.
A smaller Super Fund or Retailer might just have a single domain, whereas a globally operating bank can have separate domains split up by country, payments, institutional or retail banking.
Even a ‘Customer’ domain providing a single-view of customer to your organisation or a single-view of your organisation to the customer, is possible.
The major point however is: The Business defines a domain, its capabilities, the systems contained within and hence its boundaries. The business also owns the Domain and the systems, not IT (collaboration models between IT and Business, e.g. running IT as a charged service centrally, project based or as part of a business unit is another topic).
The Core: Semantic Hub
One of the major advantages of the BOEI approach is semantics.
Service Orientation forces core business systems to understand the semantics of the world of the target system or at least the ESB, which is NOT it’s job.
For example:
Core System A only deals with Account ID and Last Name; it then needs to understand how that translates into the ESB (canonical) or worse (because it is point to point) target system world. If canonical services are employed, each Core System Service has a Canonical representation of the very same service on the ESB, which doubles the amount of services (see ‘EDA vs SOA’ for details) and Version Hell has arrived.
The Semantic Hub decouples Source and Target Systems through well-defined message formats and the 3-forms-2-transforms pattern (see John Schlesinger’s ‘The Five Golden Rules of Integration’ for details).
The 3 message formats (forms) are:
- Source System Message
- Canonical Message
- Target System Message
The 2 message transformations occur between the
- Source System Message format to the Canonical Message format
- Canonical Message format to the Target System Message format
It sounds very simple, but let’s be honest: Message modelling is an art that needs to be understood, especially when complex 1-to-Many, Many-to-Many or reverse entity-relationships are modelled (i.e. Customer-to-CustomerAddress or Account-to-Customer and Customer-to-Account).
[Please contact me and share your experience with tooling in this area!]
Message formats are contracts between systems owners (business people that is). Those formats need to be adhered to. The Business Domain owner facilitates the definition of those formats and enforces compliance through the Semantic Hub. The compliance comes naturally because it is the foundation of an operating business.
System Design and System Integration
For all the reasons discussed in ‘Event Driven VS Service Orientated’ the main integration and communication pattern between systems is Event based.
Because Businesses are driven by events. Naturally. Purchase order, invoice, delivery, payments, etc is what makes businesses tick.
BOEI thinking affects system design, because in an Event Driven Approach (EDA) systems are expected to do their job commonly without depending on anyone else.
That means a system’s data model must be able to store all the data necessary. For example if the Account System needs customer data to verify against when accounts are altered, closed or opened then that system must have that data available. A getCustomerData service call to a CRM system would defy the scalability advantage as discussed in ‘EDA VS SOA’.
The downside of this approach is that data might be stored redundantly. This is counter acted by the fact that the business owns the systems and it’s data and hence decides on what data to store plus the level of data quality necessary for the system to function accordingly.
The Fringe: Inter-Domain Communication and Presentational Systems
Inter-Domain communication adheres to the same principles as system-to-system message exchange within a domain (intra-domain).
This means event messages from source systems must be routed through the Semantic Hub before the message reaches an inter-domain gateway. Otherwise it would be a point-to-point integration between system and a domain. And the principle we must adhere to is loose-coupling.
Information retrieval for Presentational or Channel systems is commonly powered through Request-Response type communication. Above picture shows how we take the load off core systems by feeding presentational applications through Information Services. The data repositories that power Information Services are fed by canonical event messages directly from the Semantic Hub. Those data stores can be optimised for read-operations.
REST and JSON further enhance information retrieval through a standardised way to query resources (entities) and reduced response payload.
Functionality (e.g. Internet Banking money transfer) beyond simple information retrieval is implemented by re-using existing message formats from inter-systems integrations as described above.
How is this (more) Business Oriented?
- Systems and domains are organised according to business operations
- Business events are used naturally and not ‘clouded’ trough artificial services.
- Systems and domains do not have to understand the semantics of other interfacing parts of the business with different semantics
- The business owns systems and domains
- Business domain owners enforce message compliance naturally
- Systems do not depend on other systems to do their job
- Core business functionality is decoupled from Channels
Is BOEI different to what you have got at the moment? Then ask yourself: Where is the money going to come from (Vision & Business Architecture Phase within TOGAF) ?
Look at what your current architecture offers and how it restricts or empowers the business (Baseline Architecture). Then define your Target Architectures according to the Vision & Business Architecture. Finally put priorities on the tasks coming out of your Gap Analysis.
That should give you a good understanding of what’s next on your agenda.
Conclusion
The approach described in this article is not new. It’s simply Best-Practices applied across an enterprise, using proven methodologies.
Unfortunately, in quite a number of organisations it seems as if the incumbent software vendors have a strong say on Strategy & Architecture, pushing for the latest software stack. And because most vendors have been very busy in recent years to make their entire software stack SOA ready, they don’t really want to talk about anything other than SOA.
But there was a reason why Message-Oriented-Middleware and Message Queues were invented in the nineties. And they still are a great team! And that’s why we should favour proven and purpose built technology to equip our organisations in the best possible ways.
The BOEI concept hopefully encourages you to have a closer look at your organisation, it’s goals, architectures, systems, data and processes in order to help you choose what will work best for you – instead of blindly following pretty marketing slides.
Selamat Tinggal from GA715 (just about to leave Northern Territory now),
Andy
All Things Architecture 