Enterprise Application Integration
Integration has always been a much-used word in systems circles. Unfortunately it has often remained just a much-used word, and not become a real, practical approach to system design. The ERP (Enterprise Resource Planning) offerings have shown us the way, but these are often so large, unwieldy and inflexible that integration stifles rather than releases the effective flow of information.
Now it seems that a new generation of enterprise application integration (EAI) tools is moving us nearer to a true ‘plug-and-play’ systems approach offering the speed and flexibility especially necessary for success in the e-world.
As ever with IT, the promise is, as yet, stronger than the reality and they are only moving towards being ‘plug and play’. As yet, you’ll find yourself writing significant amounts of code to create the linkages. It is possible to adopt a single tool to link a range of disparate systems but interface work will be needed for most of the links. The very flexibility of the tools and their ability to handle a vast range of data and message types means that there is a lot to learn and discovering all the abilities and nuances of any one of the tools requires a lot of training and practice. So, although they are not complex conceptually, in practical terms there is a lot of complex detail to unravel.
The kind of application that makes this complexity worthwhile is, for example, linking systems to automate the flow of documentation accompanying shipments from a distribution depot. By linking all systems to a central hub, an organisation can let staff access data on any customer anywhere in the world. A traditional approach would be to replicate a master system in each distribution location but some of these may be too small to justify that. Besides, each location might need a local component to handle local customs and tax regulations.
EAI allows the organisation to leave legacy systems in place but to overlay them with an EAI approach that can take data out of the systems, process it and send it to the right destination system.
Derry Newman, IT manager of Sony Broadcast and Professional Europe, believes a single EAI tool can meet all a company’s needs. He says Sony is using just the Crossworlds tool because EAI tools demand implementation staff with skill in both the tool and the target systems and are therefore expensive and scarce. Using multiple integration teechnologies makes it harder and more expensive to hire the necessary skills and increases a company’s dependence on the one or two people who know how to use each tool. Sony makes and distributes audio-visual products to TV production companies and corporate users. It has just finished implementing integrated in-house SAP systems and now wants to link them to its channel partners and customers. With channel partners, they aim to take cost and time out of the supply chain, while Newman sees linking to corporate customers as a strategic advantage. “The easier we are to do business with, the more ‘sticky’ a supplier we become,” he explains.
Sony chose Crossworlds because it has pre-built integration modules or connectors for SAP and for other applications. “We didn’t and still don’t entirely know what we may have to connect to, so we need something that is flexible,” says Newman. “The IT team also felt Crossworlds would require relatively few in-house resources to get up and running with each partner. That’s been borne out with our first implementation, with General Electric in France, where it took just five weeks to get from our first meeting to passing messages successfully.”
Newman admits there are drawbacks to using a single tool. “You don’t necessarily have the optimum solution,” he points out. “We’ve already had to buy an upgrade to get more connectors as we encountered an unexpected system in a trading partner. Having said that, we’ve yet to run into an integration need where we haven’t felt in a position to start discussing and planning. I can go to meetings with corporate customers and channel partners and not worry about what flavour of IT they’ve got.” With EAI you need to understand the problem before you select the tool to use. There is no one magic solution but selecting one primary EAI tool will help provide consistency across multiple integration projects.
CHOOSING AN EAI TOOL
Just ask these simple questions:
- What applications, interfaces, message formats and technologies are currently supported and what further enhancements are planned?
- How easy will it be to adapt integration projects to changes in business or technology?
What developer skills and resources will be required? How do these relate to what you already have? - Does the tool have a visual or rules-based interface which will allow it to be used by end-users?
- What support and service does the supplier itself offer and what partnering arrangements and training programmes does it have in place with systems integrators?
March 2001
Promenix, Inc., a systems integrator focused on Enterprise Application Integration (EAI), has introduced an expanded service offering for customers using older generation integration broker technology. When a company is stranded with technologies that are no longer supported by the software vendor, this offering is designed to help them move to the latest versions of IBM’s integration technology, WebSphere MQ Integrator. The service offering provides relief to companies by providing a migration path from the old to the new. IBM WebSphere MQ Integrator is a part of the MQSeries family that provides business integration and business process management (BPM) functions to the IBM WebSphere software platform for e-business. It increases the efficiency of processes such as supply chain management, enterprise resource planning, mergers and acquisitions and straight-through processing. Previously it was known as IBM MQSeries Integrator.
For more information, visit the Promenix World Wide Web site at http://www.promenix.com/.
May 2002
Butler Group, Europe’s leading independent IT analyst has launched EAI and Web Services – Cutting the cost of Enterprise Integration. This Technology Evaluation and Comparison Report reveals that the majority of EAI vendors still have a long way to go before they can offer organisations the benefits of a truly integrated enterprise.
Integration is the number one priority for CIOs, but this has yet to be turned into significant spend on specialist software. This Report examines this conundrum and investigates the various different styles of integration on offer, and discusses both the drivers for and barriers to integration. The report includes detailed Technology Audits on the products from leading EAI vendors and identifies product strengths and weaknesses. Vendors reviewed include: SeeBeyond’s Business Integration Suite, TIBCO Software and its ActiveEnterprise, IBM Websphere, Microsoft’s BizTalk Server 2002 and Global Exchange Services’ Enterprise System.
According to Ian Charlesworth, Senior Researcher at Butler Group: “The challenges or business pains that EAI aims to soothe have never been greater. Organisations are facing well-documented economic pressures that resonate heavily throughout the fabric of the business, creating new pressures whilst amplifying existing inefficiencies. This Report is designed to be an aid to those organisations looking for a suitable integration provider. It clearly points to those that are worth looking at and those that are to be avoided.”
The main aim of this report is to provide a comparison between the technology products in the EAI market and it looks at the different approaches to integration, and the strengths and weaknesses of the vendor’s offerings. The top score available was 60 marks, spread across six categories:
- BPM,
- Communications Middleware,
- Transformation & Formatting,
- Application Connectivity,
- Operations and
- Web Services.
Although the highest score is just over a respectable 75% of the potential score, Butler Group stresses that integration vendors still have a long way to go before delivering what the market requires.
Ian Charlesworth continued: “In Butler Group’s opinion, the SeeBeyond and TIBCO products currently excel. SeeBeyond has developed some excellent technology and has an extremely progressive and ambitious strategy that warrants its position. TIBCO has the technical capability married with marketing savvy and execution strategy to maintain its leading position in the marketplace for some time to come. The financial resistance and strength of the company will ensure that it endures these times of organisational frugality.”
The Report cites Mercator, BEA Systems and Global eXchange Services (GXS) as playing catch up. Whilst BEA Systems has the technical know how to catch up, GXS will need to make an acquisition to complement its existing technology to make it a serious integration contender.
Ian Charlesworth concluded: “The results of the Butler Group Report will naturally be encouraging for some and disappointing for others. Perhaps one of the biggest surprises is the excellent position of Sybase. IONA, on the other hand, finished in a disappointing position. IONA has a number of technical issues that need to be addressed first before it becomes a dominant force in the market. The methodology forces inescapable conclusions, which adds to the veracity and value of the product and ultimately, can only help the reader in making a valued decision.”
October 2002
BEA has launched Liquid Data for WebLogic aimed at extending functionality into the space occupied by EAI vendors. Liquid Data enables developers to source data from disparate databases and applications in real time to provide users with a single, consistent view of the data they need. The company aims to drive convergence of the application server, data integration and business process integration markets. They claim that the product differs from ‘low level’ EAI tools since it allows the data to be read in real time; most tools ‘dump’ the data into another application.
January 2003
Choosing the correct middleware for EAI
Hand-built interfaces do work, but when the number of applications to be integrated increases beyond a handful, a message-based IT architecture may be a better answer. Examples include:
IBM Websphere MQ
Formerly known as the MQSeries, Websphere MQ provides messaging functions for servers and clients and assures once-only message delivery. This is important because if a message were resent by mistake it could lead to, for example, a duplicate order on the receiving application.
Websphere MQ is one of the most common messaging services and it is supported by most EAI suppliers, as well as forming the backbone of IBM’s integration suite. IBM suggests it is scalable to a throughput of more than 250 million messages per day. It runs on a wide range of platforms, including Unix, Windows and IBM operating systems such as OS/390 for the mainframe, and OS/400 for the iSeries.
Websphere MQ can also generate acknowledgements that a message has been received, making it suitable for B2B environments. It supports transactional messaging, which means that operations on messages can be grouped into units of work, or transactions which may affect several tables in a database. If they all succeed, the whole transaction can then be committed but if there is a problem, the whole transaction can be cancelled. This ensures that data is always in a consistent state. It also provides a mechanism to trigger applications to start when certain conditions are met.
Microsoft Message Queuing
Of course, Microsoft has a solution! MSMQ provides guaranteed message delivery, efficient routing, security, and priority-based messaging, and it can be used for both synchronous and asynchronous scenarios.
It is fully integrated with other Windows features such as the Windows Web Server, Microsoft Internet Information Server, and the Windows 2000 security environment. The most recent version, MSMQ 3.0, is specifically designed for the Windows XP environment.
MSMQ uses a store-and-for-ward architecture, whereby messages are stored in a database in case the receiving party is not available to consume them. It uses internal sequence numbering and delivery notifications to recover from situations where messages get lost because of interruptions in the network.
Triggers allow an application to be automatically invoked based on message arrival. A trigger is associated with specific monitored queue, and is invoked for every message that arrives on the queue. A rule associated with that trigger then defines conditions that must be met and actions that should be taken.
Java Message Service
JMS is a supplier-agnostic API set developed by Sun in co-operation with leading enterprise messaging suppliers, including Oracle, BEA Systems and Tibco- these suppliers are then licensed to use the technology in their products.
The original JMS specification was published in 1998, and the latest version, 1.0.2b, was released in August 2001. It is part of the JAVA 2 Enterprise Edition suite.
The original purpose of the JMS API was to enable Java applications to access existing message-oriented systems, such as IBM’s Websphere MQ, although it can be used to provide a complete messaging capability and provides most of the functionality of other messaging systems.
JMS provides a simple way of transporting XML and other data between applications. It is a more sophisticated transport mechanism than HTTP and SMTP, in that it includes persistent messaging, internal acknowledgement of message receipt, and rules of engagement regarding transactional recovery and message redelivery.
It helps to provide reliable, asynchronous communications between different components in a distributed computing environment, including between J2EE components and legacy systems. The use of an Enterprise Java Beans container architecture in conjunction with JMS allows the concurrent consumption of messages. JMS can support both point-to-point and publish/subscribe forms of messaging, although not all implementations of the specification will provide both.
While messaging products are inherently asynchronous, in that there is no basic timing dependency between the creation and the consumption of a message, JMS allows messages to be consumed in a synchronous manner by the subscribing party explicitly calling for the message. By doing so, further activity can be blocked until the message arrives or it can time-out if the message does not arrive within a specific time limit.
Asynchronous messaging is provided by the use of a message listener, which waits for a message to arrive and then acts on its contents. JMS uses the Java Naming and Directory Interface to identify the topic of the message in publish/subscribe environment.
Tibco
Tibco uses the Information Bus (Tib) to transport messages between applications within its Tibco Rendezvous messaging solution. The company has other messaging options, acquired with its purchase of Talarian earlier this year, as well as a JMS option. Rendezvous is one of the leading proprietary messaging services, with more than 1,000 installations worldwide. It uses a distributed architecture to avoid bottlenecks and single points of failure.
Applications connecting to it can choose to encrypt messages, and select the most appropriate quality of service. Messaging can be request/reply or publish/subscribe, and can be asynchronous or synchronous. Messages can be sent over local networks or via the internet. Rendezvous messages are self-describing and can be used on many platforms, with a user-extensible type system providing support for various data formats, including XML.
Content-based routing
Most messaging services route the message to its destination based either on information held within the message header or based on the topic of the message in a publish/subscribe service. Intelligent, or content-based, routing builds on messaging services, usually within a message broker, although there are standalone operations on the market.
Based on the content of the message, it may need to be routed to different end points in the wider infrastructure. In a typical application, a message is routed by opening it up and applying a set of rules to its content to determine the parties interested in its content. Content-based routing is often dependent on the message being constructed in XML, and will usually be built on top of MOM or JMS-based messaging.
This type of service is not typically included in basic messaging systems, but may be included in some of the higher-level EAI solutions.
Choosing?
Each type of messaging middleware has its strengths and weaknesses, but two trends dominate: messaging and application servers. JMS in particular has led to a new generation of message-based applications, building on the substantial installed based of IBM Websphere MQ, Tibco Rendezvous and similar products.
Message brokers have become the core of many EAI products, and offer a widening range of capabilities. Alternatively, by building integration solutions on top of an application server, companies can benefit from the reliability, fault tolerance and standards-based approach these platforms offer.
The most appropriate middleware will depend on the type of integration required. With most organisations still unfortunately tackling integration on a project-by-project basis, it may be appropriate to select a cheap and cheerful approach where the benefits of integration do not outweigh the cost of the larger integration brokers or platform solutions.
True EAI solutions come into their own when an overall strategy of integration between the majority of an organisation’s applications has been agreed.
May 2003