In the continuing Baby Steps to SOA series, we follow Doug and the IT team behind BuyMyWidget.com as they take steps to renovate their digital asset architecture. Up next is expanding the use of the new services layers to their other applications within the business.   While focus is usually given to revenue-generating applications, the inclusion of other applications into this architecture permits all of the applications to interoperate, sharing data and functionality.  This step allows for full leveraging of all business capabilities within the organization.

The Evolutionary Roadmap

• Step One: Analyze and Plan
• Step Two: Measure It
• Step Three: Three Tiers for the Website
• Step Four: Single Sign-On
• Step Five: The Move to a CMS
• Step Six: Data Services
• Step Seven: Centralizing eCommerce
• Step Eight: Sharing the Business Tier
• Step Nine: Moving beyond the website
• Step Ten: Riding the ESB

Step Nine: Moving beyond the website.

Most organizations have an ERP (Enterprise Resource Planning) or CRM (Customer Relationship Management) system sitting inside their network, starving for knowledge about all the transactions occurring on their eCommerce website.  The sales team likely wants to pull in new possible contacts, have dashboards to monitor sales on the site, and drive marketing campaigns based on user activity.

Earlier, when the team was implementing a CMS, I suggested the use of a CMS known as Sitecore.  Some of its benefits is that the marketing can happen right there in the CMS, instead of needing to integrate back to the ERP/CRM systems.  However, it doesn’t do everything.  So even if you have implemented a CMS that is handling some of your campaigns and personalization, you will still likely need to keep using those back-end systems.

This is where the work of leveraging the built-up service layer comes into play.

Leveraging website assets

The first step is to begin updating the existing back-end systems to be able to extract data from other sources than their own database.  Generally, this means updating any reports to pull a merged set of data:  the internal data as well as that from the website.

The downside to this approach is that there are usually duplicate records between the two systems.  Contacts that have come to the website and have already been entered in the CRM, orders that have been tracked in the ERP and are also in the transaction table for the website orders, etc.  Resolving these duplicates is not always trivial, as different systems will have different unique identifiers, which may not match.

So, in order to use this data, the team needs to migrate any data from the CRM/ERP into the centralized data source used by the website, and identify potential duplicates using an algorithm.  Once the centralized data has been cleaned, the CRM/ERP system can be switched to pull data from this centralized data source.

Aside from data, business logic is also likely required.  Common business actions on the website, such as submitting an order, may also be made by support team using the internal systems.  These internal business transaction functions need to be updated to use the centralized business services developed for the website, thus ensuring that all transaction points behave the same and can be centrally managed.

During this step, it is often recognized that users of the back-end systems are allowed to do more than website users, or perhaps bypass certain restrictions.  A new version of the service endpoint will likely be needed that can support all of the needs of the back-end users, and the website may need to be updated to use a new schema for the service.

Leveraging CRM/ERP assets

While most of the integration needs will likely be to tie website data and business functionality into the back-end systems, it is possible that the website also needs to leverage data and functionality from the CRM/ERP.  One of these systems may be the system of record for store locations, or revenue numbers across the organization, or for business account information.  These are prime candidates for new data services pulling from the back-end system.  Once they are made available, the website can begin consuming and presenting this information as needed.

Additionally, these back-end systems often have business processes built into them, especially in an ERP system.  Business rules in these systems can be difficult to surface, and centralizing them into a business service endpoint may be very costly.  However, if the investment is made, a duplication of the rules is no longer required for website use and both systems can make use of the same business rules.

Third Party Systems

A very significant hurdle to overcome is the use of third-party systems.  Very few CRMs or ERPs are completely home-grown, and have usually been purchased from a vendor and then customized.  These customizations may have been done by the vendor themselves in order to preserve an upgrade path, meaning the source logic may not be available to the team.

While some systems will provide integration points in order to support teams wishing to leverage the data and functionality of these systems, these endpoints are usually limited and require additional effort to reach specific business needs.  Teams will likely need to work with the vendor to determine how and where customizations can be made to support their SOA needs.

In Our Scenario

Way back at the beginning of the process, the BuyMyWidget team sat down together to analyze and plan out the systems they would tackle.  At the time, they had determined specific systems that had a high-level of dependency:

Through the various steps, Doug and Lynn have led their team to implement SSO as a step to replace the Identity Sync.  They have also updated their website to centralize the data and business processes of their websites, including eCommerce logic surrounding payment transactions and orders.  This leaves their ERP and CRM systems as the final systems that need to be updated.

The first step is to update the ERP and CRM systems to no longer use custom identity sources that are synchronized, and instead leverage the new SSO system that has been constructed.  The ERP system is mostly customized, and the team has the source for most of it, so the work here is not difficult.  With a built-in SSO support, Lynn’s team can leverage the new SSO system easily.

The CRM system is more difficult.  This third-party system has limited integration capabilities, and does not support SSO out of the box.  Doug contacts the vendor to see what options there are, and the vendor informs him that the current version of the CRM will support this out of the box.  Having heard this before from other vendors, Doug is not entirely convinced, but back at the beginning of the project he also learned that the CRM version the team currently had was reaching end-of-life within the next few years.  At the time, that had been 4 years away, but after more than two years of effort on their current systems, this end-of-life date is rapidly approaching.  Doug decides to take the plunge to update now and support the integration efforts while also meeting his software lifecycle needs.

Having leveraged the SSO system, end-user flow between all systems is now much more fluid.  The team then tackles centralizing the data from these systems into centralized data services, as they had done with the website previously.  The new version of the CRM comes with some of these already, so the team is able to save some time by just using the existing ones instead of building new ones.

At the moment, the website is only leveraging data from these back-end systems, so the team can stop with their services at this point. It is likely that future projects will involve leveraging some of the business processes, but the team will defer this work until it is requested by the business organization.

How much will this step cost?

Because of the likelihood that third-party systems used internally likely have limited or no integration capabilities, this step can be very costly.  Most organizations will likely need to build customizations or pay their third-party vendors to construct something to speak to their other layers.  A fairly substantial project team will be needed to implement service endpoints for these applications, as well as customize the applications to use data from the previously built services.

For cost-savings, an organization can avoid direct real-time integrations and opt instead for building a custom application that surfaces data extracted via a regular import/export.  While this does allow for data-sharing, it does not allow for the third-party application to share business functionality or to use any of the services from other applications.

What’s next?

This series continues with the team pulling all of the services together via an Enterprise Service Bus.