In the continuing Baby Steps to SOA series, we follow Doug and his IT team behind BuyMyWidget.com as they take steps to renovate their digital asset architecture. Previously, we introduced the problem and the team, started planning and analysis, decided on some metrics, and now we continue on our travel through the road map with ensuring separation of our logical layers in the website!

The Evolutionary Roadmap

• Step One: Analyze and Plan
• Step Two: Measure It
• Step Three: Three Tiers for the Website
• Step Four: Single Sign-On
• Step Five: The Move to a CMS
• Step Six: Data Services
• Step Seven: Centralizing eCommerce
• Step Eight: Sharing the Business Tier
• Step Nine: Moving beyond the website
• Step Ten: Riding the ESB

Step Three: Three Tiers for the Website

If you are part of a team that has made the decision to move towards a service-oriented architecture, then you are likely already familiar with the concept of a three-tier/multi-tier architecture.  This architecture allows us to separate our business logic, our data management, and our presentation to the client.

Unfortunately, when refactoring our digital assets, we often find that one or more of the applications have not been constructed in this way.  When the logic and data layers have been interwoven into the presentation tier, this makes it incredibly difficult to introduce a service-oriented architecture.  SoA typically contains both data layer and business layer logic, without presentation, so in order to migrate we need to be able to separate these layers from each other.

This refactoring can be done as part of a targeted big-bang project, or on a gradual basis as part of a continuous improvement project.  This will depend on your team’s availability to perform the work.

Preparing for a CMS

In addition to working towards SoA for our business layer and data layers, our path is also introducing a CMS to remove content from the digital assets and place it inside its own service or application.  The CMS typically resides as part of the presentation layer, while also providing some marketing business functions as well as a content data layer.  This means the separation of the layers into our three tiers is equally important for the introduction of the CMS, as it needs to be able to integrate with the presentation layer without being interfered with by business logic and data layers for other systems.

There are a few options that can be taken here.  For a CMS, the prime focus is usually on the presentation layer.  This means that we can delay separating the business and data tiers from each other, so long as we separate the presentation tier.  This is one way to reduce the initial effort prior to the CMS introduction, although the effort will still be needed later.

Additionally, if your website is like many where I’ve introduced a CMS, the presentation layer is so intertwined with business and data calls that the separation of the presentation tier is usually the largest part of the overall effort.

Big Bang vs. Continuous Improvement

Of the many project planning options available for preparing for a CMS, and eventual SoA architecture, I recommend using either a Big Bang (all or nothing) or a Continuous Improvement (iterative development) approach.

The Big Bang Theory

From a project planning stand-point, a Big Bang approach whereby the application is completely refactored into using a CMS and SoA architecture as one big rewrite will have the least amount of overall cost.  Unfortunately, these projects usually take quite a bit of time in one chunk, and due to the large batch of work involved this usually introduces a significant amount of risk to the timeline and cost of the project.  The likelihood of accurately estimating a project’s size at this complexity level is very slim.  That being said, if resources are not an issue and there is no fixed date for launch, this will be the most cost-effective solution.  There is very little waste as the team will only deliver the final desired solution without interim steps.

Reality Check

However, in my experience, business needs change often and resources are scarce.  This means taking two or three years to complete a massive project is often impossible.  Developers are needed for pressing bug fixes, product teams are needed to adjust requirements or introduce new features, IT budgets need to be re-allocated due to unforeseen problems with existing infrastructure, etc.  For this reason I recommend a Continuous Improvement model.

With this model the team can focus on delivering an incrementally better solution and introduce new systems as their asset is ready.  There is very little risk due to the small amount of change being delivered at each iterative step, and this work can even be done in parallel or as part of other projects.   However, there is usually very little business value being delivered for the cost.  If we are refactoring the code for a future project, we are spending money now to get something later.  If this work is being embedded in ongoing maintenance projects, this will make the maintenance project reports display an increased cost.  For this reason, I would recommend that the time spent on the refactoring be logged against the future project, even if it is delivered in parallel with other projects.  This will allow for a more accurate calculation of the true cost of both maintenance and the refactoring.

Baby steps won’t win a race

There is another downside to the Continuous Improvement model.  The overall calendar time between when the team starts refactoring and when the team is ready for a CMS/SoA solution will be longer than the Big Bang approach.  The risk has been reduced, the maintenance team will gain benefits of better tier distribution faster, and the team is available for other responsibilities, but the “slow cleanup” requires extra calendar time.  These “baby steps” allow us to get to where we need to be with increased agility and reduced risk, but it will be difficult to know when this will be complete without some sort of target date and release plan.

In Our Scenario

Doug has already identified numerous projects and ongoing problems that are going to keep his team busy:

As if that weren’t enough to deal with this week, a recent viral marketing campaign has increased traffic to the primary ecommerce website well beyond anything the team has seen before. Customers are complaining about slow load-times and dropped order transactions. Dealing with this is going to consume a lot of the team on an ongoing basis, which means there is no time to just stop everything and execute the other projects immediately.

Lynn can probably focus her time on the new architecture, but very few of the rest of the development team will be available on an ongoing basis.  However, Doug has promised he can make two of the developers (Raj and Sandi) available at 50% for ongoing preparations.

Raj and Sandi are both senior developers with experience in most areas of the system, and would usually be involved in any new R&D project.  However, because of their experience and knowledge of the systems, they are also needed for critical issues such as the current eCommerce site problems happening at BuyAWidget.com.

Knowing the system so well, Raj and Sandi inform Lynn that if they work on the order transaction layer first, and refactor that out of the presentation layer, that will allow them to also more easily deal with the ongoing performance issues.  The team agrees to tackle that area first.  Since this is one of the last and most important steps to the user, given that payments are being processed, this will yield the best initial investment.

Once complete, the group plans to tackle the checkout process which leads up to the transactions, followed by the cart process, and finally the product listing of the widgets.  This should cover the majority of the business logic and data layer code that is currently interwoven into their website presentation layer.  With some estimates in place, Lynn can report back to Doug on the plan for the first run of refactorings and provide an estimated timeline.  Lynn suggests that they re-plan next steps closer to the end of this initial refactoring.

How much will this step cost?

The cost of refactoring will depend on the following factors:

1. Experience of refactoring team
2. Degree to which tiers are interlaced
3. Complexity of the site

Experience

To mitigate cost, I recommend assigning a senior resource to the team who is very experienced with the system being refactored.  If this person is also well experienced in tier architecture, this will decrease the risk as well as the cost.  This will mean that the time incurred will need to be at the cost of a senior development resource, but the benefits outweight this additional cost.

Interlaced Tiers

If the tiers are not heavily interlaced and are already well-architected, I would advise against spending budget on this step and moving directly to the next phase.  If there are only a few minor adjustments needed, this can be done as part of another project.

Complexity

For a simple website that has only two or three transactional areas where business logic or data management is required, this can likely be done by a single senior developer in about 2 months at 50%, assuming two week cycles for each refactoring group.  Regression testing will obviously be necessary, so it is recommended to roll this out as part of the standard release cycle of the product to minimize increased QA cost (assuming regression is already being performed).  If the team has moved to a model where very little regression is done on the site before changes are being released, the cost of a regression on changed functionality should be included.

If the site has multiple entry points for data entry, complex eCommerce needs, and security on access to specific pieces of data, this should probably be allocated to multiple senior developers with a dedicated QA resource for ongoing feedback.  Regression at the end will not provide fast enough results for this type of re-architecture.  A large project with multiple systems in play may require up to a year or two of both developers working part-time with this QA resource to ensure that all the systems have been properly refactored and verified.

Variance

As you can see, there is a large variance in the cost of this refactoring.  Your team should analyze their situation, and be realistic about how long it takes to refactor and retest their application.  Doing a rush job on this refactoring will not help, as the refactored code is likely to be the code that is placed directly into your services.  Estimate appropriately!

What’s next?

This series continues with the team starting on establishing a better identity management architecture with the implementation of an SSO system.