Data Migrations:

Data Migrations

Reading about the Data Migration problems experienced by a number of large financial institutions lately sends me into a cold sweat. I have worked on a number of Data Migration projects that could have had a similar outcome but thankfully didn’t.

It brings me absolutely no joy to read about this and similar software failures that have hit the headlines for all the wrong reasons. I feel the panic of the project team scrambling to fix the issues.

Data Migrations are very often implemented in parallel or are part of a bigger project such as moving from one application platform to another, two companies completing a merger and moving data from one to another, an application upgrade etc. Talking to other people in the industry, very often Data Migrations are not the initial or primary focus of one of these types of projects.

Getting the application developed and verified from a hardware platform and software point-of-view and then then the focus goes to the Data Migration. As we are all aware IT Projects very often do not run to schedule and the time it takes to solve all the unexpected issues with hardware and software, the time allocated to the Data Migration will be shorter than originally estimated.

From personal experience what has worked from my perspective was freeing up key stakeholders, business users and vendors for the day and going into workshop mode. Start off the workshop with speculating what is the worst that could happen from a Data Migration point-of-view. Between the group, agree to a number of absolute worst case scenarios and write these down. Try to limit the number to five but definitely no more than ten.

For example:

  1. A customer logs into the customer site and views data belonging to another customer
  2. A customer logs into the customer site and doesn’t see any of the data
  3. Active Trades pre-Migration disappear from the system

The basis

This will be the basis for your Test Strategy and approach in ensuring that these issues don’t occur. Use the list as a basis going forward for prioritising the defects you find when you are testing. When you are in full Testing mode of either the Data Migration or any other element of the project and you are prioritising your list of defects, if any these defects are a big factor in any of your worst case scenarios these should be prioritised over anything else.

If a defect is not linked to your worst case scenarios it should be given a lower priority. This helps you see the wood from the trees when you are at the height of defect management in a large scale project.

Back to your workshop, the next step is to go through the list of data that is being migrated and agree a priority for each data field or if you are migrating a huge set of data prioritise a data set e.g. Customer Home Address as opposed to Address_Line1, Address_Line2…but make sure that you know the data or data set that is very important to all stakeholders. I would pay extra special attention to the business in this area.

They are specialised in what they do and they should be very familiar with the data that is important to them. At the end of the workshop you (and everyone involved) should have a clearer picture of priorities for the Data Migration and the creation of the worst case scenarios should focus everyone’s efforts in getting the Data Migration done right.

When it comes to Test Strategy and Approach for the Data Migration you know what you need to prioritise when it comes to developing scripts and control reports to ensure at the very least in the early days of Data Migration verification that the vital information has been migrated successfully.

The last recommendation I would advise is to ensure that the top resources from a business user, analysis, development and testing point-of-view are working on the Migration. If there is any doubt show the worst case scenarios to the powers that be and ask them to make their choice on who should work in the area of Data Migration. Ideally the team should work well together as a cohesive unit and have the required support they need from a Management perspective. In the tough times, just focus on the great after-project party you’ll have afterwards when the migration has been successfully completed. That always helped in my experience. Cheers!

Claire Goss is the Senior Delivery Manager for Exactest Ireland. She is a graduate of DIT in Applied Sciences with Software Engineering. Claire has been a Software Testing specialist for the last eight years based both in Ireland and The Netherlands. She has worked in a variety of roles such as Test Co-ordination and Management.