In early 2015 we embarked on a journey of developing a new reconciliation system. When we started to plan migration between the legacy and the new system, we struggled to find relevant reading material. Actually, it was easier to find an article about migrating rhinos using helicopters than a roadmap of a successful software migration! In this post, we outline the five simple rules that guided our migration — hopefully they will be simpler than flying rhinos.
Reconciliation is a post trading activity that compares our view of executed trades with views held by our counter parties. In each trade we have two counter parties and both should agree on the price and quantity of the trade. In reality this is a bit more complicated and we have to compare dozens of fields.
Reconciliation software can be thought as a mixture of a diff tool and Excel. At the beginning, we download a number of files from our counter parties which are then compared with a record of trades in our database. Differences end in the hands of our reconciliation team who resolve them manually.
While reconciliation is not a glamorous task in the world of hedge funds, it is essential. Without it, we would not be able to invoice our trading partners on time. When we asked ourselves how we are going to perform the migration, our first concern was downtime. Reconciliation is the beginning of any post trade activities so any delays are likely to make our reconciliation team anxious and ready to ruin our lunch foosball sessions.
Those concerns led us to the first rule:
Rule 1: Each step of migration is reversible
What we mean here is that, at any point, we can go back to the old system with zero downtime. An easy way to achieve this is to have two systems run in parallel. After morning coffee, our reconciliation team would start working with the legacy system. As soon as they were done they would move on to the new system and perform exactly the same tasks. Both teams would now have peace of mind.
After running a small number of reconciliations every day, we started to think how we can leverage the years spent on the legacy system. Those discussions brought us to the second rule:
Rule 2: New system can consume historical data
This can mean different things but for us it meant importing configuration from the legacy system. Reconciliation software has to work with a number of counter parties. Given that standardisation is not a strong point of the financial industry, each counter party invented its own file format and its own naming scheme. The same stock could have several different names and having a good mapping configuration saved us weeks of work.
Once we had our new system running every day, we started to worry about correctness. The old system worked for years so we could use it as a blueprint. That gave us the third rule:
Rule 3: New and old system produce identical results
After the reconciliation team were done they would go and compare differences between the two systems. In case of any differences, they would follow up with us to check if an observed difference is a bug, new feature or a bug fix to faulty legacy functionality. After manual checks have been performed, we would run a number of automatic checks to ensure that both databases are consistent and correct.
So now we have a system that runs every day and is correct, what else we can ask for?
Rule 4: New system is accessible from early on
Here we get a bit philosophical. When working on a system that has the same functionality as the old one we are tempted to delay delivery until all functionality is ready. I mean we already know what to do. We just need to add some microservices and possibly squeeze Mongo and Angular somewhere, right? This thinking can be dangerous as it will lead us to create a verbatim copy of the old system. When a customer is given a version with limited functionality and is able to run both systems in parallel, he is likely to re-think the ergonomics of the old system.
To allow early adoption we used agile methods to drive our development. We brought all the goodies, weekly sprints, followed by planning meetings and end of week releases. By involving our users early, we found that the system we developed differed significantly in its design and usability from the legacy system. We also found that running our system for months before final migration uncovered a multitude of performance and stability issues.
As we got close to the end, we thought, we just have to hit the switch off button on the old system and we can book our holidays! Not exactly, there is one last thing to do:
Rule 5: Completely retire the old system
At the end of the project there is a danger of patting ourselves on the back and moving on to a new project. In reality, there is a number of other systems that point to the old system. Some of those may not even be known to us. A successful migration should migrate all those clients and aim to completely delete the old system with a single backup saved for audit purposes.
In our case the existing reconciliation system has been the main source for our fund accounting software. It is impossible to invoice our partners if we are not certain about the transaction we have made. Transaction data that has been the result of our reconciliation has been exposed by a number of stored procedures. Our new system has an export mechanism that would then feed into those procedures, making the migration seamless from the fund accounting perspective.
In this article we provided a short summary of our experiences with migration and decommissioning of an old computer system. We hope this post will shed some light on the subject and help those considering their first migration. Our system has been successfully delivered and the rules we created worked as a roadmap for multiple projects at Winton.