In today’s dynamic financial markets, one needs to make intelligent, sound, and quick decisions, the key enabler of which, is technology. While it is true that the structure of financial markets is complex, with many pillars such as regulations, taxes, and legal compliance, technology is playing an extremely critical role, having the capacity to completely ruin the mechanism of a securities market in today’s world.
A classic example of how technology can fail one, if not properly implemented, is the recent technical glitch at the National Stock Exchange, which resulted in halting of trading in two segments – Cash and Futures & Options – for three hours! The business was surely impacted because NSE occupies a very large share of spot and equity derivatives market. The communication around the technical failure was also inefficient, leaving multiple stakeholders bewildered for the same time period. The NSE made multiple attempts to bring up trading before finally getting it right at 12:30pm.
This glitch at NSE has certainly exposed how vulnerable our infrastructure is, and highlights the lessons we should learn and precautions we should take henceforth. Based on my experience of dealing with building and supporting such trading platforms for a number of international clients, here are a few things we should always keep in mind.
BCP: The backup site, also known as the BCP (business continuity plan) needs to be clearly defined, made publicly available, and executed both as mock drill at a certain frequency as well as in reality when an event kicks off. In this particular event, NSE stated that the BCP wasn’t kicked off because it was a software issue, but it is important to know how long it would have taken to get executed if it was triggered. Consumers have the right to know about these things and also need to care about them.
Technology: Redundancy for platforms at NSE level needs to be not as a backup BCP site in hot/cold mode, it also needs to be a hot/hot redundancy. Most cloud providers can help one easily achieve hot/hot redundancy across multiple physical data centers. With NSE and exchanges, this gets more complicated because they have co-location and latency obligations with clients that need to be managed in case of BCP events, and something that can be achieved in cloud, can also be achieved within in-house data centers.
While this event luckily didn’t seem to be related to a DOS attack or other malicious attacks, it’s really important to evaluate threat perception and have checks and balances to prevent systems coming down due to such issues, across multiple levels such as physical, network, application, volume based restriction etc.
Supporting processes:
Communication – It is extremely important to engage with stakeholders, for which a well-defined communication plan is imperative. The focus should be on prompt, proactive and clear communication. Details which are not available at the time of such events can always be shared later and one can come back with updates on the issue.
Technical support infrastructure – There are multiple levels at which this is typically solved in enterprises. First is proactive monitors that scrape through logs of applications in real time and come back with problems before users report them. Second is having ways to test the systems on continuous basis without waiting for markets to open. Given how soon the glitch began to surface in this case after markets opened, having a test trade infrastructure that keeps vetting system even when markets are closed, could really help.
Decision making – Having a clear chain of command and clear protocols can help quickly decide if we should kick off a disaster/BCP event.
In addition, we need to focus on building systems in a more efficient manner architecturally, which will allow easy identification and rectification of the problem. Large monolithic systems takes hours to debug, fix and restart. Hoping that we are better prepared to take on such challenges in
Source: http://bit.ly/2xQo0P8
