Anticipating the Unexpected: Takeaways on AI Amid Historic Volatility

April 29, 2020
In a time of pronounced market volatility, has artificial intelligence (AI) delivered on its central promise of enabling better decision-making? Using BBH’s Anomaly NAV Tracking System (ANTS) as a case study, the answer is an emphatic yes. But that doesn’t mean there aren’t challenges. Here’s what we have learned the past few weeks.

As global markets began reacting to the uncertainty of the COVID-19 pandemic, organizations involved in supporting trading, settlement, valuation, and other operating activities faced unprecedented volume and volatility while also transitioning their working models to incorporate social distancing practices.

One impacted process was the calculation and certification of an accurate net asset value (NAV) for investment funds. This critical function immediately became more difficult as fluctuations in security prices caused high volumes of price tolerance exceptions even with increasingly sophisticated validation models. Such models are designed to identify securities that are incorrectly valued and could, in theory, cause a NAV error. Examples of this include scenarios where a corporate action was not processed correctly (e.g. stock split), or a security was incorrectly priced.

The sophistication of these validation models has grown exponentially in recent years thanks to advances in machine learning and AI technology. BBH has recently gone through a similar transformation. Knowing that our models were producing thousands of exceptions for our analysts to review each night, with less than 1% representing a true issue, we turned to AI to help alleviate the strain of manual review and to enhance efficiency and control. In 2018, we introduced ANTS, which uses predictive analysis and machine learning to eliminate the noise of false positives and highlights exceptions that might not have been previously detected. With ANTS, analysts spend less time sorting through recurring exceptions and more time on resolving true NAV issues.

In recent weeks, however, we saw a significant increase in pricing exceptions, which makes sense considering the immense volume of trading activity since the crisis began. Realizing we could be facing a prolonged period of volatility, and knowing the answer was not to simply assign more people to reviewing exceptions, we once again turned to the model. The goal was to review adjustments to the algorithm that would further reduce exceptions in a safe and sustainable way. While we are always making incremental adjustments to the model, it was clear that addressing the impact of this crisis required rapid mobilization across multiple functions — from our data scientists to our fund accounting team and everyone in between. In mobilizing these experts, we could further sharpen the processes we instilled at the inception of our AI program. This is at the core of what we call pragmatic transformation — incremental adjustments that yield meaningful results. 

In many ways, the events this year have validated our key principles in planning ahead for adoption. Understanding that this era of “new normal” will likely throw some additional curveballs, we remain committed to an iterative approach to refining our technology. Here are the key takeaways we keep front of mind as we look to the future.           

1. Recalibrate and adjust to improve over time

No prediction model lasts forever — they will all degrade over time due to seasonality, changes in production data, new events, and patterns which lead to continual adjustments and fine tuning. Just consider the events of the last couple of months: The average daily market change of security prices in March 2020 was over 5%. The next highest month on record was October 2008 at 3.8%. Our previous model’s data set had simply not seen this type of volatility, so adjusting the model quickly was essential. In fact, when back-testing ANTS against our legacy model, we saw a reduction of ~77% in pricing exceptions that needed to be reviewed. Considering we value hundreds of thousands of unique positions, these percentage impacts reflect a significant reduction in work effort. But it was also important to understand bigger patterns that were emerging. This is where ANTS proved its mettle: instead of our analyst and data systems teams spending their time sorting through the exceptions (a task assigned to ANTS), the model identified larger emerging themes. Rather than constructing a new decision-making logic from scratch, we were able to build off a previous iteration and home in on those patterns to make small adjustments. We continue to make these adjustments and are optimistic that a new version of the model can reduce those exceptions by another 5-6%.

2. Data purity and accessibility is a pre-requisite for agile

Data, it’s often said, is the fuel that ignites the AI engine. That’s why real-time views into volumes and trends is paramount in adjusting to a dynamic environment. Data is useless unless it can be easily accessed, normalized, and understood by business users and IT alike. For us, we knew we couldn’t implement AI without a streamlined data inventory. So at the launch of ANTS, we standardized metrics to monitor AI accuracy, application utilization, and efficiency. Doing so at the beginning is paying dividends today because we are able to surface, in real-time, the most pressing pain points. These metrics are critical for the long-term success of AI adoption, giving us insight as to when and how often to retrain a model. 

3. Precision beats speed to market any day

Like many in our industry, we use the agile methodology for our technology development. But agile development requires integrated business and technology teams operating in a single framework that incorporates risk and control protocols that facilitate analysis, development, testing, and deployment – no easy feat. Coordinating among these disparate functions cannot be rushed. Nor should precision and accuracy take a back seat to speed to market. Given the business-critical function of NAV calculations, oversight and auditability is central to the sustainability of an AI program. A possible framework for consideration:

  • Designate responsibilities between model research and proposal versus model testing and acceptance
  • Determine KPIs to objectively compare and select an optimal solution
  • Ensure models are archived and documented for proper version control oversight and lineage to audit the AI decisions in a production environment

Looking Ahead

Some of the early lessons from the impacts of COVID-19 were really a validation of our initial AI and machine learning transformation journey within our fund accounting product. However, it has further influenced our thinking as we continue to refine our approach. In the above example, the process to analyze and react required data extraction, review with data scientists, and some degree of translation into business terms for our subject matter experts to make a decision on the best course of action. In the future, we see benefits in allowing the business user to test and toggle between multiple AI models within the application in real-time.

The extraordinary volumes have also proved the accuracy of our models on a larger scale. This has deepened the confidence our practitioners have in the tools and willingness to increase their utility. These events continue to emphasize the importance of new skillsets required of our workforce. Analysts no longer need to know the steps to carry out a task, but how to analyze the performance of the system to ensure it is accurate and learning correctly.

As firms consider how to respond to operational impacts of these extraordinary market events, it is evident we must continue focus on reskilling and broadening understanding of AI and machine learning in addition to developing subject matter expertise. As we look for ways to scale and improve capabilities in these technologies, this is a necessary balance in skillset and mindset. While the impacts of COVID-19 will certainly alter future operating models across the industry, we have leveraged the opportunity to refine and advance our efforts in line with our pragmatic transformation.

Brown Brothers Harriman & Co. (“BBH”) may be used as a generic term to reference the company as a whole and/or its various subsidiaries generally.  This material and any products or services may be issued or provided in multiple jurisdictions by duly authorized and regulated subsidiaries.This material is for general information and reference purposes only and does not constitute legal, tax or investment advice and is not intended as an offer to sell, or a solicitation to buy securities, services or investment products. Any reference to tax matters is not intended to be used, and may not be used, for purposes of avoiding penalties under the U.S. Internal Revenue Code, or other applicable tax regimes, or for promotion, marketing or recommendation to third parties. All information has been obtained from sources believed to be reliable, but accuracy is not guaranteed, and reliance should not be placed on the information presented. This material may not be reproduced, copied or transmitted, or any of the content disclosed to third parties, without the permission of BBH. Pursuant to information regarding the provision of applicable services or products by BBH, please note the following:Brown Brothers Harriman Fund Administration Services (Ireland) Limited and Brown Brothers Harriman Trustee Services (Ireland) Limited are regulated by the Central Bank of Ireland,Brown Brothers Harriman Investor Services Limited is authorised and regulated by the Financial Conduct Authority, Brown Brothers Harriman (Luxembourg) S.C.A is regulated by the Commission de Surveillance du Secteur Financier All trademarks and service marks included are the property of BBH or their respective owners. © Brown Brothers Harriman & Co. 2020. All rights reserved. IS-06127-2020-04-28

This browser is not fully supported by our public website and may not display or function as expected for this reason. Please note, the Infuse Portal and BBH client applications fully support the IE 11 browser.

Important Information for Non-U.S. Residents

You are required to read the following important information, which, in conjunction with the Terms and Conditions, governs your use of this website. Your use of this website and its contents constitute your acceptance of this information and those Terms and Conditions. If you do not agree with this information and the Terms and Conditions, you should immediately cease use of this website. The contents of this website have not been prepared for the benefit of investors outside of the United States. This website is not intended as a solicitation of the purchase or sale of any security or other financial instrument or any investment management services for any investor who resides in a jurisdiction other than the United States1. As a general matter, Brown Brothers Harriman & Co. and its subsidiaries (“BBH”) is not licensed or registered to solicit prospective investors and offer investment advisory services in jurisdictions outside of the United States. The information on this website is not intended to be distributed to, directed at or used by any person or entity in any jurisdiction or country where such distribution or use would be contrary to law or regulation. Persons in respect of whom such prohibitions apply must not access the website.  Under certain circumstances, BBH may provide services to investors located outside of the United States in accordance with applicable law. The conditions under which such services may be provided will be analyzed on a case-by-case basis by BBH. BBH will only accept investors from such jurisdictions or countries where it has made a determination that such an arrangement or relationship is permissible under the laws of that jurisdiction or country. The existence of this website is not intended to be a substitute for the type of analysis described above and is not intended as a solicitation of or recommendation to any prospective investor, including those located outside of the United States. Certain BBH products or services may not be available in certain jurisdictions. By choosing to access this website from any location other than the United States, you accept full responsibility for compliance with all local laws. The website contains content that has been obtained from sources that BBH believes to be reliable as of the date presented; however, BBH cannot guarantee the accuracy of such content, assure its completeness, or warrant that such information will not be changed. The content contained herein is current as of the date of issuance and is subject to change without notice. The website’s content does not constitute investment advice and should not be used as the basis for any investment decision. There is no guarantee that any investment objectives, expectations, targets described in this website or the  performance or profitability of any investment will be achieved. You understand that investing in securities and other financial instruments involves risks that may affect the value of the securities and may result in losses, including the potential loss of the principal invested, and you assume and are able to bear all such risks.  In no event shall BBH or any other affiliated party be liable for any direct, incidental, special, consequential, indirect, lost profits, loss of business or data, or punitive damages arising out of your use of this website. By clicking accept, you confirm that you accept  to the above Important Information along with Terms and Conditions.

 
1BBH sponsors UCITS Funds registered in Luxembourg, in certain jurisdictions. For information on those funds, please see bbhluxembourgfunds.com


captcha image

Type in the word seen on the picture

I am a current investor in another jurisdiction