Originally published on CUInsight.com
For those of us who experienced a partial loss of vision at a relatively young age, we likely remember the day that we learned we needed glasses. The process typically began with an elementary school teacher who noticed our permanent squint at the chalkboard. Before we knew it, we were sent to the school nurse for a primitive eye exam consisting of a few “E’s” in all of their assorted directions. Failing that, we were sent home with a note to our parents, who took us to an eye doctor, who had far more extensive tests that accurately zeroed in on what was wrong with our eyes.
Many AML monitoring system validations are a bit like that initial test in the school nurse’s office. They tend to establish whether there are broad issues and can even point to the nature of the issue to some degree. But if you were to have the school nurse prescribe your glasses, you’d probably have found that you needed to squint less, but that you were still missing some things that the other kids could see.
A true test, as most of us with glasses found, is a far more extensive series of methods and cross checks to ensure a proper prescription. Some of those strange-looking contraptions you had to look through probably made you wonder just how complicated was this eyewear business. But when you got the correct pair of lenses, and all the blurry lines in your life came into focus, you probably came away in awe of what you had been missing. Suddenly, there are trees on the mountains. There are planes in the air. The lights at night aren’t big blobs, but they’re actually smaller lights.
Bringing AML into Focus
AML validation is similar in that it works best when you know that your system is setup correctly to begin with. However, many AML validations are doing only basic tests, such as the various “E’s,” and not the full series of tests needed to ensure a prescription for clear vision.
The point of a validation is to make sure that your AML monitoring system is working as designed. In other words, you want to have a reasonable assurance that your system is triggering alerts aimed at detecting illegal and/or terrorist activity when it should. No more, no less. It is about achieving the clearest vision possible into the world of member transactional activity.
At a fundamental level, an AML system works like this. All transactions at the bank flow through various core systems and are fed through the AML system, which applies the various rule parameters to the transactions, and when it sees a certain threshold has been exceeded, either directly or through a combination of algorithms, it creates an alert for those transactions and/or member(s). For example, a system may have a structuring alert that looks for transactions that are greater than $3,000 but less than the CTR threshold of $10,000. The system would then look for multiple transactions of this nature across a specific time frame, typically around a week. Customers who have aggregated activity over a certain amount, say $10,000 or $15,000, during that time period would trigger an alert for someone in the BSA department to review. That person might then discover either that there was a legitimate reason for the cash transactions or that the transactions warrant further investigation.
Back-to-Front Analysis
Many times, a validation effort consists of looking at a sample of alerts that have been generated, then going back to the core data to confirm that the transactions actually occurred as indicated. To continue with our analogy, it’s like looking at the finished photo and determining the settings of the camera. In the previous example, the validation might consist of making sure the transactions that triggered the alert were actually cash and aggregated to more than the triggering threshold.
We call this process a back-to-front analysis, because it takes the alerts that came out of the AML system – or “the back” – and compares them to the transactions that were generated from the core system – or “the front.” Even when applying rules-based risk-rating systems, the validation methodology focuses more on verifying the combination of triggers that generated the member score and then evaluating that score against the transactions and/or rules that generated the score. That is where a lot of AML validation efforts stop.
The problem with stopping there is it doesn’t answer the question of whether all transactions that should have triggered an alert actually triggered an alert. It is good to confirm that an alert is valid, but how do you confirm that the “non-presence” of an alert is valid? Were there actually no transactions that should’ve triggered a given alert, or is there a problem with the configuration of the system that resulted in transactions that should’ve triggered an alert being missed?
Front-to-Back Analysis
To figure this out, it is necessary to conduct what we call a “front-to-back” analysis. In other words, we want to provide a reasonable assurance that the transactions from the core systems – or “the front” – that should’ve triggered an alert actually did trigger an alert in the AML system – or “the back.”
The reason many AML validation efforts omit the front-to-back validation is – frankly – that it is a lot of work. In order to conduct such a cross check, it is necessary to take a sample of the core transactions for a given period of time – say a month of data – and run those transactions through – in essence – a secondary AML system to see if there are any discrepancies between the two systems.
Of course, you spent a lot of money on your AML system, so you may not be excited to spend money on a secondary system, but how else will you be able to cross check your core data? A month of transactional data can mean hundreds of thousands or even millions of rows of data, so a manual check can likewise be cost-prohibitive.
Hence, one workable solution is to re-create the primary rule parameters in a secondary system, such as SQL, to identify those transactions that fall within the identified parameters and should have generated an alert. It’s a sizable project to recreate the rule parameters, but it allows us then to compare a list of what our secondary system generated to what actually was generated by the AML system. We then work with the BSA team and sometimes the AML system vendor to determine the cause of the discrepancies. Frequently it turns out that there are exemptions for certain customers and that the discrepancies are justified.
But there are other occasions where key issues may be identified. Examples that we’ve seen are cases where transactions were not properly imported due to some change in the system, or a transaction code had been inputted incorrectly so that certain cash transactions were coded as ACH transactions, or that somebody jumped the gun on a parameter change without going through the full approval channels. There are many possible reasons, some valid, some not, but the point is that if the front-to-back isn’t conducted, it is impossible to know whether something that should have alerted, didn’t.
Good eyesight is exceptionally valuable in many aspects of life. It is something that should be treasured. Once achieved, it must be maintained and that requires ongoing examinations and modifications. Similarly, investing the time and resources into performing a full AML monitoring system validation can save you from increased scrutiny from regulators, costly look-back projects, and heavy penalties. Doing it right the first time typically saves you time, money, and headaches later. The world was meant to be seen with 20-20 vision.
