Cornerstone Reduces False Positives by 60% in Trade Surveillance

Cornerstone Reduces False Positives by 60% in Trade Surveillance

Executive Summary

Cornerstone Advisory Group faced a significant challenge with its trade surveillance system, which generated a high volume of false positives, diverting valuable compliance resources and potentially masking genuine instances of misconduct. Golden Door Asset partnered with Cornerstone to fine-tune their existing trade surveillance system using advanced analytics and machine learning. The result was a 60% reduction in false positive alerts, significantly improving compliance team efficiency and allowing them to focus on higher-risk areas.

The Challenge

Cornerstone Advisory Group, a rapidly growing RIA managing over $3 billion in assets, was struggling with an inefficient trade surveillance system. Their existing system, implemented through Charles River IMS, generated an overwhelming number of alerts, the vast majority of which proved to be false positives. This required a significant amount of manual review by their compliance team, consuming valuable time and resources.

Specifically, the system was flagging an average of 450 alerts per month, related to potential insider trading, market manipulation, and other compliance violations. After a thorough manual review, only approximately 18 of these alerts (4%) warranted further investigation. The remaining 432 alerts (96%) were identified as false positives, triggered by factors such as algorithmic trading strategies, large block trades, or perfectly legitimate market movements.

The manual review process was not only time-consuming but also created a significant bottleneck. Each alert required an average of 2 hours of investigation, equating to 864 hours (approximately 110 business days) per month dedicated solely to dismissing false positives. This translated to a substantial cost in terms of employee salaries and lost opportunity. Cornerstone estimated that the inefficient system was costing them over $85,000 per year in wasted man-hours.

Furthermore, the high volume of false positives risked desensitizing the compliance team, potentially leading to missed instances of genuine misconduct. The constant noise made it difficult to identify and prioritize the alerts that truly warranted attention, increasing the risk of regulatory scrutiny and potential financial penalties. For example, a failure to detect and prevent even a single instance of insider trading could result in fines exceeding $1 million. The risk-adjusted cost of the false positive problem was, therefore, significant.

The Approach

Golden Door Asset approached the challenge with a data-driven methodology, focusing on identifying the root causes of the false positives and refining the alert thresholds within Charles River IMS. The approach involved several key steps:

1. Data Collection and Analysis: We began by gathering historical trading data, alert logs, and manual review records from Cornerstone's Charles River IMS. This data was analyzed to identify patterns and characteristics associated with both true positive and false positive alerts. We looked at factors such as trade size, security type, trading volume, time of day, price volatility, and relationships between accounts.

2. Alert Threshold Optimization: Based on the data analysis, we identified several instances where the alert thresholds were set too aggressively. For example, the system was flagging any trade that deviated by more than 5% from the average trading volume, regardless of the security or market conditions. We worked with Cornerstone's compliance team to adjust these thresholds, taking into account factors such as market volatility, sector-specific trends, and the size and liquidity of the traded securities.

3. Machine Learning Model Implementation: To further enhance the accuracy of the trade surveillance system, we implemented a machine learning model within Charles River IMS. This model was trained on historical trading data to predict the probability of an alert being a true positive or a false positive. The model considered a wide range of variables, including trading volume, price volatility, order types, and historical trading patterns of individual accounts.

4. Integration with Charles River IMS: The machine learning model was seamlessly integrated with Cornerstone's existing Charles River IMS. This allowed the system to automatically score each alert based on the model's prediction. Alerts with a low probability of being a true positive were automatically suppressed, while those with a high probability were prioritized for manual review.

5. Ongoing Monitoring and Refinement: The system was continuously monitored and refined based on feedback from Cornerstone's compliance team. The machine learning model was regularly retrained with new data to ensure its accuracy and effectiveness over time. This iterative process ensured that the system remained adaptive to changing market conditions and evolving trading strategies.

Technical Implementation

The technical implementation involved the following key components:

• Data Extraction and Preprocessing: Historical trading data, alert logs, and manual review records were extracted from Charles River IMS using its API. The data was then preprocessed to handle missing values, outliers, and inconsistencies. This involved techniques such as data imputation, Winsorization, and standardization.
• Feature Engineering: Relevant features were engineered from the preprocessed data. These features included:
  • Trading Volume Ratios: Ratios of current trading volume to historical trading volume, calculated over different time periods (e.g., daily, weekly, monthly).
  • Price Volatility Measures: Measures of price volatility, such as standard deviation of returns and average true range (ATR).
  • Order Type Ratios: Ratios of different order types (e.g., market orders, limit orders, stop-loss orders) to total orders.
  • Account-Specific Features: Features related to the trading history of individual accounts, such as average trade size, frequency of trading, and portfolio composition.
• Machine Learning Model Training: A gradient boosting machine (GBM) model was trained to predict the probability of an alert being a true positive or a false positive. GBM was chosen for its ability to handle complex relationships between variables and its robustness to outliers. The model was trained using a supervised learning approach, with manual review results serving as the ground truth.
• Model Evaluation: The performance of the model was evaluated using metrics such as precision, recall, F1-score, and AUC-ROC. The model was fine-tuned to optimize the balance between precision and recall, minimizing the number of false negatives (missed instances of misconduct) while also reducing the number of false positives.
• Integration with Charles River IMS API: The trained machine learning model was integrated with the Charles River IMS API. Real-time trading data was fed into the model, and the model's predictions were used to prioritize alerts for manual review. Alerts with a probability score below a certain threshold (e.g., 0.2) were automatically suppressed.
• Alert Threshold Adjustments: Initial alert thresholds were adjusted based on the data analysis. For example, the threshold for flagging unusual trading volume was increased for highly liquid stocks, while it was lowered for less liquid stocks. The specific adjustments were determined in consultation with Cornerstone's compliance team.

Results & ROI

The implementation of Golden Door Asset's solution resulted in a significant improvement in the efficiency and effectiveness of Cornerstone's trade surveillance program:

• False Positive Reduction: The number of false positive alerts was reduced by 60%, from an average of 432 per month to 173 per month.
• Manual Review Time Savings: The time spent on manual review was reduced by approximately 518 hours per month (864 hours before - 346 hours after), freeing up compliance staff to focus on higher-risk activities. The post-implementation calculation is based on the same 2 hours per alert.
• Increased True Positive Identification: The percentage of alerts that resulted in further investigation increased from 4% to 10%, indicating that the compliance team was now able to focus on alerts that were more likely to represent genuine instances of misconduct. This is calculated by retaining the 18 True Positive cases per month against the new 173 False Positive cases.
• Cost Savings: The reduction in manual review time resulted in an estimated cost savings of over $51,000 per year, based on an average hourly rate of $50 for compliance staff.
• Improved Compliance Oversight: By reducing the noise from false positives, the compliance team was able to gain a clearer picture of trading activity and identify potential compliance violations more effectively. This reduced the risk of regulatory scrutiny and potential financial penalties. Cornerstone's compliance officer stated that they felt significantly more confident in their ability to detect and prevent market abuse.

Key Takeaways

1. Data-Driven Approach is Crucial: Relying solely on generic alert thresholds can lead to a high volume of false positives. A data-driven approach, involving the analysis of historical trading data, is essential for optimizing alert parameters and improving the accuracy of trade surveillance systems.

2. Machine Learning Enhances Precision: Machine learning models can significantly enhance the precision of trade surveillance systems by learning from historical data and identifying patterns associated with both true positive and false positive alerts.

3. Integration with Existing Systems is Key: Seamless integration with existing systems, such as Charles River IMS, is crucial for ensuring the smooth and efficient operation of the trade surveillance program. This allows for the automated scoring and prioritization of alerts.

4. Collaboration is Essential: Effective trade surveillance requires close collaboration between compliance teams and technology providers. This ensures that the system is tailored to the specific needs and requirements of the firm and that the results are properly interpreted and acted upon.

5. Continuous Monitoring and Refinement are Necessary: Trade surveillance systems must be continuously monitored and refined to ensure their accuracy and effectiveness over time. This involves regularly retraining machine learning models with new data and adjusting alert thresholds based on changing market conditions and evolving trading strategies.

About Golden Door Asset

Golden Door Asset builds AI-powered intelligence tools for RIAs. Our platform helps advisors reduce compliance costs while improving the accuracy and efficiency of their surveillance program. Visit our tools to see how we can help your practice.