Introduction
Data-driven algorithms are progressively making decisions that were previously made by humans, thanks to machine learning’s growing capability. Algorithm assurance is the process used to determine whether particular algorithms adhere to their desired objectives and produce the desired results.
It is a particular kind of IT assurance that aids in risk management and control over the use of risky algorithms in both products and enterprises. These algorithms are frequently referred to in organisations as advanced analytics, artificial intelligence applications (AI), or just predictive models.
At Canarys, we see our algorithms in the same light as our workforce. We show empathy to our algorithms, just as we do for our people. They should receive the same mentoring, assistance with their development, and performance evaluations as we do for our staff. With our assistance, you can promote the growth of your algorithm legally, efficiently, and productively.
Data Curation and Verification
Data is the new language for AI-driven solutions. These solutions must be tried and tested for every modification in the input data to ensure a system runs without any whisks. We often compare this with conventional testing methods, where any modification or a slight change in the code triggers testing of the revised code. When evaluating AI-based solutions, it’s crucial to take into account the following:
Developing curated training data sets
These are collections of semi-automated input and output data. It is essential to conduct static analysis of data dependencies to enable annotation of data sources and features—a crucial component for migration and deletion.
Developing test data sets
Each test data set is carefully constructed to test every possible combination and permutation to determine whether trained models are effective. As the training progresses and the amount of data becomes more decadent, the model is further improved.
Developing system validation test suites
These are based on algorithms and test data sets. For example, for a system built to predict patient outcomes based on pathological or diagnostics reports, scenarios must be designed around risk profiling of patients for the concerned disease, patient demography, patient treatment, and other similar test scenarios.
Reporting of test results
It must statistically be done because the range-based accuracy (confidence scores) produced by the validation of ML-based algorithms differs from what is anticipated. The testers must define and determine the confidence thresholds for each outcome within a given range.
The development of impartial systems has become crucial for contemporary businesses. Supervised learning techniques have identified data contaminated with logical reasoning and biases, which currently account for more than 70% of AI use cases. This makes evaluating the input training data sets’ “bias-free” quotient a “double-edged sword.” And if we do, data biases might enter the picture.
These can be decreased by Apriori testing the input labelled data for any hidden patterns, fictitious correlations, heteroscedasticity, etc. Let’s analyse some primary biases that testers must consider when conducting AI/ML testing.
Data Skewness
The data that we frequently use to train the model is very skewed. Sentiment analysis is the most prevalent example; however, most data sets require an adequate (or equal) number of data points for each type of sentiment. As a result, the produced model is “biased” and skewed” toward the sentiments that have more extensive data sets.”
Prediction bias
In a functional system, the distribution of predicted labels should match that of the identified labels. Although not a detailed test, this diagnostic step is constructive. Changes in metrics like these frequently indicate that something needs to be addressed. This technique, for instance, can be used to find cases where the system’s behaviour changes abruptly. In such contexts, training distributions derived from historical data no longer accurately reflect the state of affairs.
Relational bias
Users are typically constrained and biased in how a data pattern or problem set should be solved based on their perception of the relational mapping of which solution would have been effective for a particular type of problem set. However, this may skew the solution to the users’ familiarity, avoiding more difficult or unusual ones.
While addressing data biases is vital, as previously mentioned, we should also consider the issue of under or over-fitting the model through training data, which occurs far too frequently and negatively influences the model’s performance. To ensure the model has generalised the solution effectively and that the trained model can be utilised in production, it is imperative to quantify the amount of over-fitting.
Canarys Algorithm Assurance Offerings
Algorithms
- Natural Language processing/understanding
- Image Processing
- Machine Processing
- Deep Learning
Data Creation and Curation
- Domain Specific Data
- Cleansing and identifying sample data set
- Contextual data clusters
- Data denoising
- Data labelling
Smart Interaction Testing
- Devices (Alexa, Siri etc.)
- AR/VR
Real-Life Testing
- Human Unbiased testing
- Challenger Model (Algorithm accuracy testing)
- Decision analysis (Explainable AI)
- Deployment and accessibility Testing
- Test Triangle( Unit, Service and UI)
- White Box and Black Box Testing
- Model back-testing
While we foresee that Explainable AI (XAI) and AutoMLtechniques will significantly improve testing effectiveness going forward, we will focus on only some of the techniques that will need to be used in real-life testing from a model and data set perspective.
Black Box and White Box testing
Like with conventional test techniques, testing ML models comprises both Black Box and White Box testing. It can be hard to locate training data sets that are extensive enough to address the requirements of “ML testing.”
Data scientists test the model performance during the model development phase by contrasting the model outputs (predicted values) with the actual values.
Following are a few methods for Black Box testing ML models:
Model performance testing
Involves testing with test data/new data sets and comparing the model performance in terms of parameters such as precision-recall, F-score, and confusion matrix (False and True positives, False and True negatives) to that of pre-determined accuracy with which the model is already built and moved into production.
Metamorphic testing
This aims to solve the test oracle issue. A tester can determine whether a system responds accurately using a test oracle. It happens when it is challenging to ascertain whether the actual output is in line with the anticipated outcomes or the expected outcomes of chosen test cases.
Dual coding/Algorithm ensemble
Multiple models using different algorithms are built, and predictions from each of them are compared, given the same input data set. For assembling a typical model to address classification problems, multiple algorithms like Random Forest or a neural network like LSTM could be used – but the model that produces the most expected outcomes is finally chosen as the default model.
Coverage Guided Fuzzing
Data fed into the ML models is designed to test all the feature activations. For instance, for a model built with neural networks, testers need test data sets that could result in the activation of each of the neurons/nodes in the neural network.
Model back-testing
Back-testing is a predictive model based on historical data. This technique is popular in estimating the performance of past models used in financial sectors, especially for trading, investment, fraud detection or credit risk assessments.
NFR (Non-Functional Requirements) testing
In addition to performance and security testing, factors such as a representative sample view of things along with the deployment approach also need to be considered while testing ML Models. Questions such as: How do we replace an existing model in production? What is our view of A/B Testing or Challenger Models? – must also be answered.