AI Audit for Predictive Models

An audit of a predictive model examines whether its forecasts or risk estimates are supported by the data, evaluation design and operating process in which they will be used. A low historical error is not enough. The auditor needs to know what was predicted, what information was available at the time, how outcomes were observed and whether the deployed implementation matches the model that was assessed.

This guide covers predictive models outside the bank-specific validation discussed in model validation for banks. Its hypothetical example is a regional equipment-demand predictor used by a rental operator to inform planning for temporary lighting units. It estimates next-week requested units by region. It does not automatically purchase equipment, commit inventory or make a financial recommendation.

The methods below are original audit procedures, with a small worked calculation to explain interpretation. They do not establish that a particular forecasting technique is best for every Saudi business. A scoped AI audit should connect the model's evidence to the actual planning decision and disclose the conditions in which that evidence is weak.

Define the target before choosing the metric

Requested demand, fulfilled orders and available capacity are different quantities. If the operator records only completed rentals, a period with no available equipment may look like a period with no demand. A model trained on that record can learn the constraints of the service rather than the demand the planner wants to anticipate.

Write the target definition in operational terms: the unit, region, time interval, request status and treatment of cancellations or duplicate enquiries. Identify the system that records the outcome and the person responsible for resolving ambiguous cases. A label such as “weekly demand” is not enough if different teams count enquiries, reservations and fulfilled rentals differently.

State the prediction horizon and decision time. A next-week forecast issued on Monday differs from a forecast updated after part of the week has occurred. The audit should examine the information available at that issue time, including when booking changes or regional schedules were actually known. Otherwise, a historical result can quietly benefit from hindsight.

Map the prediction to the decision it supports

Identify how planners use the output. They may adjust a review queue, consider transfers or request more information. The model's numerical accuracy and the quality of the eventual planning decision are related but separate. A useful forecast can be ignored, while a weak forecast can be masked by experienced staff correcting it.

Define the relevant consequences of underprediction and overprediction. They may differ by region, equipment class or planning horizon. Choose evaluation measures that illuminate those consequences without pretending a generic metric determines the business decision. Where monetary assumptions are used, label them as internal assumptions and have the appropriate owner approve them.

Record exclusions and fallback rules. A model may have insufficient history for a new region or newly introduced equipment. The interface should make that limitation operationally visible rather than presenting an ordinary-looking forecast. An audit should test whether the claimed use boundary is actually enforced or communicated to planners.

Create a forecast evidence specification

Original evidence specification for the hypothetical equipment-demand predictor

Element	Definition to freeze	Evidence needed
Target	Requested lighting units for the defined next-week period.	Request-status rules, deduplication and cancellation treatment.
Prediction origin	The planning cut-off when the forecast is issued.	Timestamped input snapshot and forecast record.
Permitted information	Only data available by that cut-off.	Source availability times and transformation history.
Population	Named regions, equipment class and eligible periods.	Inventory of included and excluded series.
Reference outcome	Outcome under the approved demand definition.	Mature request records and reconciliation to the source system.
Comparison	A defensible baseline using the same information and horizon.	Baseline procedure, settings and results.
Decision boundary	Planning support only under the stated workflow.	Interface, use policy and observed planner actions.

Freeze the specification before selecting favourable results. If the target or outcome rules change, explain the effect on comparability. A revised definition may be better, but it should not be presented as an unexplained performance gain by the same model on the same task.

Reconstruct what was known at each forecast origin

Inspect feature availability, not just event dates. A booking may relate to next week but arrive after the historical forecast was issued. A revised regional total may overwrite an earlier estimate. The audit needs the value available at the forecast origin or a defensible reconstruction, with limitations where the original state is unavailable.

Review preprocessing within the time split. Normalisation, missing-value treatment, feature selection and tuning should not learn from future evaluation observations. A dataset can be sorted by date while a transformation still uses the full history. Trace the actual preparation and fitting operations rather than relying on labels such as “time-aware”.

Check whether features encode the outcome indirectly. A final booking status, a post-period reconciliation field or a manually corrected demand figure can make a backtest look excellent while being unavailable in real planning. The evaluation integrity guide addresses leakage and selective test use across AI systems.

Use evaluation windows that resemble forecasting

Forecasting: Principles and Practice describes rolling-origin evaluation using information preceding each forecast and discusses matching evaluation to the required horizon. This is methodological guidance from the authors, not a regulatory requirement. The audit should verify the actual dates and procedure used for the model under review.

At each historical origin, fit or update the model according to the proposed operating process, generate the forecast and compare it with the later outcome. Preserve which observations influenced each run. If production retrains monthly, a backtest that refits after every new observation may describe a different service unless that difference is intentional and disclosed.

Separate horizons when they support different decisions. Good one-week-ahead performance does not establish useful four-week-ahead performance. Overlapping forecast windows and repeated observations from one region are related; the uncertainty analysis should respect that structure. Do not treat every forecast cell as an independent experiment.

Compare against a meaningful baseline

A baseline might use a recent observed level, a relevant seasonal comparison or the current planning method. Select it for the task and available history, not because it is easy for the model to beat. Give the baseline the same permitted information and outcome definition. If it uses a different horizon or later data, the comparison needs qualification.

Evaluate more than one period where the evidence permits. A model can outperform during a stable interval and underperform after a change in request patterns. Segment results by relevant conditions, while keeping counts and uncertainty visible. The goal is to understand where added complexity helps and where the simpler approach remains competitive.

Preserve the baseline procedure alongside the model artefact. A comparison cannot be reproduced if “manual forecast” refers to an undocumented recollection or a later revised plan. Where the historical human forecast is unavailable, say so and use an explicitly defined alternative rather than inventing a record of what planners would have predicted.

Separate model selection from confirmation

Ask how the final model was chosen from the candidates tried. If developers repeatedly inspected the same historical periods while choosing features, tuning settings or revising exclusions, those periods influenced development. Calling them a test set does not restore their independence. Record the sequence of decisions and distinguish exploratory comparisons from evidence obtained after the design was frozen.

For the rental example, a defensible confirmation exercise might use a later reserved interval that was not available during model selection, with the same issue schedule and data-availability boundary intended for operation. That is a proposed procedure, not a fixed rule for every dataset. Sparse history may make a separate interval impractical; the report should then explain the resulting uncertainty and any restricted initial use, rather than inventing a clean holdout.

Check whether the apparent gain depends on one region or a favourable demand cycle. Examine the direction and stability of differences across appropriate historical origins without treating correlated observations as independent proof. Preserve unfavourable comparisons. If a material segment has too little evidence, name it explicitly and identify what additional observation would support a decision.

After deployment, an evaluation window becomes historical evidence for that released version. If its results are used to redesign the next model, disclose that development role. Maintain a sequence of version, selection evidence and subsequent confirmation so that repeated reuse of a convenient benchmark is not mistaken for repeated independent validation.

Worked example: average error and its limits

The following hypothetical four-period example uses one-week-ahead forecasts issued at separate historical origins for one region. All outcomes are assumed observed under the same demand definition. It illustrates arithmetic only; four observations are not adequate evidence for a production deployment decision.

Illustrative forecast comparison in requested equipment units

Period	Actual demand	Model forecast	Baseline forecast	Model absolute error	Baseline absolute error
1	8	10	8	2	0
2	12	10	8	2	4
3	0	2	4	2	4
4	10	8	8	2	2

The model's mean absolute error is eight total units of absolute error divided by four periods, or 2 units. The baseline's is ten divided by four, or 2.5 units. The model has 20% lower mean absolute error in this small illustrative set. That statement describes the table; it is not evidence of a general improvement for other regions or future periods.

Inspect direction as well as magnitude. The model overpredicts periods 1 and 3 and underpredicts periods 2 and 4. Those errors may have different operational consequences. The zero-demand period also makes an ordinary percentage-error calculation problematic because it would divide by zero. Do not silently drop such periods to obtain an attractive percentage metric.

Review uncertainty and decision-relevant error

If the model produces intervals or quantiles, examine what they are intended to represent and how they were evaluated. A point forecast and a prediction interval answer different questions. Check empirical coverage over suitable periods and groups, together with interval width. An extremely wide interval can cover outcomes while offering little planning value.

Keep calibration of uncertainty separate from average error. A model may have useful point predictions but underestimate variability. Review whether the uncertainty method reflects relevant conditions, including new regions and unusual request patterns. Do not extend a well-behaved interval result from one population to another without evidence.

Where the business uses asymmetric loss or a service-level objective, document the assumptions and decision process. A lower overall error does not guarantee the preferred balance between shortage and idle capacity. The auditor should explain what the metric shows and what operational judgement remains, rather than presenting one optimisation target as universally correct.

Examine missingness, zeros and constrained observations

A missing record, a true zero and a system outage are not the same outcome. Inspect how each enters training and evaluation. Filling every missing week with zero can create a false demand pattern. Removing every zero can make the model appear useful only when demand was already present.

Capacity constraints can also affect the observed target. If staff stop recording enquiries when equipment is unavailable, the source may understate requested demand. The audit should identify that data-generation process and what can be concluded from it. A model cannot recover an unobserved quantity merely because the output is labelled demand.

Review cancellations and duplicate requests consistently. A later cancellation may be relevant to fulfilled demand but not to the original planning question. Define the intended outcome with the owner, preserve alternative measures where useful and avoid changing the rule after seeing which one favours the model.

Test the deployed scoring and planning pipeline

Reconcile a controlled input snapshot through feature preparation, model output, regional aggregation and the planner's interface. A correct model file can be undermined by stale inputs, a changed unit conversion or an incorrect join. Compare intermediate values where needed to locate the mechanism instead of attributing every discrepancy to model quality.

Check the version recorded with each forecast and the behaviour when inputs arrive late. A fallback forecast may be appropriate, but users should be able to distinguish it from the normal model output. Test the documented process for unavailable regions, missing required fields and failed scoring jobs.

Inspect aggregation. A total forecast can look accurate while regional errors cancel each other. If planners allocate equipment by region, regional performance matters. The audit should report at the level of the decision and explain whether totals reconcile, rather than choosing whichever aggregation produces the best score.

Review human adjustments without assuming they are ground truth

Planners may know about events or constraints not captured by the model. Record the original forecast, adjustment, reason and information available at the time. Compare outcomes where evidence permits, but recognise that adjusted cases are selected rather than random. A simple adjusted-versus-unadjusted comparison may not identify the effect of the human intervention.

Determine whether repeated adjustments indicate a model limitation, a missing input or an undocumented change in the target. A planner correcting a recurring regional issue may be compensating for a defect that should be addressed directly. The human oversight guide examines whether reviewers have the information and authority their role requires.

Do not automatically use every human adjustment as a future training label. Some are preferences or precautionary decisions rather than observations of demand. Preserve the distinction between the forecast, the chosen plan and the eventual outcome. Mixing them can teach the model to reproduce previous planning actions instead of predicting the intended target.

Monitor change without treating drift as a diagnosis

A change in inputs or forecast errors can signal a problem, but it does not identify the cause by itself. Possible explanations include data-pipeline changes, new request behaviour, a changed portfolio of regions or a new outcome definition. Investigate those alternatives before declaring that the model has deteriorated or needs retraining.

Use relevant monitoring windows and outcome maturity. Recent forecasts may not yet have complete outcomes. Compare like horizons and populations, and retain missing or pending outcomes visibly. The continuous auditing guide explains how recurring evidence and periodic review can work together without implying that every forecast is independently audited.

Define response ownership. Some signals need data correction, some require a use restriction and others justify model redevelopment. Retraining on a corrupted target can reinforce the problem. The audit should examine the decision process and the evidence supporting the chosen response, not only whether a new model was deployed.

Write findings that separate observation from explanation

A finding might establish that a backtest used booking updates received after the forecast origin. The supported conclusion is that the reported result does not represent the stated information boundary. It does not automatically establish how poor the live model is. The next step is a corrected as-of evaluation, with the affected decisions identified.

Another finding might concern production aggregation rather than statistical quality. If regional outputs are assigned to the wrong labels, engineering correction and implementation reconciliation are needed. Keep that mechanism separate from a recommendation to replace the forecasting algorithm, which the evidence may not support.

Use retesting evidence to close the actual finding. Preserve the original data, calculation or pipeline discrepancy, identify the correction and rerun the affected procedure. State remaining population and time limits through the coverage record. A successful correction of one feature does not validate every use of the model.

Prepare for a predictive-model audit

Provide the target definition, forecast origins, feature availability records, model and pipeline versions, outcome rules, baseline procedure and complete result tables. Include examples of missing data, human adjustments and known weak segments. A reproducible modest result is more useful than a polished chart whose historical information boundary cannot be reconstructed.

Discuss an audit of your predictive model and planning evidence to define the questions that matter for use, identify the data needed to answer them and establish which decisions remain conditional on further evidence.