Production AI Institute — vendor-neutral certification for AI practitioners
Verify a credentialFor organisationsContact

Exam development process

PAI examinations are built to a structured development process. This page describes the standards that govern how AIDA and AIMA examination items are authored, reviewed, and maintained.

Development principles

Practitioner-authored
Examination items are grounded in real deployment decisions — drawn from the failure modes, tradeoffs, and edge cases that arise in production AI environments. Items are not sourced from academic literature or vendor documentation.
Framework-anchored
Every item maps to a specific PSF domain and competency level. The exam blueprint (published at /certify/blueprint) shows the weighting of each domain. Passing the exam demonstrates coverage across all eight domains, not just the ones a candidate finds comfortable.
Scenario-based
Items present realistic production scenarios — an AI system behaving unexpectedly, a data governance decision, an architecture choice — rather than testing vocabulary recall. The correct answer requires applying PSF principles, not memorising definitions.
Regularly reviewed
The item bank is reviewed annually. Items with poor discrimination (where high performers and low performers choose answers at similar rates) are flagged for revision or removal. Items are also reviewed when the PSF is updated to ensure continued alignment.

Development stages

01
Domain mapping
The exam blueprint defines the competencies to be tested within each PSF domain, drawing on the framework specification and practitioner-reported incident patterns.
02
Item authoring
Practitioners with relevant domain experience author candidate items. Each item is submitted with a rationale explaining the correct answer, the distractor logic, and the PSF reference.
03
Expert review
Items are reviewed by a practitioner who was not involved in authoring the item. Reviewers assess accuracy, scenario authenticity, and freedom from vendor-specific assumptions.
04
Sensitivity review
Items are reviewed for unintended cultural assumptions, ambiguous language, and any content that might disadvantage candidates from particular backgrounds or regions.
05
Staged introduction
New items are introduced alongside established items. Performance data is collected and reviewed before any item is treated as representative of the scored pool.
06
Performance review
Items that generate consistent confusion or low-quality feedback are revised or retired. Item quality is assessed against real candidate responses over time.
07
Ongoing maintenance
The scored item pool is reviewed annually. Items are retired if the PSF is updated such that the item no longer accurately reflects the standard, or if statistical performance degrades.

Examination specifications

AIDA
AI Deployment Associate
Items20 scored items
Time45 minutes
FormatMultiple choice, single correct answer
Pass mark70% (14/20)
DomainsAll eight PSF domains
AIMA
AI Management Associate
Items15 scored items
Time45 minutes
FormatMultiple choice, single correct answer
Pass mark73% (11/15)
DomainsGovernance, risk, compliance, team management
View all certifications →