Framework NoteAugust 2024CC BY 4.0
PSF v1.0 Rationale and Development History
This document records the reasoning behind each PSF domain and the choices made during framework development. It is intended to help practitioners understand why the framework is structured as it is, and to provide the basis for community feedback on future versions.
Why eight domains
The eight-domain structure emerged from iterative analysis of documented AI production failures and near-misses collected during the framework’s development phase. The initial scoping exercise identified fourteen candidate domains. These were consolidated to eight through a process of merging domains where the engineering controls were substantially similar, and eliminating domains where the failure modes were adequately covered by existing non-AI-specific standards (such as ISO 27001 for general information security).
The eight remaining domains represent the areas where AI systems in production settings create failure modes that are meaningfully different from those of conventional software — either because the failure modes are novel (prompt injection, hallucination, model drift) or because existing standards do not provide sufficient specificity for AI deployment contexts (human oversight governance, output contract design).
Domain-level rationale
PSF-1 Input Governance
Rationale: Placed first because input controls are the first line of defence and the earliest intervention point. Input governance failures — particularly prompt injection — are the most common root cause in the incident dataset that informed the framework. An early domain position also signals that input validation is not optional or post-deployment; it is a design-phase requirement.
Alternatives considered: An alternative structure placed input and output controls in a single domain. This was rejected because the engineering controls are sufficiently different to warrant separate treatment, and because conflating them in assessments obscured weaknesses in one that were masked by strength in the other.
PSF-2 Output Validation
Rationale: Output contracts — explicit specifications of what a valid AI output looks like — are the primary mechanism for making AI behaviour verifiable. Without an output contract, the system cannot detect when the AI is producing incorrect, malformed, or out-of-scope outputs. The domain name ‘Output Validation’ was chosen over ‘Output Contracts’ to emphasise that validation is an ongoing runtime activity, not just a design artefact.
Alternatives considered: Several reviewers suggested that output validation should be part of PSF-4 (Observability). This was rejected on the grounds that output validation is a functional correctness control, not a monitoring control, and that treating it as observability risk undervaluing it.
PSF-3 Data Protection
Rationale: AI systems present specific data protection risks not fully addressed by general data governance frameworks: training data memorisation, PII leakage through model outputs, and cross-context contamination when RAG systems surface documents to users who should not see them. PSF-3 addresses these AI-specific risks rather than duplicating GDPR compliance.
Alternatives considered: PSF-3 was initially broader, covering data quality for training as well as runtime data protection. The training data quality elements were removed from v1.0 on the grounds that the framework targets deployment, not development, and that training data governance is a provider responsibility rather than a deployer responsibility.
PSF-4 Observability
Rationale: Observability is the foundational requirement for all other controls: without logging and alerting, practitioners cannot know whether their input validation, output contracts, or human oversight mechanisms are functioning. The domain was defined broadly to include not just technical logging but the organisational capacity to act on what is observed.
Alternatives considered: The alternative was to include observability requirements within each other domain rather than as a standalone. This was rejected because it would allow organisations to satisfy domain-level requirements while having no integrated monitoring capability.
PSF-5 Deployment Safety
Rationale: AI deployments have failure modes specific to deployment transitions — particularly silent model version changes and the absence of rollback procedures — that are not well-covered by existing change management frameworks. The blast radius concept — limiting the maximum scope of AI-initiated actions — was introduced in PSF-5 to address the risk of unbounded autonomous action.
Alternatives considered: Whether blast radius governance should be in PSF-5 or PSF-6 was debated at length. It was placed in PSF-5 because blast radius is a deployment configuration decision, set before the system runs, rather than a runtime human oversight decision.
PSF-6 Human Oversight
Rationale: Human oversight governance was given a dedicated domain because the failure mode — the rubber-stamp problem — is structural and not addressed by any other domain. The autonomy level taxonomy (L0–L4) was introduced in PSF-6 to provide a vocabulary for specifying and auditing oversight requirements consistently across different system types.
Alternatives considered: Some reviewers argued that human oversight requirements belong in individual domain assessments rather than as a standalone domain. This was rejected because oversight governance failures cut across all domains and require organisational-level treatment.
PSF-7 Security
Rationale: Existing security standards (ISO 27001, SOC 2) do not address AI-specific attack vectors such as prompt injection, model inversion, or adversarial perturbation. PSF-7 covers these AI-specific threats. It does not attempt to replace general security standards and assessors should confirm that general standards are also met.
Alternatives considered: Whether PSF-7 should be merged with PSF-1 (which also addresses prompt injection) was considered. The two domains were kept separate because PSF-1 addresses input governance as a functional correctness concern, while PSF-7 addresses it as an adversarial security concern. The controls overlap but the risk framing differs.
PSF-8 Vendor Resilience
Rationale: Production AI systems depend on external model providers in ways that create resilience risks not present in conventional software supply chains: model deprecation, provider outages, silent capability changes, and pricing changes that alter system economics. PSF-8 addresses these risks and was placed last to signal that it is a dependency on all other domains — resilience cannot be assessed until the rest of the system is defined.
Alternatives considered: PSF-8 was initially titled ‘Supply Chain Resilience’. It was renamed ‘Vendor Resilience’ to make clear that it addresses the specific vendor relationship risks of AI deployment rather than the broader supply chain concerns addressed by existing frameworks.
Public comment period findings
The PSF v1.0 draft was open for public comment for 60 days prior to release. The most substantive feedback themes, and how they were addressed, are summarised below.
Domain weighting in AIDA assessment
Multiple respondents noted that the 20-question AIDA exam did not reflect the incident frequency data showing PSF-1 and PSF-2 as the most common failure domains. The question distribution was adjusted to increase PSF-1 and PSF-2 representation from 2 questions each to 3 questions each in the final exam design.
L4 autonomy level definition
Several practitioners argued that L4 (fully autonomous) should be prohibited rather than merely not recommended for high-stakes decisions. The framework retains the not-recommended formulation on the grounds that some fully autonomous operations on low-consequence tasks are appropriate, and that prohibiting L4 categorically would create compliance overhead for benign use cases.
Vendor resilience as a deployer obligation
Some respondents argued that vendor resilience is a provider responsibility, not a deployer responsibility, and should not be in a deployer-facing framework. The domain was retained because deployers who depend on a single provider without fallback are making a resilience decision, regardless of where the risk originates.