A template guide for governance teams
Model Documentation for Compliance: Model Cards and FactSheets
This guide provides governance teams with a structured approach to using model cards and FactSheets for AI model documentation to meet compliance requirements. It details the key components, recommended practices, and implementation considerations for effective model risk management.
In this guide · 6 steps
Enterprises face increasing regulatory scrutiny regarding the governance of AI models, especially in regulated industries. Effective documentation is critical to demonstrate compliance with internal policies and external regulations such as the EU AI Act, Federal Reserve guidance, or SEC standards. Two prominent documentation approaches are model cards and FactSheets, designed to provide transparent, standardized information about an AI model’s purpose, performance, limitations, and governance.
1. Understanding model cards and FactSheets
Model cards were introduced by Google researchers in 2018 as a concise, structured way to communicate the key characteristics of AI models. They focus on explaining the model’s intended use, data, evaluation metrics, fairness considerations, and ethical implications. FactSheets, proposed by IBM in 2019, extend this concept by adding a compliance-focused checklist that includes provenance, security controls, and validation processes.
Both documentation formats are recommended by industry and regulatory groups to improve transparency and accountability. The Partnership on AI and NIST highlighted model cards and FactSheets as best practices in their AI risk management frameworks. Governance teams should select or adapt these tools based on their compliance needs and operational environment.
2. Key components of model cards
A standard model card typically contains these sections: model details (name, version, developer team), intended use cases and users, training data description, model performance metrics with evaluation datasets, fairness and bias assessments, ethical considerations, and caveats or limitations. The document aims to be concise, easily interpretable by both technical and non-technical stakeholders, and updateable throughout the model lifecycle.
For compliance, governance teams should ensure the model card covers data lineage, dataset demographics, known biases, and results from stress tests or scenario analyses. Including an executive summary can help compliance officers quickly ascertain model risks and controls.
3. Key components of FactSheets
IBM’s FactSheets focus on a questionnaire-style format that addresses specific compliance points. These include model development process, testing and validation procedures, performance monitoring plans, information on fairness and explainability tools used, security measures like data encryption or access controls, and change management protocols.
FactSheets often incorporate checklists that map each documented item to a regulatory or internal control requirement. This structure supports audit readiness by making it possible to demonstrate due diligence across multiple compliance dimensions within one document.
4. Implementing model documentation in governance workflows
Governance teams should embed model card and FactSheet creation into the AI lifecycle early, ideally during model development or pre-deployment stages. Automating data collection for performance metrics and version tracking can enhance accuracy and reduce manual effort. Collaboration with data science, risk management, and legal teams ensures that documentation aligns with compliance standards.
Periodic updates to these documents are necessary to reflect model retraining, data drift, discovered biases, or changes in regulatory frameworks. Integration with AI governance platforms like IBM Watson OpenScale, Azure Machine Learning, or Google Cloud AI Platform can provide version control and access management for model documentation artifacts.
5. Template overview and practical tips
A practical template for model cards should begin with a header section listing the model name, version, ownership, and approval status. Subsequent sections can follow this structure: 1) Model overview and intended use 2) Training and validation data description 3) Evaluation metrics and performance results 4) Fairness and bias assessment 5) Limitations and caveats 6) Responsible AI considerations 7) Compliance alignment summary.
For FactSheets, include a structured questionnaire covering development controls, testing protocols, security practices, operational monitoring, incident handling, and compliance mapping. Maintain a version history with author and reviewer annotations to support audit trails.
Best practice
Involve cross-functional review cycles for model documentation. Legal, compliance, and security teams provide critical feedback to ensure completeness and regulatory alignment.
6. Conclusion: Ensuring compliance through structured model documentation
Model cards and FactSheets serve complementary roles in meeting AI governance and compliance requirements. Model cards facilitate transparency and ethical reflection, while FactSheets emphasize compliance controls and audit readiness. Governance teams should tailor these templates to their regulatory environment and embed documentation as a mandatory deliverable within the AI life cycle. Doing so strengthens risk management and evidences due diligence to regulators and auditors.
Model Documentation Compliance Checklist
- Define scope and audience for model documentation
- Include detailed model metadata and ownership
- Document training data sources, characteristics, and biases
- Report evaluation metrics across diverse datasets
- Assess fairness and impact on protected groups
- Describe ethical considerations and limitations
- Map documentation items to regulatory requirements
- Implement version control and access governance
- Establish updating schedule tied to model lifecycle events
- Conduct regular cross-functional reviews and approvals