AI Ethics Checklist

Document the specific problem your AI system aims to solve and intended benefits

Map all stakeholders who will be affected by the AI system, including end users and impacted communities

Conduct risk assessment for potential negative impacts across different user groups

Evaluate whether AI is the best solution or if simpler alternatives exist

Define measurable success criteria beyond technical performance (fairness, safety, user satisfaction)

Design documentation and communication strategy for how the AI system works

Conduct privacy impact assessment if processing personal data

Ensure diverse perspectives in the team designing the AI system

Obtain proper informed consent for data collection and use

Collect only data that is necessary and relevant for the stated purpose

Ensure training data represents diverse user populations to avoid bias

Verify legitimacy and ethics of data sources (avoid scraped or unauthorized data)

Identify and implement special protections for sensitive attributes (race, religion, health, etc.)

Define and document how long data will be retained and disposal procedures

Implement encryption, access controls, and security protocols for data storage

Ensure all third-party data providers have proper licenses and ethical standards

Test model performance across different demographic groups to identify bias

Implement techniques to explain model decisions (SHAP, LIME, attention visualization)

Test model behavior with edge cases, adversarial examples, and out-of-distribution data

Ensure acceptable performance levels across all demographic groups

Document how and when the model fails, and potential consequences

Create comprehensive model cards documenting architecture, training data, and limitations

Implement version control for models, datasets, and training code

Measure and consider carbon footprint of training and inference

Clearly disclose when users are interacting with an AI system

Implement human-in-the-loop or human-on-the-loop for high-stakes decisions

Provide users ability to opt-out of AI-driven decisions where appropriate

Ensure AI system interface is accessible to users with disabilities

Implement channels for users to report issues, errors, or concerns

Establish procedures for responding to AI system failures or harmful outputs

Clear terms outlining AI system capabilities, limitations, and user responsibilities

Consider phased deployment to monitor real-world performance before full launch

Continuously monitor model performance and accuracy in production

Regularly audit for emerging bias or fairness issues in production

Monitor for distribution shifts in input data that may degrade performance

Measure actual impact on users compared to intended outcomes

Conduct periodic third-party audits of AI system fairness and safety

Establish criteria and process for when model should be retrained

Regularly communicate changes, updates, and performance to stakeholders

Have a plan for responsible decommissioning if system becomes obsolete or harmful