A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations
CoRR(2024)
摘要
Responsible design of AI systems is a shared goal across HCI and AI
communities. Responsible AI (RAI) tools have been developed to support
practitioners to identify, assess, and mitigate ethical issues during AI
development. These tools take many forms (e.g., design playbooks, software
toolkits, documentation protocols). However, research suggests that use of RAI
tools is shaped by organizational contexts, raising questions about how
effective such tools are in practice. To better understand how RAI tools are –
and might be – evaluated, we conducted a qualitative analysis of 37
publications that discuss evaluations of RAI tools. We find that most
evaluations focus on usability, while questions of tools' effectiveness in
changing AI development are sidelined. While usability evaluations are an
important approach to evaluate RAI tools, we draw on evaluation approaches from
other fields to highlight developer- and community-level steps to support
evaluations of RAI tools' effectiveness in shaping AI development practices and
outcomes.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要