Photo credit: www.darkreading.com
PRESS RELEASE
The Chief Digital and Artificial Intelligence Office (CDAO) has successfully wrapped up a pilot of the Crowdsourced AI Red-Teaming (CAIRT) Assurance Program, focusing on the implementation of Large-Language Model (LLM) chatbots in military medicine. This initiative aims to enhance the Department of Defense’s (DoD) approach to AI assurance and risk mitigation by leveraging a grassroots, crowdsourced model. Such strategies are designed to gather extensive data and engage a diverse range of participants.
This CAIRT pilot was led by Humane Intelligence, a technology firm dedicated to building a collaborative framework for algorithmic assessments, in partnership with the Defense Health Agency (DHA) and the Program Executive Office, Defense Healthcare Management Systems (PEO DHMS). By applying red-teaming techniques, which involve using adversarial methods to rigorously test system resilience, Humane Intelligence successfully identified specific vulnerabilities within the systems tested. The approach attracted participants keen on engaging with innovative technologies, providing them the chance to contribute to the enhancement of these systems. In a prior exercise held in spring 2024, the CDAO executed a notable red-teaming initiative that deployed a financial bounty focused on unknown risks associated with LLMs.
During this latest pilot, crowdsourced red-teaming was applied to explore two potential applications within military medicine: clinical note summarization and a medical advisory chatbot. Over 200 participants, including clinical providers and healthcare analysts from the DHA, the Uniformed Services University of the Health Sciences, and various military branches, engaged in the exercise, which evaluated three well-known LLMs. The results revealed over 800 instances of possible vulnerabilities and biases related to the use of these technologies in the identified use cases. Importantly, the exercise will lead to the creation of reusable and scalable outputs through benchmark datasets, which are essential for assessing future vendors and tools to ensure they meet anticipated performance standards. Additionally, the insights gained will be instrumental in developing policies and best practices within the DoD for the responsible application of Generative AI (GenAI), ultimately enhancing military healthcare services. If these applications are deployed and fall under the definitions established in OMB M-24-10, they will comply with requisite risk management protocols.
Dr. Matthew Johnson, the CDAO’s lead for this project, commented on its significance, stating, “The implementation of GenAI within the DoD is still in the formative stages of experimentation. This program serves as a foundational strategy for generating substantial testing data, identifying areas for further exploration, and validating options for risk mitigation, which will inform future research, development, and assurance efforts for GenAI systems that may be fielded.”
The insights acquired from this pilot, along with ongoing assessments of LLMs and AI systems through the CAIRT Assurance Program, will play a vital role in advancing the capabilities of the CDAO’s AI Rapid Capabilities Cell, enhancing the effectiveness of GenAI missions, and fostering confidence in DoD applications.
About the CDAO
Established in June 2022, the CDAO is committed to the integration and optimization of AI capabilities within the DoD. The office’s mission involves accelerating the Department’s utilization of data, analytics, and AI while fostering the advancement of digital infrastructure and policy initiatives to deliver scalable AI solutions for both enterprise and joint operational scenarios, ultimately strengthening national defense against evolving threats.
To learn more about the CDAO, please visit our website at ai.mil. Additionally, you can connect with the CDAO on LinkedIn (@ DoD Chief Digital and Artificial Intelligence Office) and X, formerly known as Twitter (@dodcdao). Further updates and information can also be accessed on the CDAO Unit Page on DVIDS.
Source
www.darkreading.com