Artificial Intelligence Test, Evaluation, Verification, and Validation (TEVV) Capability Request for Information (RFI)

Introduction

The Defense Intelligence Agency (DIA) recognizes that the development, deployment, and operation of artificial intelligence (AI) and AI-enabled capabilities (AIEC) within the Intelligence Community (IC) and Department of War (DoW) depends fundamentally upon rigorous, comprehensive, and continuous test, evaluation, verification, and validation (TEVV) activities. As AIEC become increasingly central to intelligence operations and national security decision-making, establishing trust through demonstrated reliability, cybersecurity, safety, accuracy, robustness, and adherence to ethical and legal standards is paramount. Effective TEVV of AIEC serves as the cornerstone for building confidence among operational users, mission owners, senior leadership, and policy makers that AI systems perform as intended, produce trustworthy outputs, mitigate unintended risks, and align with established legal and regulatory frameworks. DIA seeks to implement a mature AI TEVV capability to accelerate the adoption and acceptance of transformative AI technologies throughout the Defense Intelligence Enterprise (DIE).

Background and Motivation

DIA is exploring methods to develop standardized, scalable, and measurable TEVV metrics, as well as improve upon existing TEVV frameworks to assess technical performance of AIEC. These metrics must also effectively communicate the trustworthiness and operational readiness of AIEC to users, senior leaders, decision makers, and policy makers who must authorize their deployment into high-consequence environments.

DIA's responsibility extends beyond internal enterprise systems to include oversight authorities across multiple DIE domains under various DIE Manager (DIEM) authorities. While DIA seeks an enterprise solution for this Agency, certain mission areas may require specialized or extended TEVV approaches. DIA recognizes the critical importance of partnering with ongoing AI TEVV initiatives across DoW components, IC elements, and other federal agencies to leverage existing investments, reduce duplication of effort, promote interoperability, and align to broader community-wide standards and best practices that are rapidly evolving.

General Requirements

Therefore, DIA seeks concepts that provide both enterprise-level standardization and the flexibility to accommodate mission-specific validation requirements, while minimizing duplicative or conflicting processes. DIA believes that effective AI TEVV must address a spectrum of system types supporting analysis, collection, operations, targeting, and business management, amongst others. Each category may require tailored TEVV approaches while maintaining consistency with overarching standards and enterprise governance. Furthermore, AIEC supporting DIA may span from development/training environments through operational deployment across multiple security domains (NIPRNET, SIPRNET, JWICS, etc..) and classification levels. Solutions must account for testing and validation requirements across this continuum, including appropriate isolation, data management, and security considerations for each domain. Lastly, DIA is particularly interested in TEVV capabilities that incorporate automation to enable continuous testing and evaluation throughout the AIEC’s lifecycle, from development through deployment and operational monitoring.

Conclusion

Any proposed capability must align with applicable policies and directives including relevant Intelligence Community Directives (ICDs), Department of Defense (DoD) Manuals (DoDMs), and DoD Instructions (DoDIs), while incorporating both traditional DoW Developmental Test & Evaluation (DT&E) and Operational Test & Evaluation (OT&E) best practices and recently-published AI-specific guidance, frameworks, and testing manuals that address the distinctive challenges of evaluating non-deterministic, data-dependent systems.

Through this RFI, DIA seeks to discover industry best practices, innovative approaches, mature methodologies, and state-of-the-art tooling to bolster its AI TEVV capabilities and address the unique requirements of intelligence mission systems. DIA invites industry partners to share their expertise, lessons learned, technical solutions, and recommended strategies that will enable the Agency to establish a robust, scalable, and sustainable AI TEVV capability appropriate for the intelligence enterprise.