Skip to content

IARPA Request for Information – Characterizing Large Language Model Biases, Threats and Vulnerabilities (IARPA-RFI-23-03)

Home / News / IARPA Request for Information – Characterizing Large Language Model Biases, Threats and Vulnerabilities (IARPA-RFI-23-03)

IARPA Request for Information – Characterizing Large Language Model Biases, Threats and Vulnerabilities (IARPA-RFI-23-03)

twitterlinkedin

IARPA is seeking information on established characterizations of vulnerabilities and threats that could impact the safe use of large language models (LLMs) by intelligence analysts.

This RFI aims to elicit frameworks to categorize and characterize vulnerabilities and threats associated with LLM technologies, specifically in the context of their potential use in intelligence analysis.

Responses to this RFI should answer any or all of the following questions:

  1. Does your organization have a framework for classifying and understanding the range of LLM threats and/or vulnerabilities? If so, please describe this framework, and briefly articulate for each threat and/or vulnerability and its risks.
  2. Has your organization identified specific LLM threats and vulnerabilities that are not well-characterized by prior taxonomies (cf “OWASP Top 10 for LLM”)?  If so, please provide specific descriptions of each such threat and/or vulnerability and its impacts.
  3. Does your organization have any novel methods to detect or mitigate threats to users posed by LLM vulnerabilities?
  4. Does your organization have novel methods to quantify confidence in LLM outputs?

For the purpose of this RFI, IARPA is interested in the characterizations and methods for both “white box” models (some privileged access to parameters or code) and “black box” models (no privileged access to parameters and code).

Responses are due 12 August 2023.

Read more

Documentation

Request for Information – Characterizing Large Language Model Biases, Threats and Vulnerabilities

You may be also interested in ...