保护大型语言模型标记器以保护人工智能应用程序

在最近的一篇博文中，NVIDIA 的 AI 红队揭示了大型语言模型 (LLM) 标记器中的潜在漏洞，并提供了缓解这些风险的策略。

Large language models (LLMs) have rapidly gained prominence in various AI applications. However, ensuring their security is crucial, as vulnerabilities can arise at different stages of the LLM pipeline. One critical component that often goes overlooked is the tokenizer, which plays a pivotal role in preparing input strings for LLM processing. If not adequately secured, tokenizers can introduce potential risks that might go unnoticed.

大型语言模型（LLM）在各种人工智能应用中迅速获得关注。然而，确保其安全性至关重要，因为 LLM 管道的不同阶段可能会出现漏洞。分词器是一个经常被忽视的关键组件，它在为 LLM 处理准备输入字符串方面发挥着关键作用。如果没有充分保护，代币化器可能会带来可能被忽视的潜在风险。

In a recent blog post, NVIDIA's AI Red Team sheds light on these vulnerabilities and provides strategies to mitigate them. Here's a summary of their findings and recommendations.

在最近的一篇博文中，NVIDIA 的 AI 红队揭示了这些漏洞并提供了缓解这些漏洞的策略。以下是他们的发现和建议的摘要。

Understanding the Vulnerability

了解漏洞

Most LLMs, such as those from OpenAI and NVIDIA, share a common architecture, consisting of a tokenizer, encoder, decoder, and loss function. Among these components, the tokenizer is typically reused across multiple models and is usually stored as a plaintext .json file in the model directory.

大多数法学硕士（例如 OpenAI 和 NVIDIA 的法学硕士）共享一个通用架构，由分词器、编码器、解码器和损失函数组成。在这些组件中，标记生成器通常在多个模型中重用，并且通常作为纯文本 .json 文件存储在模型目录中。

This design makes sense from a practical standpoint, as it eliminates the need to re-tokenize data for each model and reduces overall model size. However, it also introduces a vulnerability, as anyone with sufficient privileges can access and modify the tokenizer configuration.

从实用的角度来看，这种设计是有意义的，因为它消除了为每个模型重新标记数据的需要，并减少了整体模型的大小。然而，它也引入了一个漏洞，因为任何拥有足够权限的人都可以访问和修改标记器配置。

An attacker could potentially alter the tokenizer's mapping of strings to token IDs, creating discrepancies between the user's input and the model's interpretation. For example, they could change the mapping of the word “deny” to the token ID associated with “allow.”

攻击者可能会更改标记生成器从字符串到标记 ID 的映射，从而在用户输入和模型解释之间造成差异。例如，他们可以更改单词“拒绝”到与“允许”关联的令牌 ID 的映射。

As a result, when a user inputs a string containing the word “deny,” the tokenizer would assign it the token ID for “allow,” fundamentally changing the meaning of the input. This scenario exemplifies an encoding attack, where the model processes an altered version of the user's intended input.

因此，当用户输入包含“拒绝”一词的字符串时，分词器会为其分配“允许”的标记 ID，从根本上改变输入的含义。此场景举例说明了编码攻击，其中模型处理用户预期输入的更改版本。

Attack Vectors and Exploitation

攻击向量和利用

There are several attack vectors that can be used to target tokenizers. One method involves placing a script in the Jupyter startup directory to modify the tokenizer before the pipeline initializes. This approach would enable an attacker to alter the tokenizer configuration without directly accessing the model code.

有多种攻击向量可用于针对分词器。一种方法是将脚本放置在 Jupyter 启动目录中，以在管道初始化之前修改分词器。这种方法将使攻击者能够在不直接访问模型代码的情况下更改标记器配置。

Another strategy could involve modifying tokenizer files during the container build process, facilitating a supply chain attack. For instance, an attacker might introduce a backdoor into the tokenizer by adding a custom mapping that assigns a unique token ID to a specific string.

另一种策略可能涉及在容器构建过程中修改标记器文件，从而促进供应链攻击。例如，攻击者可能会通过添加将唯一令牌 ID 分配给特定字符串的自定义映射，将后门引入到分词器中。

This backdoor could then be used to trigger a particular behavior or response from the model, even if the input string is modified by other parties. Such techniques highlight the importance of maintaining a secure and controlled build environment.

然后，即使输入字符串被其他方修改，该后门也可用于触发模型的特定行为或响应。此类技术强调了维护安全且受控的构建环境的重要性。

Moreover, attackers might attempt to exploit cache behaviors by directing the system to use a cache directory under their control, enabling them to inject malicious configurations. These actions emphasize the need for runtime integrity verifications to complement static configuration checks.

此外，攻击者可能会尝试通过指示系统使用他们控制下的缓存目录来利用缓存行为，从而使他们能够注入恶意配置。这些操作强调需要运行时完整性验证来补充静态配置检查。

Mitigation Strategies

缓解策略

To counter these threats, NVIDIA recommends several mitigation strategies.

为了应对这些威胁，NVIDIA 推荐了多种缓解策略。

1. Strong Versioning and Auditing:

1. 强大的版本控制和审计：

When tokenizers are inherited as upstream dependencies, their versions should be explicitly specified and audited to ensure they align with the intended model configuration. This practice helps prevent inheriting outdated or compromised tokenizer versions.

当标记生成器作为上游依赖项继承时，应显式指定并审核其版本，以确保它们与预期的模型配置保持一致。这种做法有助于防止继承过时或受损的标记生成器版本。

2. Runtime Integrity Checks:

2. 运行时完整性检查：

Implementing runtime integrity checks can help detect unauthorized modifications to the tokenizer at runtime. These checks can verify the integrity of the tokenizer file and its contents, ensuring that it operates as intended and has not been tampered with during execution.

实施运行时完整性检查可以帮助检测运行时对标记生成器的未经授权的修改。这些检查可以验证分词器文件及其内容的完整性，确保其按预期运行并且在执行过程中未被篡改。

3. Logging Practices:

3. 记录实践：

Logging practices can aid in forensic analysis by providing a clear record of input and output strings, helping to identify any anomalies resulting from tokenizer manipulation. Logs should capture both the original input string and the tokenized output, enabling investigators to quickly pinpoint any discrepancies.

日志记录实践可以通过提供输入和输出字符串的清晰记录来帮助取证分析，帮助识别由标记器操作引起的任何异常。日志应捕获原始输入字符串和标记化输出，使调查人员能够快速查明任何差异。

Conclusion

结论

The security of LLM tokenizers is paramount to maintaining the integrity of AI applications. Malicious modifications to tokenizer configurations can lead to severe discrepancies between user intent and model interpretation, undermining the reliability of LLMs.

LLM 标记器的安全性对于维护人工智能应用程序的完整性至关重要。对分词器配置的恶意修改可能会导致用户意图和模型解释之间的严重差异，从而损害法学硕士的可靠性。

By adopting robust security measures, including version control, auditing, and runtime verification, organizations can safeguard their AI systems against such vulnerabilities. For more insights on AI security and to stay updated on the latest developments, consider exploring the upcoming NVIDIA Deep Learning Institute course on Adversarial Machine Learning.

通过采用强大的安全措施，包括版本控制、审计和运行时验证，组织可以保护其人工智能系统免受此类漏洞的影响。如需了解有关 AI 安全性的更多见解并了解最新进展，请考虑探索即将推出的 NVIDIA 深度学习学院有关对抗性机器学习的课程。