保護大型語言模型標記器以保護人工智慧應用程式

在最近的一篇部落格文章中，NVIDIA 的 AI 紅隊揭示了大型語言模型 (LLM) 標記器中的潛在漏洞，並提供了緩解這些風險的策略。

Large language models (LLMs) have rapidly gained prominence in various AI applications. However, ensuring their security is crucial, as vulnerabilities can arise at different stages of the LLM pipeline. One critical component that often goes overlooked is the tokenizer, which plays a pivotal role in preparing input strings for LLM processing. If not adequately secured, tokenizers can introduce potential risks that might go unnoticed.

大型語言模型（LLM）在各種人工智慧應用中迅速獲得關注。然而，確保其安全性至關重要，因為 LLM 管道的不同階段可能會出現漏洞。分詞器是一個經常被忽視的關鍵組件，它在為 LLM 處理準備輸入字串方面發揮關鍵作用。如果沒有充分保護，代幣化器可能會帶來可能被忽視的潛在風險。

In a recent blog post, NVIDIA's AI Red Team sheds light on these vulnerabilities and provides strategies to mitigate them. Here's a summary of their findings and recommendations.

在最近的一篇部落格文章中，NVIDIA 的 AI 紅隊揭示了這些漏洞並提供了緩解這些漏洞的策略。以下是他們的發現和建議的摘要。

Understanding the Vulnerability

了解漏洞

Most LLMs, such as those from OpenAI and NVIDIA, share a common architecture, consisting of a tokenizer, encoder, decoder, and loss function. Among these components, the tokenizer is typically reused across multiple models and is usually stored as a plaintext .json file in the model directory.

大多數法學碩士（例如來自 OpenAI 和 NVIDIA 的法學碩士）共享一個通用架構，由分詞器、編碼器、解碼器和損失函數組成。在這些元件中，標記產生器通常在多個模型之間重複使用，並且通常作為純文字 .json 檔案儲存在模型目錄中。

This design makes sense from a practical standpoint, as it eliminates the need to re-tokenize data for each model and reduces overall model size. However, it also introduces a vulnerability, as anyone with sufficient privileges can access and modify the tokenizer configuration.

從實用的角度來看，這種設計是有意義的，因為它消除了為每個模型重新標記資料的需要，並減少了整體模型的大小。然而，它也引入了一個漏洞，因為任何擁有足夠權限的人都可以存取和修改標記器配置。

An attacker could potentially alter the tokenizer's mapping of strings to token IDs, creating discrepancies between the user's input and the model's interpretation. For example, they could change the mapping of the word “deny” to the token ID associated with “allow.”

攻擊者可能會更改標記生成器從字串到標記 ID 的映射，從而在使用者輸入和模型解釋之間造成差異。例如，他們可以更改單字「拒絕」到與「允許」關聯的令牌 ID 的對應。

As a result, when a user inputs a string containing the word “deny,” the tokenizer would assign it the token ID for “allow,” fundamentally changing the meaning of the input. This scenario exemplifies an encoding attack, where the model processes an altered version of the user's intended input.

因此，當使用者輸入包含「拒絕」一詞的字串時，分詞器會為其分配「允許」的標記 ID，從根本上改變輸入的含義。此場景舉例說明了編碼攻擊，其中模型處理使用者預期輸入的變更版本。

Attack Vectors and Exploitation

攻擊向量和利用

There are several attack vectors that can be used to target tokenizers. One method involves placing a script in the Jupyter startup directory to modify the tokenizer before the pipeline initializes. This approach would enable an attacker to alter the tokenizer configuration without directly accessing the model code.

有多種攻擊向量可用於針對分詞器。一種方法是將腳本放置在 Jupyter 啟動目錄中，以便在管道初始化之前修改分詞器。這種方法將使攻擊者能夠在不直接存取模型程式碼的情況下更改標記器配置。

Another strategy could involve modifying tokenizer files during the container build process, facilitating a supply chain attack. For instance, an attacker might introduce a backdoor into the tokenizer by adding a custom mapping that assigns a unique token ID to a specific string.

另一種策略可能涉及在容器建置過程中修改標記器文件，從而促進供應鏈攻擊。例如，攻擊者可能會透過新增將唯一令牌 ID 指派給特定字串的自訂映射，將後門引入分詞器。

This backdoor could then be used to trigger a particular behavior or response from the model, even if the input string is modified by other parties. Such techniques highlight the importance of maintaining a secure and controlled build environment.

然後，即使輸入字串被其他方修改，該後門也可用於觸發模型的特定行為或回應。此類技術強調了維護安全且受控的建置環境的重要性。

Moreover, attackers might attempt to exploit cache behaviors by directing the system to use a cache directory under their control, enabling them to inject malicious configurations. These actions emphasize the need for runtime integrity verifications to complement static configuration checks.

此外，攻擊者可能會嘗試透過指示系統使用他們控制下的快取目錄來利用快取行為，從而使他們能夠注入惡意配置。這些操作強調需要運行時完整性驗證來補充靜態設定檢查。

Mitigation Strategies

緩解策略

To counter these threats, NVIDIA recommends several mitigation strategies.

為了應對這些威脅，NVIDIA 推薦了多種緩解策略。

1. Strong Versioning and Auditing:

1. 強大的版本控制和審計：

When tokenizers are inherited as upstream dependencies, their versions should be explicitly specified and audited to ensure they align with the intended model configuration. This practice helps prevent inheriting outdated or compromised tokenizer versions.

當標記生成器作為上游依賴項繼承時，應明確指定並審核其版本，以確保它們與預期的模型配置保持一致。這種做法有助於防止繼承過時或受損的標記產生器版本。

2. Runtime Integrity Checks:

2. 運行時完整性檢查：

Implementing runtime integrity checks can help detect unauthorized modifications to the tokenizer at runtime. These checks can verify the integrity of the tokenizer file and its contents, ensuring that it operates as intended and has not been tampered with during execution.

實施運行時完整性檢查可以幫助檢測運行時對標記產生器的未經授權的修改。這些檢查可以驗證分詞器檔案及其內容的完整性，確保其按預期運行並且在執行過程中未被篡改。

3. Logging Practices:

3. 記錄實踐：

Logging practices can aid in forensic analysis by providing a clear record of input and output strings, helping to identify any anomalies resulting from tokenizer manipulation. Logs should capture both the original input string and the tokenized output, enabling investigators to quickly pinpoint any discrepancies.

日誌記錄實務可以透過提供輸入和輸出字串的清晰記錄來幫助取證分析，幫助識別標記器操作引起的任何異常。日誌應捕獲原始輸入字串和標記化輸出，使調查人員能夠快速查明任何差異。

Conclusion

結論

The security of LLM tokenizers is paramount to maintaining the integrity of AI applications. Malicious modifications to tokenizer configurations can lead to severe discrepancies between user intent and model interpretation, undermining the reliability of LLMs.

LLM 標記器的安全性對於維護人工智慧應用程式的完整性至關重要。對分詞器配置的惡意修改可能會導致使用者意圖和模型解釋之間的嚴重差異，從而損害法學碩士的可靠性。

By adopting robust security measures, including version control, auditing, and runtime verification, organizations can safeguard their AI systems against such vulnerabilities. For more insights on AI security and to stay updated on the latest developments, consider exploring the upcoming NVIDIA Deep Learning Institute course on Adversarial Machine Learning.

透過採用強大的安全措施，包括版本控制、稽核和執行時間驗證，組織可以保護其人工智慧系統免受此類漏洞的影響。如欲了解更多有關 AI 安全性的見解並了解最新進展，請考慮探索即將推出的 NVIDIA 深度學習學院有關對抗性機器學習的課程。