Apple 的 ReDrafter：透過閃電般快速的令牌生成徹底改變語言模型訓練

Apple 憑藉其創新方法使語言模型訓練速度快如閃電，在機器學習領域取得了顯著突破。這家科技巨頭的新方法稱為 ReDrafter，它將透過顯著加速代幣生成過程來徹底改變我們建構和部署人工智慧模型的方式。

Apple has made a remarkable breakthrough in the world of machine learning with its innovative approach to making language model training lightning-fast. The tech giant’s new method, called ReDrafter, is set to revolutionize the way we build and deploy AI models by significantly accelerating the token generation process.

Apple 憑藉其創新方法使語言模型訓練速度快如閃電，在機器學習領域取得了顯著突破。這家科技巨頭的新方法稱為 ReDrafter，它將透過顯著加速代幣生成過程來徹底改變我們建構和部署人工智慧模型的方式。

The Challenges in Building AI Models

建構人工智慧模型的挑戰

Developing large language models (LLMs) is known to be a resource-intensive undertaking. Traditional methods require substantial hardware investments and incur high energy costs. Earlier this year, Apple introduced ReDrafter, an open-sourced technique aimed at streamlining this process.

眾所周知，開發大型語言模型 (LLM) 是一項資源密集型工作。傳統方法需要大量的硬體投資並產生高昂的能源成本。今年早些時候，蘋果推出了 ReDrafter，這是一項旨在簡化此流程的開源技術。

A Breakthrough in Speed

速度的突破

ReDrafter, which utilizes a Recurrent Neural Network (RNN) draft model, leverages a unique combination of beam search and dynamic tree attention. This innovation has led to LLM token generation speeds up to 3.5 times faster than conventional auto-regressive techniques. Now, Apple's ReDrafter is ready for prime time, particularly with Nvidia GPUs.

ReDrafter 採用循環神經網路 (RNN) 草圖模型，利用波束搜尋和動態樹注意力的獨特組合。這項創新使 LLM 代幣生成速度比傳統自回歸技術快 3.5 倍。現在，Apple 的 ReDrafter 已準備好迎接黃金時段，特別是在 Nvidia GPU 的幫助下。

Collaboration with Nvidia

與英偉達合作

Apple collaborated closely with Nvidia to integrate ReDrafter into the Nvidia TensorRT-LLM framework. This partnership has resulted in a significant 2.7-times speed increase in token generation during testing on Nvidia’s powerful GPUs, offering substantial benefits in terms of efficiency and hardware reduction.

Apple 與 Nvidia 密切合作，將 ReDrafter 整合到 Nvidia TensorRT-LLM 框架中。在 Nvidia 強大的 GPU 上進行測試時，這種合作關係使代幣生成速度顯著提高了 2.7 倍，在效率和硬體減少方面帶來了巨大的好處。

Impact on the AI Community

對人工智慧社群的影響

This advancement will not only mean faster responses for users but also reduced hardware expenses for companies, paving the way for more sophisticated AI models. Nvidia hailed the collaboration as enhancing TensorRT-LLM’s flexibility and power.

這項進步不僅意味著用戶的回應速度更快，而且還減少了公司的硬體支出，為更複雜的人工智慧模式鋪平了道路。 Nvidia 稱讚此次合作增強了 TensorRT-LLM 的靈活性和功能。

In light of these advances, Apple continues to explore new frontiers, previously indicating potential efficiency gains from using Amazon’s Trainium2 chip for future AI model training.

鑑於這些進步，蘋果公司繼續探索新領域，此前曾表明使用亞馬遜的 Trainium2 晶片進行未來人工智慧模型訓練可能會帶來效率提升。