Salesforce AI的CODA-1.7B：離散擴散代碼生成的飛躍

探索Salesforce AI的CODA-1.7B，這是代碼生成的離散擴散模型，提供較少的參數和可重現管道的競爭性能。

Salesforce AI is making waves with CoDA-1.7B, a compact yet powerful discrete-diffusion model designed for code generation. This model leverages bidirectional context and parallel token updates, marking a significant advancement in the field.

Salesforce AI正在使用CODA-1.7B進行浪潮，這是一種專為代碼生成而設計的緊湊而強大的離散擴散模型。該模型利用雙向上下文和平行的令牌更新，標誌著該領域的顯著進步。

Understanding CoDA-1.7B's Architecture and Training

了解CODA-1.7B的架構和培訓

CoDA-1.7B adapts a 1.7B-parameter backbone to discrete diffusion for text. It iteratively denoises masked sequences using full-sequence attention, enabling native infilling and non-autoregressive decoding. The model card documents a three-stage pipeline: pre-training with bidirectional masking, supervised post-training, and progressive denoising at inference. Reproducible scripts for TPU pre-training, GPU fine-tuning, and evaluation are also provided.

CODA-1.7B適應1.7B參數骨幹，以使文本離散擴散。它迭代地使用全面的注意來掩蓋了掩蓋序列，從而實現了本地填充和非自動向解析的解碼。該模型卡記錄了三階段的管道：雙向掩蔽，監督後訓練和推理時進行性降解的預訓練。還提供了用於TPU預訓練，GPU微調和評估的可再現腳本。

Benchmark Performance

基準性能

CoDA-1.7B-Instruct demonstrates impressive results on standard code-gen suites, including HumanEval (54.3%), HumanEval+ (47.6%), MBPP (47.2%), MBPP+ (63.2%), and EvalPlus aggregate (55.4%) pass@1. These results are competitive with some 7B diffusion models, such as Dream-7B-Instruct (57.9% HumanEval), while utilizing significantly fewer parameters.

CODA-1.7B教學結構在標準代碼套件上表現出令人印象深刻的結果，包括Humaneval（54.3％），HumaneVal+（47.6％），MBPP（47.2％），MBPP+（63.2％）和Evalplus croxpregate（55.4％）Pass@1。這些結果與某些7B擴散模型（例如Dream-7B - 教學（57.9％）人類Val）具有競爭力，同時使用明顯較少的參數。

Inference Behavior and Deployment

推理行為和部署

Generation cost in CoDA is governed by the number of diffusion steps. Users can tune latency/quality trade-offs using parameters like STEPS and ALG="entropy". The model updates tokens in parallel under full attention, which targets lower wall-clock latency at small scale compared with larger diffusion models. The release includes a FastAPI server with OpenAI-compatible APIs and an interactive CLI for local inference. Model cards and a Hugging Face collection centralize artifacts, with checkpoints published under CC BY-NC 4.0 on Hugging Face.

CODA中的發電成本受擴散步驟的數量約束。用戶可以使用步驟和ALG =“熵”等參數來調整潛伏期/質量權衡。該模型在全部注意力下並行更新標記，與較大的擴散模型相比，該模型以較小的壁式潛伏期為目標。該版本包括具有兼容openAI的API的FastAPI服務器和用於本地推斷的交互式CLI。模型卡和擁抱的臉部收集集中式人工製品，在CC BY-NC 4.0下出版了檢查點。

Our Take

我們的看法

CoDA-1.7B is a valuable reference for discrete-diffusion code generation at a smaller scale. Its bidirectional denoising with parallel token updates and reproducible pipeline from pre-training to SFT and serving make it an accessible and practical tool. The ability to tune throughput/quality using step count and decoding knobs is also operationally advantageous. I believe CoDA-1.7B is a step toward making AI code generation more efficient and accessible to developers.

CODA-1.7B是較小規模的離散擴散代碼生成的寶貴參考。它的雙向denoings通過平行令牌更新和可再現的管道從預培訓到SFT和服務，使其成為一種易於訪問且實用的工具。使用步驟計數和解碼旋鈕調整吞吐量/質量的能力在操作上也有利。我認為CODA-1.7B是使AI代碼生成更有效且可為開發人員訪問的一步。

So, what are you waiting for? Dive into the world of CoDA-1.7B and see how it can revolutionize your code generation workflow!

那麼，您還在等什麼？潛入Coda-1.7b的世界，看看它如何徹底改變您的代碼生成工作流程！