Kimi K2.7-Code Reduces Thinking Tokens by 30%—Experts Question the Validity of the Benchmarks
This week, Moonshot AI unveiled the latest version of its K2 coding model, named Kimi K2.7-Code, which is an open-source update designed to enhance reasoning efficiency and deliver significant performance improvements.
K2.7-Code is constructed using the same trillion-parameter mixture-of-experts architecture as its forerunner, K2.6, and integrates seamlessly via an OpenAI-compatible API, making it easier for teams currently utilizing K2.6 in their production environments.
When K2.6 was introduced in April, it became the leader on OpenRouter’s weekly LLM leaderboard, which ranks models based on actual API routing choices made by developers—not on self-reported benchmarks.
According to Moonshot AI, K2.7-Code is designed to minimize what they describe as “overthinking.” It reportedly reduces the use of thinking tokens by 30% compared to K2.6, which could lower inference costs for teams implementing automated workflows. Nevertheless, questions regarding whether these efficiency improvements will hold up in independent benchmarks have started to surface within the community.
Overview of Kimi K2.7-Code
K2.7-Code is released under a Modified MIT license, with model weights accessible through HuggingFace. The model can be deployed using vLLM or SGLang, operating solely in thinking mode, without allowing for temperature adjustments—Moonshot AI has fixed this parameter at 1.0, limiting flexibility in output determinism that other models might offer.
A significant change in K2.7-Code compared to K2.6 is its approach to generating low-level code. Whereas K2.6 relied on library wrappers and established frameworks for implementation, K2.7-Code creates implementations directly. Moonshot AI asserts that this leads to more consistent generalization across programming languages such as Rust, Go, and Python, as well as various tasks including frontend development, DevOps, and performance tuning.
Moonshot AI claims that K2.7-Code demonstrates a 21.8% improvement on Kimi Code Bench v2, an 11% boost on Program Bench, and a 31.5% uplift on MLS Bench Lite—all proprietary benchmarks evaluated by Moonshot AI. However, the model has not yet been tested against DeepSWE, an independent coding benchmark that offers a broader 70-point range across models, unlike SWE-Bench Pro’s narrower 30-point spread, making it a more precise indicator for teams configuring their model routing systems.
VB Transform · July 14–15 · Menlo Park · Inference & AI infrastructure
GM saw a 300% rise in merged PRs after restructuring for agents. Here’s their achievement.
The infrastructure segment at Transform features real-time video generation, machine-to-machine reasoning stacks, and the challenges of operating agents at an enterprise scale.
See the full agenda →
A Closer Look at Model Performance
The situation becomes more nuanced when looking at performance outside Moonshot’s own benchmarks.
Researcher Elliot Arledge compared K2.7-Code with K2.6 and Claude Fable 5 using KernelBench-Hard, a public benchmark designed for GPU kernel optimization. His detailed logs are available on kernelbench.com.
“K2.7 is more straightforward but not necessarily better,” Arledge noted on X.
In five out of six tasks, K2.7-Code generated genuine Triton kernels, where K2.6 had utilized library wrappers. However, two of these kernels contained bugs originating from the model itself. The MoE kernel’s performance declined from K2.6’s score of 0.222 to 0.157.
Arledge observed, “For reference, Fable excels in every scenario where it does not fail outright.”
Meanwhile, Sugumaran Balasubramaniyan, a developer who created a model-task-router for the Hermes Agent platform utilizing DeepSWE as a benchmark, publicly challenged Moonshot AI regarding its benchmark selections in light of the K2.7-Code announcement.
He commented, “It’s important to note that every model claims double-digit improvements on its own testing suite.” Balasubramaniyan highlighted that K2.6 scored 24% on DeepSWE, alongside GPT-5.4-mini, and inquired if Moonshot AI would evaluate K2.7-Code against the same standard.
He also mentioned that it required 13 rounds of review to finalize the benchmark data for his router, and expressed he would be inclined to route coding tasks to K2.7-Code if the independent data proves reliable.
Implications for Enterprises
The efficiency of tokens can be directly leveraged by teams using K2.6. They can switch to K2.7-Code through the OpenAI-compatible API, anticipating reduced inference costs for automated workflows without the need for architectural changes. The claimed 30% reduction in thinking tokens is provided by Moonshot, but the smooth integration process allows for testing against existing workloads before making a commitment.
The critical question remains whether these efficiency benefits are applicable to a team’s specific task distribution. Testing K2.7-Code with real workloads prior to modifying gateway settings is a prudent approach for evaluation.
