Thinking Machines Unveils Inkling: A Compact Open Source AI Model Achieving Impressive Performance at Just One-Fourth the Size of Its Predecessor

In an exciting development for AI enthusiasts, Thinking Machines has unveiled Inkling-Small, a new language model that follows the launch of Inkling just two weeks prior. Led by former OpenAI CTO Mira Murati, this well-capitalized startup has successfully introduced a smaller version of its original model without significant loss in performance—marking a notable achievement in the AI field.

Inkling-Small features 276 billion parameters and operates under an open-source Apache 2.0 license. It nearly matches the performance of its larger counterpart, which has 975 billion parameters, on the Artificial Analysis Intelligence Index. Capable of processing text, images, and audio inputs, Inkling-Small offers an impressive context window of up to one million tokens.

With a reduced activation of 12 billion parameters per token in comparison to Inkling’s 41 billion, Inkling-Small manages to maintain a strong performance across coding, reasoning, and multimodal tasks.

This new model appeals to enterprises primarily because it leverages capabilities effectively while significantly lowering computational demands, inference costs, and deployment challenges. While it remains unsuitable for personal devices like laptops or standard workstations, its operational demands are much more manageable than those of its larger predecessor, which is 3.5 times the size.

In an effort to ease access for developers, Thinking Machines has made the full weights available on Hugging Face and introduced fine-tuning capabilities through its Tinker API.

As part of its launch, the company is offering a limited-time discount of 50%, resulting in the API pricing for the standard 64K-context Inkling-Small model being set at $0.58 per million for input tokens, $1.44 per million for output tokens, and $1.73 per million for training tokens. Cached prefill requests are priced at $0.116 per million tokens, and a 256K-context variation is also available for higher costs.

Remarkable performance relative to size

According to the Artificial Analysis, Inkling-Small achieved a score of 40 on the Intelligence Index, just one point shy of Inkling’s score of 41. This is impressive considering the size difference; Inkling-Small has significantly fewer parameters yet still manages to deliver comparable performance. Remarkably, no other open-weight models of its size or smaller have scored higher on the same index.

In various evaluations, Inkling-Small has even outperformed its larger counterpart. For instance, it secured a score of 80.2% on SWE-bench Verified, surpassing Inkling, which scored 77.6%. Similarly, it reached 64.7% on Terminal Bench 2.1, compared to Inkling’s 63.8%. Furthermore, it achieved better results in other assessments like SciCode, Humanity’s Last Exam, GPQA Diamond, and CritPt.

However, while Inkling-Small excels in several areas, it still lags behind in factual knowledge and certain complex tasks, scoring 15.5% on τ³-Banking versus Inkling’s 23.7%. Its AA Omniscience score also reflects weaker factual coverage, even though it has a slightly lower hallucination rate.

For enterprises, while Inkling-Small is well-suited for tasks like coding assistance, document analysis, and multimodal workflows, organizations handling critical factual tasks will still need strategies for retrieval, verification, and human oversight.

Understanding the model’s architecture

Inkling-Small utilizes a sparse Mixture-of-Experts architecture. The model card from Thinking Machines explains its setup, where a 42-layer decoder directs each token to six of 256 specialized experts, while two shared experts remain active at all times. This design highlights the discrepancy between its total 276 billion parameters and its 12 billion active parameters, as only a fraction is engaged during inference.

Additionally, the model is designed to be natively multimodal, processing images, audio, and text in a coordinated manner. Potential applications include coding assistants, chatbots, and retrieval-augmented systems.

Developers also have the flexibility to adjust the model’s reasoning effort based on the complexity of a task, striking a balance between quality, latency, and resource expenditure.

Despite its size, it’s still not for consumer devices

Notably, despite the name Inkling-Small, this model is not intended for consumer use. The standard BF16 checkpoint necessitates a minimum of 600 GB in GPU memory, with the company recommending configurations of either four NVIDIA B300 GPUs or eight NVIDIA H200 GPUs.

A quantized NVFP4 checkpoint offers a reduced requirement of roughly 180 GB, which allows for single or dual GPU configurations. As such, ordinary laptops, desktops, and even high-powered gaming rigs are typically unable to meet these specifications.

The practical deployment scenarios for Inkling-Small are geared toward enterprise GPU servers, cloud environments, and specialized inference solutions. Thus, its “Small” designation compares it to the original Inkling rather than implying suitability for local models.

The size reduction is significant, allowing organizations to lower hosting costs and simplify capacity planning, which is particularly important for companies aiming for self-hosting options while maintaining control over data, model behavior, and fine-tuning capabilities.

Given its open-source nature, further quantization and adaptations are likely to follow, potentially leading to versions more compatible with consumer-level hardware.

Licensing advantages for enterprises

The choice of Apache 2.0 licensing may prove as vital as the model’s performance metrics. This widely recognized permissive license gives organizations the ability to use, modify, and commercialize the model—making it advantageous for commercial products as long as license requirements are managed.

This flexibility contrasts sharply with various other custom “open” AI licenses, which often involve revenue conditions, branding regulations, or operational constraints for large-scale deployments. As more AI companies release model weights outside of traditional open-source licenses, having the Apache 2.0 license can simplify compliance for legal and procurement teams.

Optimizing model development processes

Thinking Machines has demonstrated agility in its development approach, showing how quickly it has turned its initial large model launch into a systematic engineering practice. Researcher Horace He commented that the release of Inkling-Small felt much more streamlined compared to the first Inkling, indicating an evolving internal process.

Inkling-Small’s development benefited from enhancements to the pre-training data mix and adjustments to machine learning protocols. The team also implemented reinforcement learning for further agentic coding improvement over a two-week period.

Mira Murati echoed this sentiment, outlining that Inkling-Small offers near-equivalent performance to Inkling but at a fraction of the size while being readily available for immediate fine-tuning on Tinker.

Considerations for enterprises and AI developers

The company is distributing complete BF16 and NVFP4 checkpoints alongside deployment support through various toolsets. This versatility grants developers multiple pathways for deployment, whether via API access, fine-tuning on Tinker, relying on third-party providers, or private infrastructure operation.

While Inkling-Small may not be suited for individual practitioners to download and run locally, it presents an appealing option for businesses weighing the pros and cons between a large flagship model and a more streamlined open-weight solution. The blend of similar performance metrics, improved results on specific tasks, cost-efficient token pricing, and a smaller hardware footprint make it an attractive choice.

The broader implications of this launch are significant. Thinking Machines aims to establish that their advancements with Inkling are not isolated events, but rather part of an ongoing commitment to refining their model family and training processes, paving the way for the routine production of open-weight multimodal systems.