Thinking Machines Launches Inkling: A Multimodal Transformer Model

Thinking Machines has introduced Inkling, a multimodal transformer model capable of processing text, image, and audio inputs to generate text outputs. This model is significant in expanding the capabilities of AI systems by bridging multiple modalities into a unified framework.

Inkling has the potential to reshape multimodal AI interactions, making it versatile for various applications from content generation to enhanced user interface experiences. Modal has partnered with Thinking Machines to support Inkling as a Managed Endpoint, incorporating token-based pricing that facilitates accessibility from the outset.

Inkling: The dawn of a versatile model

The name Inkling reflects its potential—a seed from which greater innovations can grow. With a staggering 975 billion parameters, it boasts 41 billion active parameters dedicated to executing tasks efficiently. Inkling is designed with a 1 million token context window, allowing it to handle substantial input sequences, which is particularly beneficial for applications requiring extensive context.

This architecture is primarily a mixture-of-experts transformer that is trained to favor breadth over depth, making it an effective base model for specialized adaptations. Inkling's design inherently prioritizes speed, as it employs a structure that ensures low active parameters through sparse mixture-of-experts, resulting in reduced computational demands while handling large datasets.

In practical scenarios, Inkling demonstrates impressive performance metrics. For instance, on agentic workloads run using 8 B200 GPUs, it achieves 250 tokens per second per user at a remarkable 2.5 million tokens per minute throughput. This performance indicates a 67% increase in interactivity when compared to the model’s built-in speculative path operating at matching throughput levels.

Advancements in Local Attention Mechanisms

Inkling’s architecture leverages a unique local attention setup that improves efficiency. It operates under a framework where, for every six attention layers, five utilize sliding window attention focusing on recent tokens, and only one employs full attention across the entire sequence. This balance enables the model to allocate computational resources more effectively, maximizing the speed of processing while accurately responding to user inputs.

Comparatively, this trend towards optimizing local attention is becoming common in top-performing models. For example, recent iterations from models like Qwen 3.5 and 3.6 have adopted similar techniques, such as Gated DeltaNet linear attention layers, to improve computational speed without increasing parameter counts. By focusing more on recent tokens, Inkling efficiently manages longer sequences while still achieving high responsiveness, thus setting a standard for upcoming models.

Inkling is equipped with a multi-token prediction (MTP) head used for speculative decoding. However, traditional MTP implementations face limitations as they draft tokens sequentially, which can slow overall performance. Therefore, Thinking Machines is pushing forward with innovative speculative techniques like DFlash, which allows for the generation of complete token blocks in a single parallel pass, radically expediting the drafting process.

Future Directions: DFlash and Beyond

DFlash incorporates distinctive improvements that ensure it remains relevant as models progress. It achieves this by relying entirely on local attention for each drafting layer without reverting to global attention mechanisms, which can burden the model. This crucial strategic choice allows DFlash to continue delivering results aligned with the rapid advancements in model architecture.

The architecture's shift to causal layers enhances compatibility with optimized attention kernels, such as trtllm, demonstrating a clear path toward enhanced computational efficiency. As models like Inkling continue to evolve, leveraging local attention and efficient decoding processes will be vital for maintaining performance at scale.

Thinking Machines is committed to maximizing DFlash's capabilities, envisioning a future where it can not only keep pace with but also facilitate advancements in newer models within the shared open frontier. The team believes in the ongoing applicability of such methods across various deployments and is prepared to evolve in line with technological advancements in the AI landscape.

Inkling is currently available as an OpenAI-compatible shared endpoint via Modal Auto Endpoints. Its introduction opens the door for further experimentation and development in multimodal AI interactions, encouraging wider use of integrative models.

To explore Inkling’s capabilities and engage with its functionalities, developers are encouraged to adopt the model through [Modal endpoints](https://modal.com/endpoints?model=thinkingmachines%2FInkling-NVFP4&type=managed)—an excellent opportunity to witness firsthand the potentials of modern AI. Feedback from users will contribute to understanding the model's impact and directions for future improvements.