Challenge:

Aion Labs builds a layer on top of existing LLMs to improve performance and accuracy. They do this by leveraging LLMs ability to communicate with each other. With the latest reasoning models generate a large number of tokens. Each model's result is passed to the next model, with every step adding latency. Without extreme high speed inference, this will quickly result in delays.

A significant part of the Aion Labs process utilizes large numbers of sequential calls. When using interactive processes, such as with a coding assistant, this can result in long wait times that users will find unacceptable. Fast inference directly translates into the LLM being able to do more in a shorter amount of time, which is essential to Aion Labs.

The need for high speed inference has become more critical with the rise of agentic AI. Aion Labs is seeing a noticeable trend to agentic AI with more and more companies building AI that is either fully autonomous or has a high degree of autonomy. With this autonomy comes a requirement for exceptional accuracy. These are sequential processes and if at any point in the sequence of steps there is an error then the whole process fails. In these scenarios, Aion Labs is using parallel prompting to move from 99.9% accuracy to 99.99% will matter and can unlock new use cases.

Solution:

To achieve their goals, Aion Labs is using a combination of DeepSeek R1 671B, as well the distill variant of DeepSeek, Llama models, and even their own fine tuned models. They are building a complete system composed of many models, that leverages the fastest inference to deliver a solution that is more accurate than any of the individual models.

Aion Labs uses SambaCloud, powered by the SambaNova RDU, the SN40L. SambaCloud delivers the fastest inference on the largest and best open-source models, including the fastest performance available for DeepSeek R1 671B.