Challenge:

Maitai provides fully managed inference as-a-service, specifically for enterprises. They fine-tune customer models for their individual applications then host those models on the fastest hardware available to deliver what they term as “no compromise inference.” Their goal is to deliver ultra-low latency and extremely high throughput with accuracy that improves over time, but were challenged with delivering the performance their customers expect. 

The customers Maitai serves are typically enterprises that have undergone a proof of concept, but are struggling to bring their fine-tuned models to production. Speed, latency, or accuracy are common challenges they face, as well as GPU-based infrastructure not being able to deliver the performance they need. Regulatory compliance requirements and reliability are also top concerns for many of their customers. 

Solution:

Maitai can now quickly and easily bring their customers' model checkpoints to SambaNova and host those models on SambaCloud. Powered by the SN40L RDU, SambaCloud delivers the high throughput and ultra-low latency that Maitai and their customers require.

Now, Maitai is able to better satisfy their customers by running their models on SambaCloud to meet their demanding performance requirements with the required levels of privacy and security.