The greatest value of large language models (LLMs) to enterprise organizations lies in their ability to correctly answer user prompts. In these use cases, it is especially important that the model responds to prompts with the most accurate and up-to-date information and does not hallucinate. That is why we are proud to announce that we have added Retrieval Augmented Generation (RAG) capabilities to SambaNova Suite, as part of a multi-step model process around pre-training, fine-tuning and then retrieval.
When a user queries an LLM through a prompt, the LLM then goes to the data that it has been trained on to obtain a response. The accuracy of that response will depend on the quality and relevance of the data that the model has to source a reply. If the model has been trained on public data, then it can only respond based upon the data that was available at the time it was trained. If that information is now outdated, the model will give an outdated answer. If the model cannot find a correct answer, then it may hallucinate. This is why it is so important that the model be able to access data that is both accurate and current.
The challenge then becomes that training large monolithic models on big data sets can be extraordinarily costly and time consuming. As a result, if you are using a closed model there is a reasonable chance that it was trained on data that is either outdated or that was not available to respond to a current question.
With the addition of RAG support, SambaNova can help mitigate this as the model can now “retrieve” new information through a process of searching and analyzing external data sets, such as a knowledgebase. This enables the model to use information in that knowledgebase to build a response to user prompts, even though it has not been directly trained on that data. This can greatly increase the accuracy of the response, which will ultimately drive greater value to the organization.
By adding RAG support, SambaNova Suite continues to deliver the performance, accuracy, and flexibility to power the largest models.
For more information on SambaNova RAGs support, read this blog.