Generative AI is seen as the gamechanger of artificial intelligence. Rightly so, as you could read in Part 1 of this series, Generative AI so far: how we got this far.
We are now on the eve of another breakthrough in the possibilities of generative AI. Retrieval-augmented generation is going to solve one of the biggest challenges of generative AI.
Platforms like ChatGPT, Midjourney and DALL-E create new data and content, from texts to images and even entire websites, all based on big data. And they deliver handsome work. However, they have two problems: they don’t do source identification. So you don’t know what the model based an outcome on. And they don’t use actual knowledge from information sources within your organization. So as a user, you can’t blindly assume that the output of generative AI is correct and accurate either.
Enter retrieval-augmented generation.
Retrieval-augmented generation (RAG) combines the generative power of Large Language Models (LLMs) such as ChatGPT, with carefully chosen and known data sources. You can think of RAG as a fact check. You first extract relevant and current information from known sources, with specific knowledge needed within your organization. Then you use this information to generate a response. So it combines the “retrieval” of relevant information with the “generation” of answers.
You can give RAG access to various relevant information sources within your organization: manuals, protocols, contracts, practices, best practices, policies, guidelines, quotations, customer information and so on. The system can extract Information from various file formats – such as PDF, Powerpoint and Word documents.
So RAG increases the usability of LLMs because they can dynamically use external, current information in their answers and analyses. This allows you to use knowledge from those sources to answer questions, just as you already do with ChatGPT to access online resources. Another big advantage of RAG is that it also mentions the source in the answer.
An example. RAG is ideally suited to develop a chatbot that can answer frequently asked questions from customers. RAG can use mental models learned in an LLM, allowing it to understand questions and formulate answers that are clear to the person who asked the question.
For a hospital, we once read a list of frequently asked questions into a RAG. One of these questions was about the system requirements of a mobile app provided by the hospital. The answer stated that the phone should support at least iOS 14.
To properly answer this question, you need to understand the context. Not all questioners use words like “system requirements” or “iOS. An LLM understands this, because it has already learned the relationship between the version of a phone, the supported iOS versions and can reason with that, without having to encounter these specific search terms
And so an AI assistant with RAG of knowledge about the system requirement and the user’s query can respond with:
‘Unfortunately, your iPhone 6 cannot use our mobile app. Your phone does not support the required iOS version 14. At a minimum, an iPhone 6S is required to use the mobile app.’
There are also challenges you will face. For general LLMs such as ChatGPT and Bard, the knowledge they are trained with can become outdated. The same is true for information resources within your own organization become obsolete. It is therefore important to put processes and protocols in place that ensure the quality of these sources if you base an AI assistant on RAG. Agreements will need to be made when new information becomes available to the RAG and what update and delete strategy should be applied. You typically do this on a per-resource basis that you connect to the RAG, and so that requires time and attention.
Other challenges you face in practice is that it is complicated to provide the right information from documents to the system. It is possible to submit documents in their entirety to, which makes implementation easy, but that produces poor quality results. It is important to strike a balance between specific knowledge in small parts of a document and the overall big picture of the document. It is also important to pay close attention to automated testing and evaluation of RAG systems.
So RAG can combine the best of both worlds: the unprecedented possibilities of generative AI, and specific knowledge within your own organization. Moreover, by citing sources, you can be sure that information is correct and up-to-date. But that requires maintenance, testing and evaluation.