Artificial intelligence has been around since 1955. In principle, that is, because the capabilities of ChatGPT now cannot compare the first AI program developed nearly seventy years ago.
That AI has been the topic of much conversation over the past year is therefore not surprising. In particular, the emergence and rapid development of generative AI, such as ChatGPT, Midjourney and DALL-E, has captured the imagination.
In this two-part series, we look at the history and future of generative AI. First: how did we get this far? What exactly is generative AI and what technologies is it made up of?
In the next installment, we discuss the next stage in the evolution of generative AI. If the omens do not deceive us, we are on the eve of another major breakthrough in the capabilities of generative AI. Retrieval-augmented generation is going to solve many of the problems we face today.
We start with the basics. What are the building blocks of generative AI?
Machine learning (ML) is at the core of many AI systems. ML models are able to learn from data without being explicitly programmed for specific situations. These models recognize patterns and regularities in data sets and learn from them.
A good example is filtering spam in your e-mail. An ML model can be trained with sample e-mails, which contain spam as well as “real” messages. The system learns to differentiate based on attributes such as keywords, sender and formatting. Once trained, the model can assess new emails and filter spam effectively.
Deep learning is a subset of machine learning that uses neural networks. These consist of multiple layers (hence “deep”) and can thus recognize complex patterns in large data sets. Deep learning models are therefore ideally suited for image recognition and classification. They can analyze images and identify objects such as faces, vehicles and landscapes.
This technology is used, for example, in self-driving cars, where it is crucial that the systems recognize pedestrians, road signs and other vehicles. Deep learning typically requires large amounts of training data and computing power. The many layers in neural networks enable the systems to abstract increasingly complex features and patterns from the raw data. This leads to accurate models for image recognition and classification.
Reinforcement learning (RL) is another form of machine learning, in which a model learns by interacting with its environment. The model receives rewards for correct decisions and sometimes punishments for mistakes, with the goal: to achieve optimal behavior for performing a specific task.
An example of RL is a computer game, where an AI agent learns to play a game without prior knowledge. The agent tries different strategies and learns which actions lead to success, often measured by scores or achievement of certain goals in the game.
This technology has been used to improve GPT-4, for example. Bad outcomes were penalized, good outcomes rewarded.
Generative AI goes one step further. This form of AI not only analyzes and classifies data, but also generates new content. Think artwork, music, or even text that is indistinguishable from human work.
An example is OpenAI’s GPT-4, a language model capable of generating persuasive text. This can range from writing articles to answering questions in natural language. These systems learn from a huge dataset of existing texts to create new, coherent and relevant content and are also called large language models.
Together, these AI technologies make up the spectrum of modern AI. Machine learning distills insights from data. Deep learning goes a step further by discovering even more complex patterns in large data sets. Reinforcement learning introduces the aspect of learning through interaction with an environment. Large language models combine all these techniques: they create new data and content, from texts to images. One of the biggest challenges: LLMs do not currently do source identification. So you don’t know what the model based an outcome on.
That will change with retrieval-augmented generation. This technique is designed to improve the quality of generated answers. Relevant facts are retrieved from an external database or knowledge system. This gives an LLM a fact check, so to speak, to ensure that the latest and most accurate information is used.
Retrieval-agumented generation is therefore seen as the next important step in the evolution of generative AI, because it answers one of the biggest challenges today: you can never be sure if the output of generative AI is reliable and accurate.
You can read more about how the technology works, what benefits you can gain from it and how RAG can be applied in practice in part two of this series:
The Next Stage in the Evolution of Generative AI: Retrieval-Augmented Generation as a Fact Checker