Learn RAG architecture from Andrew Ng and Zain Hasan

Transcript

Now let's look at the overall architecture of a RAG system and how these components work together. Let's start by looking at how you normally use a large language model. You type a prompt and send it to the LLM. The LLM processes the prompt and generates a response. The user experience for RAG system is actually identical. You submit a prompt and get back a response inside the system. However, there's a few more steps. When the RAG system receives your prompt, it first routes it to the retriever. The retriever has access to the knowledge base, which practically speaking, is just a database of useful documents. The retriever queries the database, returning documents that it determines are most relevant to the prompt. Next, the system creates an augmented prompt, incorporating information from the relevant documents into the original prompt. For example, an augmented prompt might be answer the following question. Why are hotels in Vancouver so expensive this coming weekend? Here are 5 relevant articles that may help you. Spawned. And then insert text from the articles. At this point, the system operates like any other LLM. The augmented prompt is sent to the LLM, and it generates a response. The LLM is able to respond based on both the knowledge it learned from its training data as well as any additional context provided by the retrieve documents. The user experience is the same as always, perhaps with a little bit of added latency. You type a prompt and you get back a response. Thanks to that side route through the retriever, however, there's a higher likelihood the response is. Accurate, up-to-date, and context aware.