Introduction
Since the ChatGPT boom started, Large Language Models(LLMs) have been utilized for personal use due to their deep language understanding and ability to automate human-level tasks. LLMs have proven excellent for handling everyday tasks, such as looking up recipes, preparing for job interviews, automating text generation, and so much more.
LLMs are a powerful tool that has also proven beneficial to many businesses. Knowron offers a solution that lets users benefit from mighty transformer architecture even more than everyday ChatGPT users by combining an enormous amount of language LLM’s capabilities with internal, company-specific data. One can imagine that it is not that different from Iron Man asking J.A.R.V.I.S. about his costume specifications in the middle of a fight. J.A.R.V.I.S. has access to information about costume specifications and his enemies’s weaknesses. He can pull up relevant information in seconds and deliver it in an easy-to-understand, human-like way.
In this article, we will first learn the concept of Knowledge Graphs, understand why combining them with Large Language Models can be beneficial, and then look into the why and hows of Microsoft’s GraphRAG. If you are curious to know more about the classical RAG approach, we would encourage you to check out the following article from our Blogpost
Knowledge graphs are a structured way to store information.
Knowledge Graphs (KGs) are graphs where nodes represent Entities, i.e., objects like people, companies, and countries. Edges represent relationships between those objects, like a wheel (entity one) - is a part of (relationship) → car (entity 2). For example, Knowron (entity 1) - is located(relationship) → in Munich (entity 2). KGs store information and capture logical connections between concepts. They are inherently structured and modular.
Knowledge Graphs are not a new concept. The term "Knowledge Graph" was coined over forty years ago, and KGs have been widely used in NLP-related tasks for a long time. Google utilizes KG in its search engine. Using Knowledge Graphs as a factual reference for LLMs is a current trend in the industry. LLMs supported by KG outperform other RAG approaches in high-level summarization tasks, clarity, and specificity.
GraphRAG
Knowledge Graph-supported RAG is a fascinating field. The structure and logical relationships provided by KG enable LLMs to understand target topics on a higher level. Knowledge Graph-supported RAG can help summarize multiple documents and connect facts and insights from all of them. A real-life example would be that you have pages of notes from a university course and a couple of days time before an exam. You want your AI assistant to dot the most important topics and concepts to understand the gist of the subject to study efficiently without going too much into the details. Knowledge Graph's ability to draw generalizations from data supported by LLMs language capabilities would provide reliable results fast without losing a high-level overview of the topic.
How does GraphRAG work?
GraphRAG is a Knowledge Graph based RAG solution that Microsoft developed. Let's break down how it works.
The GraphRAG consists of two primary stages:
- Indexing Stage
- Query Stage
Indexing
Indexing is a process of creating a knowledge graph from an unstructured text.
Text is split into chunks. The length of the chunks is adjustable. We split the text into chunks so that the text will be short enough for LLM not to miss anything important. If the text fed to LLM gets too big, LLM starts forgetting or missing things. That is the reason why shorter text chunks lead to better results.
Entity and Relationship extraction: LLM is prompted to extract entities(i.e., objects like countries, names, and machine parts) and relationships between them from text chunks. Small graphs are built from resulting entities and relationships; similar graphs are merged. Intuitively, we find the lowest level of entities and relationships between them. Claims extraction: we extract positive factual statements and some related metadata.
Graph augmentation. Leiden clustering is used to find communities. Communities here refer to groups of entities that are similar to each other. That helps us to structure our data and to organize our data semantically. The graph is embedded using the Node2Vec algorithm.
Community summarization. In this step, LLM is utilized to create a text description of every community. The community description will shortly explain what kind of entities are included and what is the gist of this node group. Communities are hierarchical, community report for a top node would describe the entire graph
Document processing. In this step, a document table is created to keep track of the documents from which text chunks originate.
Entity extraction and community report generation are all powered by LLMs, hence prompts are used. To achieve better results, a domain-specific tailored knowledge graph is needed. To construct one, it is advised to adjust the prompts and specify what kind of entities and relationships should be extracted. Microsoft offers an automated prompt-tuning solution. In our opinion, double checking what kind of prompts were generated and tailoring them for your specific use case is advised.
Query:
Microsoft GraphRAG offers 2 types of search: global and local.
The local search method generates answers by pulling up relevant information from the Knowledge Graph and raw text. This method is best used for tasks that require understanding specific entities mentioned in the documents (e.g. What is the recommended temperature of a hotbed?).
Local search finds entities most similar to the user's input. It uses those nodes as an entry point into the graph. Local search pulls up entities connected to entry point nodes and utilizes related raw text chunks.
Global Search
The global search method goes through all community reports to find relevant information. Since community reports generalize their communities, LLM can generalize as well. That is how a high-level understanding of the whole dataset is made possible.
The global search approach splits community reports into text chunks, goes through every chunk, and tries to answer users' queries based on it. Every produced answer is rated, and the best intermediate answers are used to produce the final answer.
Conclusion
GraphRAG is a fascinating new technology that opens up a lot of possibilities. A high-level understanding of a domain allows models to solve an even wider pallet of business use cases, enabling an even wider automation of complex tasks. We are thrilled to explore what use cases.
