news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Machine Heart Report

Editor: Panda W

The knowledge graph never retires!

LLM is very powerful, but it also has some obvious shortcomings, such as hallucination problems, poor interpretability, failure to grasp the key points of the problem, privacy and security issues, etc. Retrieval-augmented generation (RAG) can greatly improve the generation quality and usefulness of LLM.

Earlier this month, Microsoft released GraphRAG, the most powerful open source RAG knowledge base solution. The project became popular as soon as it was launched, and now the number of stars has reached 10.5k.

• Project address: https://github.com/microsoft/graphrag
• Official documentation: https://microsoft.github.io/graphrag/

Some say it's more powerful than regular RAG:

GraphRAG uses LLM to generate a knowledge graph, which can significantly improve question answering performance when performing document analysis on complex information, especially when dealing with private data.

Comparison results between GraphRAG and traditional RAG

Nowadays, RAG is a technique that uses real-world information to improve LLM output. It is an important component of most LLM-based tools. Generally speaking, RAG uses vector similarity as a search, which is called Baseline RAG. However, Baseline RAG does not perform perfectly in some cases. For example:

• Baseline RAG has difficulty connecting the dots. This occurs when answering a question requires traversing different pieces of information through shared attributes to provide new, integrated insights;
• Baseline RAG performs poorly when asked to fully understand the summarized semantic concepts in a large dataset or even a single large document.

GraphRAG proposed by Microsoft uses LLM to create a knowledge graph based on the input text library. This graph combines community summaries and the output of graph machine learning to enhance prompts when querying. GraphRAG shows significant improvements in answering the above two types of questions, demonstrating performance that surpasses previous methods in processing private datasets.

However, as everyone got to know GraphRAG more deeply, they found that its principles and contents were really difficult to understand.

Recently, Neo4j CTO Philip Rathle published a blog post titled "GraphRAG Manifesto: Adding Knowledge to Generative AI". Rathle used easy-to-understand language to introduce in detail the principles of GraphRAG, the difference from traditional RAG, the advantages of GraphRAG, etc.

“Your next generative AI application will likely use a knowledge graph,” he said.

Neo4j CTO Philip Rathle

Let’s take a look at this article below.

We are slowly coming to realize that to do something truly meaningful with generative AI, you can’t just rely on autoregressive LLMs to make decisions for you.

I know what you’re thinking: “Use RAG.” Or fine-tune, or wait for GPT-5.

Yes. Techniques like vector-based Retrieval-augmented Generation (RAG) and fine-tuning can help you. And they do solve some use cases well enough. But there is a class of use cases that makes all these techniques fail.

For many problems, vector-based RAG (and fine-tuning) approaches essentially increase the probability of the correct answer. But neither technique can provide a degree of certainty about the correct answer. They often lack context and are difficult to relate to what you already know. In addition, these tools do not provide clues to understand the reasons for a particular decision.

Let's go back to 2012, when Google launched its second-generation search engine and published a landmark blog post, "Introducing the Knowledge Graph: things, not strings." They found that if they used the knowledge graph to organize all the things represented by strings on the web pages in addition to performing various string processing, it would be possible to bring a leap forward in search.

A similar pattern is now emerging in the field of generative AI, where many generative AI projects are hitting a wall, with the quality of their results limited by the fact that their solutions deal with strings, not things.

Fast forward to today, and cutting-edge AI engineers and academic researchers have rediscovered what Google once discovered: the secret to breaking this bottleneck is the knowledge graph. In other words, it is to introduce knowledge about things into statistical text-based technology. It works similarly to other RAGs, except that in addition to vector indexing, it also calls the knowledge graph. In other words: GraphRAG! (GraphRAG = Knowledge Graph + RAG)

The goal of this article is to provide a comprehensive and accessible introduction to GraphRAG. Research shows that if you build your data into a knowledge graph and use it through RAG, you can get a variety of powerful advantages. There is a lot of research that proves that GraphRAG can answer most or all questions you ask of LLM better than RAG using only plain vectors.

This advantage alone is enough to greatly promote the adoption of GraphRAG.

But it doesn't stop there; because the data is visible when building the application, it is also simpler to develop.

The third advantage of GraphRAG is that both humans and machines can understand graphs and perform reasoning on them very well. Therefore, building applications with GraphRAG is simpler and easier, and produces better results, while also being easier to explain and audit (which is critical for many industries).

I believe GraphRAG will replace vector-only RAG as the default RAG architecture for most use cases. This post will explain why.

What is a graph?

First we must clarify what a graph is.

Graph is also often translated as "图", but it is also easily confused with concepts such as image and picture. For the sake of distinction, this article only uses the translation of "图谱".

The graph probably looks like this:

Graph Example

Although this picture is often used as an example of a knowledge graph, its origin and author are no longer traceable.

or like this:

Game of Thrones character relationship map, from William Lyon

or like this:

London Underground map. Fun fact: Transport for London recently deployed a graph-based digital twin application to improve incident response capabilities and reduce congestion.

In other words, a graph is not a chart.

We won't dwell on the definition here and will just assume that you already understand what a graph is.

If you understand the above pictures, then you can probably see how the underlying knowledge graph data (stored in a graph database) can be queried and used as part of a RAG workflow. That is, GraphRAG.

Two forms of presenting knowledge: vectors and graphs

The core of a typical RAG is vector search, which is to find and return conceptually similar text from candidate written materials based on an input text block. This automation works well and basic searches are very useful.

But every time you perform a search, you may not have thought about what a vector is or how similarity is calculated. Let's take a look at Apple. It presents different forms from a human perspective, a vector perspective, and a graph perspective:

Apple from human perspective, vector perspective and graph perspective

In human perception, the representation of an apple is complex and multidimensional, and its characteristics cannot be fully described on paper. Here we can poetically imagine that this red photo can represent an apple in terms of perception and concept.

The vector representation of this apple is an array. The magic of vectors is that they each capture the essence of their corresponding text in an encoded form. But in the context of RAG, vectors are only needed when you need to determine how similar one piece of text is to another. For this, it is simple to perform a similarity calculation and check the match. However, if you want to understand the meaning inside the vector, understand what is represented in the text, and gain insight into its relationship to the larger context, then using vector representation is not enough.

In contrast, knowledge graphs represent the world in a declarative form — or, in AI parlance, symbolic. Therefore, both humans and machines can understand knowledge graphs and perform reasoning based on them. This is important, and we’ll get to that later.

In addition, you can query, visualize, annotate, modify and extend the knowledge graph. The knowledge graph is a world model that represents the world in your current work area.

GraphRAG and RAG

The two are not in competition. Both vector queries and graph queries are useful for RAG. As Jerry Liu, founder of LlamaIndex, pointed out: When thinking about GraphRAG, it is helpful to include vectors. This is different from "vector-only RAG" - which is based entirely on similarities between text embeddings.

Basically, GraphRAG is a RAG, but with a knowledge graph in the search path. As you will see below, the core pattern of GraphRAG is very simple. It has the same architecture as RAG using vectors, but with a knowledge graph layer.

GraphRAG Mode

A common pattern in GraphRAG

As you can see, a graph query is triggered in the above figure. It can choose whether to include a vector similarity component. You can choose to store the graph and vector separately in two different databases, or use a graph database that supports vector search, such as Neo4j.

Here is a common pattern for using GraphRAG:

1. Perform a vector search or keyword search to find an initial set of nodes;

2. Traverse the graph and bring back information about relevant nodes;

3. (Optional) Re-rank documents using a graph-based ranking algorithm such as PageRank

Different use cases will have different usage patterns. Like all research directions in AI today, GraphRAG is a rich research area with new discoveries emerging every week.

GraphRAG Lifecycle

Generative AI using GraphRAG follows the same pattern as any other RAG application, starting with a “create graph” step:

GraphRAG Lifecycle

Creating a graph is similar to chunking a document and loading it into a vector database. Tool development has made graph creation quite simple. Here are three good news:

1. Graphs are very iterative - you can start with a "minimum viable graph" and then expand on it.

2. Once you add data to your knowledge graph, you can easily evolve it. You can add more types of data to gain and exploit data network effects. You can also improve the quality of your data to increase the value of your application.

3. The field is evolving rapidly, which means that graph creation will only get easier as the tools become more sophisticated.

Adding the atlas creation step to the previous picture, we get the following workflow:

Adding a graph creation step

Let’s take a look at what benefits GraphRAG can bring.

Why use GraphRAG?

The advantages of GraphRAG over vector-only RAG fall into three main categories:

1. Higher accuracy and more complete answers (uptime/production advantage)

2. Once the knowledge graph is created, it will be easier to build and maintain RAG applications (development time advantage)

3. Better explainability, traceability, and access control (governance advantage)

Let’s take a closer look at these advantages.

1. Higher accuracy and more useful answers

The first (and most immediately visible) advantage of GraphRAG is that its responses are of higher quality. We can see a lot of evidence supporting this observation both in academia and industry.

Take this example from Data.world, a data catalog company. At the end of 2023, they published a study showing that GraphRAG can improve the accuracy of LLM responses by an average of 3 times on 43 business questions. This benchmark study provides evidence that knowledge graphs can significantly improve response accuracy.

The knowledge graph improved the accuracy of LLM responses by 54.2 percentage points, or about 3 times.

Microsoft also provided a series of evidence, including a research blog in February 2024, "GraphRAG: Unlocking LLM discovery on narrative private data", and the related research paper "From Local to Global: A Graph RAG Approach to Query-Focused Summarization", and software: https://github.com/microsoft/graphrag (the GraphRAG mentioned at the beginning of the article).

Among them, they observed that the baseline RAG using vectors has the following two problems:

• Baseline RAGs have difficulty connecting the dots. In order to synthesize different information to gain new insights, it is necessary to traverse different pieces of information through shared attributes. At this time, baseline RAGs have difficulty connecting different pieces of information.
• The baseline RAG performs poorly when asked to fully understand the semantic concepts summarized over large datasets or even a single large document.

Microsoft found that: "By using the knowledge graph generated by LLM, GraphRAG can significantly improve the "search" part of RAG, filling the context window with more relevant content, thereby obtaining better answers and obtaining sources of evidence." They also found that compared with other alternative methods, GraphRAG requires 26% to 97% fewer tokens, so it can not only provide better answers, but also has lower costs and better scalability.

Going deeper into accuracy, we know that having the right answer is important, but having a useful answer is also important. People have found that GraphRAG not only makes answers more accurate, but also makes them richer, more complete, and more useful.

A great example is LinkedIn’s recent paper “Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering”, which describes the impact of GraphRAG on its customer service application. GraphRAG improved the correctness and richness of its customer service answers, making them more useful, and reduced the median time it took its customer service team to resolve each question by 28.6%.

A similar example was presented at Neo4j’s Generative AI workshop. Here are the answers for a set of SEC filings using the “vector + GraphRAG” vs. the “vector only” approach:

Comparison of the "Vector Only" and "Vector + GraphRAG" methods

Note the difference between “describe the characteristics of companies that could be affected by lithium shortages” and “list specific companies that could be affected.” If you’re an investor who wants to rebalance your portfolio based on market changes, or a company that wants to realign its supply chain based on a natural disaster, the information on the right side of the chart above is certainly much more important than the information on the left side. Both answers are accurate here. But the right answer is clearly more useful.

Episode 23 of Jesus Barrasa's Going Meta gives another great example: starting with a vocabulary map using legal documents.

We also see new examples from academia and industry from time to time. For example, Charles Borderie of Lettria gives a comparison example of the "vector only" and "vector + GraphRAG" methods; GraphRAG relies on an LLM-based text-to-graph workflow to organize 10,000 financial articles into a knowledge graph:

Comparison of Retriever-only and Graph Retriever Methods

It can be seen that compared with using ordinary RAG, using GraphRAG not only improves the quality of the answer, but also reduces the number of tokens in its answer by one third.

Another example from Writer. They recently published a RAG benchmark report based on the RobustQA framework, comparing their GraphRAG-based approach with other similar tools. GraphRAG scored 86%, significantly better than other methods (between 33% and 76%), while having similar or better latency performance.

Results of the RAG method accuracy and response time evaluation

GraphRAG is benefiting a wide variety of generative AI applications. Knowledge graphs open the way to making generative AI results more accurate and useful.

2. Improved data understanding and faster iteration

Knowledge graphs are intuitive both conceptually and visually, and exploring them often leads to new insights.

Many knowledge graph users have shared that once they’ve invested effort into their knowledge graph, it helps them build and debug their own generative AI applications in unexpected ways. This is partly because if you can view the data as a graph, you can see that the data underlying these applications presents a vivid picture of the data.

Graphs allow you to trace back answers, find data, and follow the cause-and-effect chain all the way.

Let's look at the lithium shortage example above. If you visualize the vector, you'll get something like the following, but with more rows and columns.

Vector Visualization

If you convert your data into a graph, you can understand it in a way that a vector representation cannot.

Here’s an example from a recent webinar by LlamaIndex showing their ability to use the “MENTIONS” relation to extract vectorized chunks of words (lexicon graph) and the graph of LLM-extracted entities (domain graph) and connect the two:

Extracting vocabulary graph and domain graph

(There are also many examples using tools like Langchain, Haystack, and SpringAI.)

You can see the rich structure of the data in this diagram, and imagine the new development and debugging possibilities it opens up. Each piece of data has its own value, and the structure itself stores and conveys additional meaning that you can use to make your application more intelligent.

This is not just about visualization. It’s about making your data structures convey and store meaning. Here’s a reaction from a developer at a well-known fintech company, who had just introduced the knowledge graph into their RAG workflow a week ago:

Developers’ reaction to GraphRAG

This developer’s reaction fits very well with the assumption of “test-driven development”, which is to verify (not trust) that the answer is correct. Personally, I would be horrified if I were to give 100% autonomy to an AI whose decisions were completely opaque. More specifically, even if you are not an AI doomsayer, you will agree that it would be valuable if you could not map a block of words or documents related to "Apple, Inc." to "Apple Corps" (which are two completely different companies). Since it is ultimately the data that drives generative AI decisions, it is arguably the most important thing to evaluate and ensure the correctness of the data.

3. Governance: Explainability, Security, and More

The greater the impact of generative AI decisions, the more you need to convince the people who are ultimately responsible when decisions go wrong. This often involves auditing every decision. This requires a reliable and repeatable record of good decisions. But that’s not enough. You also need to explain the reasons behind adopting or abandoning a decision.

LLMs by themselves don’t do this very well. Yes, you can refer to the documents used to arrive at that decision. But those documents don’t explain the decision itself — not to mention that LLMs make up references. Knowledge graphs are on another level entirely, making the reasoning logic of generative AIs much clearer and making it easier to explain inputs.

Continuing with the example above: Charles from Lettria loaded entities extracted from 10,000 financial articles into a knowledge graph and executed GraphRAG with an LLM. We see that this does provide better answers. Let's look at the data:

Loading entities extracted from 10,000 financial articles into the knowledge graph

First, think of data as a graph. In addition, we can navigate and query this data, and we can correct and update it at any time. The governance advantage is that it becomes much easier to view and audit the "world model" of this data. Compared with using a vector version of the same data, using a graph makes it more likely that the person in charge will understand the reasons behind the decision.

In terms of ensuring quality, if you can put your data in a knowledge graph, it’s much easier to find errors and surprises and trace them back to their source. You can also get provenance and confidence information in the graph, which can then be used for calculations and interpretations. This is simply not possible with a vector-only version of the same data, and as we discussed earlier, vectorized data is difficult for average (or even non-average) people to understand.

Knowledge graphs can also significantly enhance security and privacy.

Security and privacy are often not a big deal when building a prototype, but they are critical if you are going to build it into a product. In regulated industries like banking or healthcare, any employee’s data access rights depend on their job position.

Whether it is LLM or vector database, there is no good way to limit the scope of data access. Knowledge graph can provide a good solution, regulating the scope of participants' access to the database through permission control, so that they cannot see data that they are not allowed to see. The following is a simple security policy that can implement fine-grained permission control in knowledge graph:

A simple security strategy that can be implemented in knowledge graphs

Creating a knowledge graph

What does it take to build a knowledge graph? The first step is to understand the two types of graphs that are most relevant to generative AI applications.

The domain graph represents the world model related to the current application. Here is a simple example:

Field Map

A lexical graph is a graph of the document structure. The most basic lexical graph consists of nodes consisting of word chunks:

Vocabulary Map

It is often extended to include relationships between chunks, document objects (such as tables), chapters, paragraphs, page numbers, document names or numbers, collections, sources, etc. You can also combine the domain graph and the vocabulary graph together, as shown below:

Combining the Domain and Vocabulary Layers

The creation of vocabulary graphs is simple, mainly simple parsing and chunking. As for domain graphs, there are different creation paths depending on the source of the data (from structured data sources, unstructured data sources, or both). Fortunately, tools for creating knowledge graphs from unstructured data sources are developing rapidly.

For example, the new Neo4j Knowledge Graph Builder can automatically create knowledge graphs using PDF documents, web pages, YouTube videos, and Wikipedia articles. The whole process is very simple, just click a few buttons, and then you can visualize and query the domain and vocabulary graph of the text you input. This tool is powerful and fun, and can greatly reduce the threshold for creating knowledge graphs.

As for structured data (such as structured data stored by your company about customers, products, geographic locations, etc.), it can be directly mapped into a knowledge graph. For example, for the most common structured data stored in a relational database, some standard tools can be used to map the relationships into a graph based on proven and reliable rules.

Using the Knowledge Graph

With the knowledge graph, you can do GraphRAG, and there are many frameworks to choose from, such as LlamaIndex Property Graph Index, Langchain integrated Neo4j and Haystack integrated version. This field is developing very fast, but now the programming method is becoming very simple.

The same is true for graph creation, with tools such as Neo4j Importer (which can import and map tabular data into graphs through a graphical interface) and the aforementioned Neo4j Knowledge Graph Builder. The following figure summarizes the steps for building a knowledge graph.

Automatically build knowledge graphs for generative AI

Using knowledge graphs can also map human language questions into graph database queries. Neo4j has released an open source tool, NeoConverse, which helps query knowledge graphs using natural language: https://neo4j.com/labs/genai-ecosystem/neoconverse/

While using graphs does require a bit of learning when you first start, the good news is that it gets easier as the tools evolve.

Summary: GraphRAG is the inevitable future of RAG

The inherent word-based computation and language skills of LLM coupled with vector-based RAG can lead to very good results. In order to consistently get good results, you have to go beyond the string level and build a world model on top of the word model. Similarly, Google found that in order to master search capabilities, they had to go beyond simple text analysis and draw relationships between the things represented by the strings. We are starting to see the same pattern emerging in the AI ​​world. This pattern is GraphRAG.

The development curve of technology is S-shaped: when one technology reaches its peak, another technology will promote progress and surpass the former. As generative AI develops, the requirements of related applications will also increase - from high-quality answers to explainability to fine-grained control over data access rights and privacy and security, the value of knowledge graphs will become more prominent.

The evolution of generative AI

Your next generative AI application will likely use a knowledge graph.

Reference link: https://neo4j.com/blog/graphrag-manifesto/