Implementing GraphRAG with Neo4j and LLMs

GraphRAG extends traditional retrieval-augmented generation approaches by incorporating graph database queries to retrieve structured, context-rich knowledge before passing it to large language models for answer generation. This guide uses Neo4j as the graph database and OpenAI’s GPT-4 as the language model for demonstration.

1. Prerequisites and environment setup

Ensure you have a running Neo4j instance (community or enterprise edition) with Neo4j version 5.x or later, as it supports enhanced query capabilities beneficial for knowledge graphs. You will also need Python 3.8 or above and OpenAI API access (API key required). This example uses the neo4j Python driver v5 and openai Python package v0.27.4.

1. Install Neo4j and start the server locally or on cloud-hosted service.
2. Create or import a domain knowledge graph into Neo4j. Sample datasets include academic papers, organizational structures, or product catalogs.
3. Set up a Python virtual environment and install dependencies with `pip install neo4j openai`.
4. Configure Neo4j connection parameters and OpenAI API key securely in your environment variables.

2. Step 1: Connect to Neo4j and run graph queries

Use the neo4j Python driver to execute Cypher queries. The goal in GraphRAG is to retrieve a subgraph relevant to the input query, which will feed into the LLM as additional context.

Example: Given a user query about a research topic, retrieve linked entities such as authors, papers, and institutions.

```python from neo4j import GraphDatabase class Neo4jConnector: def __init__(self, uri, user, password): self.driver = GraphDatabase.driver(uri, auth=(user, password)) def close(self): self.driver.close() def fetch_subgraph(self, topic): query = ''' MATCH (t:Topic {name: $topic})<-[:RELATED_TO]-(p:Paper)-[:AUTHORED_BY]->(a:Author) RETURN p.title AS paper, a.name AS author LIMIT 10 ''' with self.driver.session() as session: results = session.run(query, topic=topic) return [(record["paper"], record["author"]) for record in results] # Usage example neo4j_conn = Neo4jConnector("bolt://localhost:7687", "neo4j", "password") subgraph = neo4j_conn.fetch_subgraph("Graph Neural Networks") print(subgraph) neo4j_conn.close() ```

3. Step 2: Format graph data for LLM input

Transform the results from Neo4j into a concise, human-readable context snippet. Use a template to convert key-value pairs into sentences or structured bullet points.

```python def format_subgraph_data(subgraph): context_lines = [f"Paper: {paper}\nAuthor: {author}" for paper, author in subgraph] return "\n---\n".join(context_lines) context_text = format_subgraph_data(subgraph) print(context_text) ```

4. Step 3: Query the LLM using the graph-augmented context

Pass the user question along with the formatted graph context as prompt to the LLM. OpenAI’s GPT-4 model is suitable given its extensive knowledge and prompt handling.

```python import openai openai.api_key = "${OPENAI_API_KEY}" def ask_llm(question, context): prompt = f"Use the following information to answer the question:\n{context}\nQuestion: {question}\nAnswer:" response = openai.ChatCompletion.create( model="gpt-4", messages=[{"role": "user", "content": prompt}], max_tokens=300, temperature=0 ) return response.choices[0].message.content.strip() question = "Who are the main authors contributing to Graph Neural Networks?" answer = ask_llm(question, context_text) print(answer) ```

5. Step 4: Integrate and iterate for production usage

For production, encapsulate the Neo4j queries and LLM calls into services or microservices. Implement query expansion, caching, and dynamic subgraph depth controls for scalability. Monitor latency and cost—OpenAI’s API pricing for GPT-4 is approximately $0.03 per 1,000 prompt tokens and $0.06 per 1,000 completion tokens (as of June 2024). Neo4j can handle thousands of queries per second on appropriately sized hardware.

Logging query results and LLM responses will help tune graph queries and prompt templates over time. The GraphRAG approach benefits from the structured richness of Neo4j while addressing language understanding with the LLM.

6. Conclusion

GraphRAG improves retrieval augmentation by leveraging graph-structured knowledge bases, here implemented with Neo4j, to supplement LLM prompts with precise, relational data. This pattern enhances factual accuracy and context sensitivity in generated answers, critical for complex enterprise knowledge scenarios.