GraphRAG in Action: From Commercial Contracts to a Dynamic Q&A Agent | by Ed Sandoval | Nov, 2024

However first, for these of us not aware of business legislation, let’s begin with a quick intro to the contract evaluate drawback.

Business contract evaluate is a labor-intensive course of involving paralegals and junior legal professionals meticulously figuring out important info in a contract.

“Contract evaluate is the method of totally studying a contract to know the rights and obligations of a person or firm signing it and assess the related affect”.
Hendrycks, Burns et al, NeurIPS 2021, in CUAD an Expert-Annotated NLP Dataset for Legal Contract Review

The primary stage of contract evaluate includes reviewing tons of of pages of contracts to search out the related clauses or obligations. Contract reviewers should determine whether or not related clauses exist, what they are saying in the event that they do exist, and preserve monitor of the place they’re described.

For instance, They need to decide whether or not the contract is a 3-year contract or a 1-year contract. They need to decide the top date of a contract. They need to decide whether or not a clause is, say, an Anti-assignment or an Exclusivity clause…”
Hendrycks, Burns et al, NeurIPS 2021, in CUAD an Expert-Annotated NLP Dataset for Legal Contract Review

It’s a activity that calls for thoroughness however typically suffers from inefficiencies however it’s appropriate for a Massive Language Mannequin!

As soon as the primary stage is accomplished, senior legislation practitioners can begin to look at contracts for weaknesses and dangers. That is an space the place a Q&A agent powered by an LLM and grounded by info saved in Information Graph is an ideal Copilot for a authorized knowledgeable.

The rest of this weblog will describe every of the steps on this course of. Alongside the way in which, I’ll use code snippets as an instance the primary concepts.

The 4 steps are:

1. Extracting Related Info from Contracts (LLM + Contract)
2. Storing info extracted right into a Information Graph (Neo4j)
3. Growing easy KG Information Retrieval Features (Python)
4. Constructing a Q&A Agent dealing with advanced questions (Semantic Kernel, LLM, Neo4j)

The CUAD (Contract Understanding Atticus Dataset) is a CC BY 4.0 licensed and publicly obtainable dataset of over 13,000 expert-labeled clauses throughout 510 authorized contracts, designed to assist construct AI fashions for contract evaluate. It covers a variety of essential authorized clauses, equivalent to confidentiality, termination, and indemnity, that are important for contract evaluation.

We are going to use three contracts from this dataset to showcase how our strategy to successfully extract and analyze key authorized info, constructing a data graph and leveraging it for exact, advanced query answering.

The three contracts mixed comprise a complete of 95 pages.

It’s comparatively simple to immediate an LLM to extract exact info from contracts and generate a JSON output, representing the related info from the contract.

In business evaluate, a immediate could be drafted to to find every of the important components talked about above — events, dates, clauses — and summarize them neatly in a machine-readable (JSON) file.

Extraction Immediate (simplified)

Reply the next questions utilizing info solely on this contract
[Contract.pdf]

1) What sort of contract is that this?
2) Who’re the events and their roles? The place are they included? Title state and nation (use ISO 3166 Nation identify)
3) What’s the Settlement Date?
4) What’s the Efficient date?

For every of the next sorts of contract clauses, extract two items of knowledge:
a) A Sure/No that signifies when you suppose the clause is discovered on this contract
b) A listing of excerpts that signifies this clause sort exists.

Contract Clause varieties: Aggressive Restriction Exception, Non-Compete Clause, Exclusivity, No-Solicit Of Prospects, No-Solicit Of Workers, Non-Disparagement, Termination For Comfort, Rofr/Rofo/Rofn, Change Of Management, Anti-Project, Uncapped Legal responsibility, Cap On Legal responsibility

Present your remaining reply in a JSON doc.

Please observe that the above part exhibits a simplified model of the extraction immediate. A full model could be seen here. You can see that the the final a part of the immediate specifies the specified format of the JSON doc. That is helpful in making certain a constant JSON schema output.

This activity is comparatively easy in Python. The foremost()perform beneath is designed to course of a set of PDF contract information by extracting related authorized info (extraction_prompt), utilizing OpenAI gpt-4o and saving the leads to JSON format.

def foremost():
pdf_files = [filename for filename in os.listdir('./data/input/') if filename.endswith('.pdf')]
for pdf_filename in pdf_files:
print('Processing ' + pdf_filename + '...')    
# Extract content material from PDF utilizing the assistant
complete_response = process_pdf('./information/enter/' + pdf_filename)
# Log the whole response to debug
save_json_string_to_file(complete_response, './information/debug/complete_response_' + pdf_filename + '.json')

The “process_pdf” perform makes use of “OpenAI gpt-4o” to carry out data extraction from the contract with an “extraction immediate”.

def process_pdf(pdf_filename):
# Create OpenAI message thread
thread = shopper.beta.threads.create()
# Add PDF file to the thread
file = shopper.information.create(file=open(pdf_filename, "rb"), function="assistants")
# Create message with contract as attachment and extraction_prompt
shopper.beta.threads.messages.create(thread_id=thread.id,position="consumer",
attachments=[
Attachment(
file_id=file.id, tools=[AttachmentToolFileSearch(type="file_search")])
],
content material=extraction_prompt,
)
# Run the message thread
run = shopper.beta.threads.runs.create_and_poll(
thread_id=thread.id, assistant_id=pdf_assistant.id, timeout=1000)
# Retrieve messages
messages_cursor = shopper.beta.threads.messages.record(thread_id=thread.id)
messages = [message for message in messages_cursor]
# Return final message in Thread 
return messages[0].content material[0].textual content.worth

For every contract, the message returned by “process_pdf” seems like

{
"settlement": {
"agreement_name": "Advertising and marketing Affiliate Settlement",
"agreement_type": "Advertising and marketing Affiliate Settlement",
"effective_date": "Might 8, 2014",
"expiration_date": "December 31, 2014",
"renewal_term": "1 12 months",
"Notice_period_to_Terminate_Renewal": "30 days",
"events": [
{
"role": "Company",
"name": "Birch First Global Investments Inc.",
"incorporation_country": "United States Virgin Islands",
"incorporation_state": "N/A"
},
{
"role": "Marketing Affiliate",
"name": "Mount Knowledge Holdings Inc.",
"incorporation_country": "United States",
"incorporation_state": "Nevada"
}
],
"governing_law": {
"nation": "United States",
"state": "Nevada",
"most_favored_country": "United States"
},
"clauses": [
{
"clause_type": "Competitive Restriction Exception",
"exists": false,
"excerpts": []
},
{
"clause_type": "Exclusivity",
"exists": true,
"excerpts": [
"Company hereby grants to MA the right to advertise, market and sell to corporate users, government agencies and educational facilities for their own internal purposes only, not for remarketing or redistribution."
]
},
{
"clause_type": "Non-Disparagement",
"exists": true,
"excerpts": [
"MA agrees to conduct business in a manner that reflects favorably at all times on the Technology sold and the good name, goodwill and reputation of Company."
]
},
{
"clause_type": "Termination For Comfort",
"exists": true,
"excerpts": [
"This Agreement may be terminated by either party at the expiration of its term or any renewal term upon thirty (30) days written notice to the other party."
]
},
{
"clause_type": "Anti-Project",
"exists": true,
"excerpts": [
"MA may not assign, sell, lease or otherwise transfer in whole or in part any of the rights granted pursuant to this Agreement without prior written approval of Company."
]
},
{
"clause_type": "Value Restrictions",
"exists": true,
"excerpts": [
"Company reserves the right to change its prices and/or fees, from time to time, in its sole and absolute discretion."
]
},
{
"clause_type": "Minimal Dedication",
"exists": true,
"excerpts": [
"MA commits to purchase a minimum of 100 Units in aggregate within the Territory within the first six months of term of this Agreement."
]
},
{
"clause_type": "IP Possession Project",
"exists": true,
"excerpts": [
"Title to the Technology and all copyrights in Technology shall remain with Company and/or its Affiliates."
]
},
{
"clause_type": "License grant",
"exists": true,
"excerpts": [
"Company hereby grants to MA the right to advertise, market and sell the Technology listed in Schedule A of this Agreement."
]
},
{
"clause_type": "Non-Transferable License",
"exists": true,
"excerpts": [
"MA acknowledges that MA and its Clients receive no title to the Technology contained on the Technology."
]
},
{
"clause_type": "Cap On Legal responsibility",
"exists": true,
"excerpts": [
"In no event shall Company be liable to MA, its Clients, or any third party for any tort or contract damages or indirect, special, general, incidental or consequential damages."
]
},
{
"clause_type": "Guarantee Length",
"exists": true,
"excerpts": [
"Company's sole and exclusive liability for the warranty provided shall be to correct the Technology to operate in substantial accordance with its then current specifications."
]
}
]
}
}

With every contract now as a JSON file, the following step is to create a Information Graph in Neo4J.

At this level is helpful to spend a while designing the info mannequin. It is advisable to contemplate some key questions:

• What do nodes and relationships on this graph signify?
• What are the primary properties for every node and relationship?,
• Ought to there be any properties listed?
• Which properties want vector embeddings to allow semantic similarity search on them?

In our case, an acceptable design (schema) consists of the primary entities: Agreements (contracts), their clauses, the organizations who’re events to the settlement and the relationships amongst them.

A visible illustration of the schema is proven beneath.

Node properties:
Settlement {agreement_type: STRING, contract_id: INTEGER,
effective_date: STRING, expiration_date: STRING,
renewal_term: STRING, identify: STRING}
ContractClause {identify: STRING, sort: STRING}
ClauseType {identify: STRING}
Nation {identify: STRING}
Excerpt {textual content: STRING}
Group {identify: STRING}
Relationship properties:
IS_PARTY_TO {position: STRING}
GOVERNED_BY_LAW {state: STRING}
HAS_CLAUSE {sort: STRING}
INCORPORATED_IN {state: STRING}

Solely the “Excerpts” — the brief textual content items recognized by the LLM in Step 1 — require textual content embeddings. This strategy dramatically reduces the variety of vectors and the dimensions of the vector index wanted to signify every contract, making the method extra environment friendly and scalable.

A simplified model of a python script loading every JSON right into a Information Graph with the above schema seems like

NEO4J_URI=os.getenv('NEO4J_URI', 'bolt://localhost:7687')
NEO4J_USER=os.getenv('NEO4J_USERNAME', 'neo4j')
NEO4J_PASSWORD=os.getenv('NEO4J_PASSWORD')
OPENAI_API_KEY = os.getenv('OPENAI_API_KEY')
JSON_CONTRACT_FOLDER = './information/output/'
driver = GraphDatabase.driver(NEO4J_URI, auth=(NEO4J_USER, NEO4J_PASSWORD))
contract_id = 1
json_contracts = [filename for filename in os.listdir(JSON_CONTRACT_FOLDER) if filename.endswith('.json')]
for json_contract in json_contracts:
with open(JSON_CONTRACT_FOLDER + json_contract,'r') as file:
json_string = file.learn()
json_data = json.hundreds(json_string)
settlement = json_data['agreement']
settlement['contract_id'] = contract_id
driver.execute_query(CREATE_GRAPH_STATEMENT,  information=json_data)
contract_id+=1
create_full_text_indices(driver)
driver.execute_query(CREATE_VECTOR_INDEX_STATEMENT)
print ("Producing Embeddings for Contract Excerpts...")
driver.execute_query(EMBEDDINGS_STATEMENT, token = OPENAI_API_KEY)

Right here the “CREATE_GRAPH_STATEMENT” is the one “advanced” piece. It’s a CYPHER assertion that maps the Contract (JSON) into the nodes and relationships within the Information Graph.

The complete Cypher assertion is beneath

CREATE_GRAPH_STATEMENT = """
WITH $information AS information
WITH information.settlement as a
MERGE (settlement:Settlement {contract_id: a.contract_id})
ON CREATE SET 
settlement.contract_id  = a.contract_id,
settlement.identify = a.agreement_name,
settlement.effective_date = a.effective_date,
settlement.expiration_date = a.expiration_date,
settlement.agreement_type = a.agreement_type,
settlement.renewal_term = a.renewal_term,
settlement.most_favored_country = a.governing_law.most_favored_country
//settlement.Notice_period_to_Terminate_Renewal = a.Notice_period_to_Terminate_Renewal
MERGE (gl_country:Nation {identify: a.governing_law.nation})
MERGE (settlement)-[gbl:GOVERNED_BY_LAW]->(gl_country)
SET gbl.state = a.governing_law.state
FOREACH (celebration IN a.events |
// todo correct world id for the celebration
MERGE (p:Group {identify: celebration.identify})
MERGE (p)-[ipt:IS_PARTY_TO]->(settlement)
SET ipt.position = celebration.position
MERGE (country_of_incorporation:Nation {identify: celebration.incorporation_country})
MERGE (p)-[incorporated:INCORPORATED_IN]->(country_of_incorporation)
SET included.state = celebration.incorporation_state
)
WITH a, settlement, [clause IN a.clauses WHERE clause.exists = true] AS valid_clauses
FOREACH (clause IN valid_clauses |
CREATE (cl:ContractClause {sort: clause.clause_type})
MERGE (settlement)-[clt:HAS_CLAUSE]->(cl)
SET clt.sort = clause.clause_type
// ON CREATE SET c.excerpts = clause.excerpts
FOREACH (excerpt IN clause.excerpts |
MERGE (cl)-[:HAS_EXCERPT]->(e:Excerpt {textual content: excerpt})
)
//hyperlink clauses to a Clause Kind label
MERGE (clType:ClauseType{identify: clause.clause_type})
MERGE (cl)-[:HAS_TYPE]->(clType)
)"""

Right here’s a breakdown of what the assertion does:

Information Binding

WITH $information AS information
WITH information.settlement as a

• $information is the enter information being handed into the question in JSON format. It incorporates details about an settlement (contract).
• The second line assigns information.settlement to the alias a, so the contract particulars could be referenced within the subsequent question.

Upsert the Settlement Node

MERGE (settlement:Settlement {contract_id: a.contract_id})
ON CREATE SET 
settlement.identify = a.agreement_name,
settlement.effective_date = a.effective_date,
settlement.expiration_date = a.expiration_date,
settlement.agreement_type = a.agreement_type,
settlement.renewal_term = a.renewal_term,
settlement.most_favored_country = a.governing_law.most_favored_country

• MERGE makes an attempt to search out an current Settlement node with the required contract_id. If no such node exists, it creates one.
• The ON CREATE SET clause units varied properties on the newly created Settlement node, equivalent to contract_id, agreement_name, effective_date, and different agreement-related fields from the JSON enter.

Create Governing Regulation Relationship

MERGE (gl_country:Nation {identify: a.governing_law.nation})
MERGE (settlement)-[gbl:GOVERNED_BY_LAW]->(gl_country)
SET gbl.state = a.governing_law.state

• This creates or merges a Nation node for the governing legislation nation related to the settlement.
• Then, it creates or merges a relationship GOVERNED_BY_LAW between the Settlement and Nation.
• It additionally units the state property of the GOVERNED_BY_LAW relationship

Create Celebration and Incorporation Relationships

FOREACH (celebration IN a.events |
MERGE (p:Group {identify: celebration.identify})
MERGE (p)-[ipt:IS_PARTY_TO]->(settlement)
SET ipt.position = celebration.position
MERGE (country_of_incorporation:Nation {identify: celebration.incorporation_country})
MERGE (p)-[incorporated:INCORPORATED_IN]->(country_of_incorporation)
SET included.state = celebration.incorporation_state
)

For every celebration within the contract (a.events), it:

• Upserts (Merge) an Group node for the celebration.
• Creates an IS_PARTY_TO relationship between the Group and the Settlement, setting the position of the celebration (e.g., purchaser, vendor).
• Merges a Nation node for the nation through which the group is included.
• Creates an INCORPORATED_IN relationship between the group and the incorporation nation, and units the state the place the group is included

Create Contract Clauses and Excerpts

WITH a, settlement, [clause IN a.clauses WHERE clause.exists = true] AS valid_clauses
FOREACH (clause IN valid_clauses |
CREATE (cl:ContractClause {sort: clause.clause_type})
MERGE (settlement)-[clt:HAS_CLAUSE]->(cl)
SET clt.sort = clause.clause_type
FOREACH (excerpt IN clause.excerpts |
MERGE (cl)-[:HAS_EXCERPT]->(e:Excerpt {textual content: excerpt})
)
MERGE (clType:ClauseType{identify: clause.clause_type})
MERGE (cl)-[:HAS_TYPE]->(clType)
)

• This half first filters the record of clauses (a.clauses) to incorporate solely these the place clause.exists = true (i.e., clauses with excerpts recognized by the LLM in Step 1)
• For every clause:
• It creates a ContractClause node with a identify and sort comparable to the clause sort.
• A HAS_CLAUSE relationship is established between the Settlement and the ContractClause.
• For every excerpt related to the clause, it creates an Excerpt node and hyperlinks it to the ContractClause utilizing a HAS_EXCERPT relationship.
• Lastly, a ClauseType node is created (or merged) for the kind of the clause, and the ContractClause is linked to the ClauseType utilizing a HAS_TYPE relationship.

As soon as the import script runs, a single contract could be visualized in Neo4J as a Information Graph

The three contracts within the data graph required solely a small graph (below 100 nodes and fewer than 200 relationships). Most significantly, solely 40–50 vector embeddings for the Excerpts are wanted. This information graph with a small variety of vectors can now be used to energy a fairly highly effective Q&A agent.

With the contracts now structured in a Information Graph, the following step includes making a small set of graph information retrieval features. These features function the core constructing blocks, permitting us to develop a Q&A agent in step 4.

Let’s outline just a few primary information retrieval features:

1. Retrieve primary particulars a couple of contract (given a contract ID)
2. Discover contracts involving a particular group (given a partial group identify)
3. Discover contracts that DO NOT comprise a selected clause sort
4. Discover contracts comprise a particular sort of clause
5. Discover contracts based mostly on the semantic similarity with the textual content (Excerpt) in a clause (e.g., contracts mentioning the usage of “prohibited objects”)
6. Run a pure language question in opposition to all contracts within the database. For instance, an aggregation question that counts “what number of contracts meet sure circumstances”.

In step 4, we’ll construct a Q&A utilizing the Microsoft Semantic Kernel library. This library simplifies the agent constructing course of. It permits builders to outline the features and instruments that an Agent may have at its disposal to reply a query.

To be able to simplify the mixing between Neo4J and the Semantic Kernel library, let’s outline a ContractPlugin that defines the “signature” of every our information retrieval features. Word the @kernel_function decorator for every of the features and in addition the sort info and outline offered for every perform.

Semantic Kernel makes use of the idea of a “Plugin” class to encapsulate a bunch of features obtainable to an Agent. It is going to use the adorned features, sort info and documentation to tell the LLM perform calling capabilities about features obtainable.

from typing import Record, Elective, Annotated
from AgreementSchema import Settlement, ClauseType
from semantic_kernel.features import kernel_function
from ContractService import  ContractSearchService
class ContractPlugin:
def __init__(self, contract_search_service: ContractSearchService ):
self.contract_search_service = contract_search_service
@kernel_function
async def get_contract(self, contract_id: int) -> Annotated[Agreement, "A contract"]:
"""Will get particulars a couple of contract with the given id."""
return await self.contract_search_service.get_contract(contract_id)
@kernel_function
async def get_contracts(self, organization_name: str) -> Annotated[List[Agreement], "A listing of contracts"]:
"""Will get primary particulars about all contracts the place one of many events has a reputation just like the given group identify."""
return await self.contract_search_service.get_contracts(organization_name)
@kernel_function
async def get_contracts_without_clause(self, clause_type: ClauseType) -> Annotated[List[Agreement], "A listing of contracts"]:
"""Will get primary particulars from contracts with no clause of the given sort."""
return await self.contract_search_service.get_contracts_without_clause(clause_type=clause_type)
@kernel_function
async def get_contracts_with_clause_type(self, clause_type: ClauseType) -> Annotated[List[Agreement], "A listing of contracts"]:
"""Will get primary particulars from contracts with a clause of the given sort."""
return await self.contract_search_service.get_contracts_with_clause_type(clause_type=clause_type)
@kernel_function
async def get_contracts_similar_text(self, clause_text: str) -> Annotated[List[Agreement], "A listing of contracts with comparable textual content in considered one of their clauses"]:
"""Will get primary particulars from contracts having semantically comparable textual content in considered one of their clauses to the to the 'clause_text' offered."""
return await self.contract_search_service.get_contracts_similar_text(clause_text=clause_text)
@kernel_function
async def answer_aggregation_question(self, user_question: str) -> Annotated[str, "An answer to user_question"]:
"""Reply obtained by turning user_question right into a CYPHER question"""
return await self.contract_search_service.answer_aggregation_question(user_question=user_question)

I’d advocate exploring the “ContractService” class that incorporates the implementations of every of the above features. Every perform workout routines a a special information retrieval method.

Let’s stroll by the implementation of a few of these features as they showcase completely different GraphRAG information retrieval methods / patterns

Get Contract (from contract ID) — A Cypher-based retrieval perform

The get_contract(self, contract_id: int), is an asynchronous methodology designed to retrieve particulars a couple of particular contract (Settlement) from a Neo4J database utilizing a Cypher question. The perform returns an Settlement object populated with details about the settlement, clauses, events, and their relationships.

Right here’s the implementation of this perform

async def get_contract(self, contract_id: int) -> Settlement:
GET_CONTRACT_BY_ID_QUERY = """
MATCH (a:Settlement {contract_id: $contract_id})-[:HAS_CLAUSE]->(clause:ContractClause)
WITH a, gather(clause) as clauses
MATCH (nation:Nation)-[i:INCORPORATED_IN]-(p:Group)-[r:IS_PARTY_TO]-(a)
WITH a, clauses, gather(p) as events, gather(nation) as nations, gather(r) as roles, gather(i) as states
RETURN a as settlement, clauses, events, nations, roles, states
"""
agreement_node = {}
information, _, _  = self._driver.execute_query(GET_CONTRACT_BY_ID_QUERY,{'contract_id':contract_id})
if (len(information)==1):
agreement_node =    information[0].get('settlement')
party_list =        information[0].get('events')
role_list =         information[0].get('roles')
country_list =      information[0].get('nations')
state_list =        information[0].get('states')
clause_list =       information[0].get('clauses')
return await self._get_agreement(
agreement_node, format="lengthy",
party_list=party_list, role_list=role_list,
country_list=country_list,state_list=state_list,
clause_list=clause_list
)

A very powerful element is the The Cypher question in GET_CONTRACT_BY_ID_QUERY This question is executed utilizing contract_id equipped as enter parameter. The output is the matching Settlement, its clauses and events concerned (every celebration has a job and nation/state of incorporation)

The info is then handed to an utility perform _get_agreementwhich merely maps the info to an “Settlement”. The settlement is a TypedDict outlined as

class Settlement(TypedDict):  
contract_id: int
agreement_name: str
agreement_type: str
effective_date: str
expiration_date: str
renewal_term: str
notice_period_to_terminate_Renewal: str
events: Record[Party]
clauses: Record[ContractClause]

Get Contracts WITHOUT a Clause sort — One other Cypher retrieval perform

This perform illustrate a robust characteristic of a data graph, which is to check for the absence of a relationship.

The get_contracts_without_clause() perform retrieves all contracts (Agreements) from the Neo4J database that don’t comprise a particular sort of clause. The perform takes a ClauseType as enter and returns a listing of Settlement objects that match the situation.

The sort of information retrieval info can’t be simply carried out with vector search. The complete implementation follows

async def get_contracts_without_clause(self, clause_type: ClauseType) -> Record[Agreement]:
GET_CONTRACT_WITHOUT_CLAUSE_TYPE_QUERY = """
MATCH (a:Settlement)
OPTIONAL MATCH (a)-[:HAS_CLAUSE]->(cc:ContractClause {sort: $clause_type})
WITH a,cc
WHERE cc is NULL
WITH a
MATCH (nation:Nation)-[i:INCORPORATED_IN]-(p:Group)-[r:IS_PARTY_TO]-(a)
RETURN a as settlement, gather(p) as events, gather(r) as roles, gather(nation) as nations, gather(i) as states
"""
#run the Cypher question
information, _ , _ = self._driver.execute_query(GET_CONTRACT_WITHOUT_CLAUSE_TYPE_QUERY,{'clause_type':clause_type.worth})
all_agreements = []
for row in information:
agreement_node =  row['agreement']
party_list =  row['parties']
role_list =  row['roles']
country_list = row['countries']
state_list = row['states']
settlement : Settlement = await self._get_agreement(
format="brief",
agreement_node=agreement_node,
party_list=party_list,
role_list=role_list,
country_list=country_list,
state_list=state_list
)
all_agreements.append(settlement)
return all_agreements

As soon as once more, the format is just like the earlier perform. A Cypher question,GET_CONTRACTS_WITHOUT_CLAUSE_TYPE_QUERY , defines the nodes and relationship patterns to be matched. It performs an non-compulsory match to filters out contracts that do comprise a clause sort, and collects associated information concerning the settlement, such because the concerned events and their particulars.

The perform then constructs and returns a listing of Settlement objects, which encapsulate all of the related info for every matching settlement.

Get Contract with Semantically Related Textual content — A Vector-Search + Graph information retrieval perform

The get_contracts_similar_text() perform is designed to search out agreements (contracts) that comprise clauses with textual content just like a offered clause_text. It makes use of semantic vector search to determine associated Excerpts after which traverses the graph to return details about the corresponding agreements and clauses, the place these excerpts got here from.

This perform leverages a vector index outlined on the “textual content” property of every Excerpt. It makes use of the not too long ago launched Neo4J GraphRAG package to simplify the Cypher code wanted to run semantic search + Graph traversal code.

async def get_contracts_similar_text(self, clause_text: str) -> Record[Agreement]:
#Cypher to traverse from the semantically comparable excerpts again to the settlement
EXCERPT_TO_AGREEMENT_TRAVERSAL_QUERY="""
MATCH (a:Settlement)-[:HAS_CLAUSE]->(cc:ContractClause)-[:HAS_EXCERPT]-(node) 
RETURN a.identify as agreement_name, a.contract_id as contract_id, cc.sort as clause_type, node.textual content as excerpt
"""
#Arrange vector Cypher retriever
retriever = VectorCypherRetriever(
driver= self._driver,  
index_name="excerpt_embedding",
embedder=self._openai_embedder, 
retrieval_query=EXCERPT_TO_AGREEMENT_TRAVERSAL_QUERY,
result_formatter=my_vector_search_excerpt_record_formatter
)
# run vector search question on excerpts and get outcomes containing the related settlement and clause 
retriever_result = retriever.search(query_text=clause_text, top_k=3)
#arrange Record of Agreements (with partial information) to be returned
agreements = []
for merchandise in retriever_result.objects:
//extract info from returned objects and append settlement to outcomes
// full code not proven right here however obtainable on the Github repo
return agreements

Let’s go over the primary parts of this information retrieval perform

• The Neo4j GraphRAG VectorCypherRetriever permits a developer to carry out semantic similarity on a vector index. In our case, for every semantically comparable Excerpt “node” discovered, an extra Cypher expression is used to fetch further nodes within the graph associated to the node.
• The parameters of the VectorCypherRetriever are simple. The index_name is the vector index on which to run semantic similarity. The embedder generates a vector embedding for a bit of textual content. The driver is simply an occasion of a Neo4j Python driver. The retrieval_query specify the extra nodes and relationships related with ever “Excerpt” node recognized by semantic similarity
• The EXCERPT_TO_AGREEMENT_TRAVERSAL_QUERY
  specifies the extra nodes to be retrieved. On this case, for each Excerpt, we’re retrieving its associated Contract Clause and corresponding Settlement

EXCERPT_TO_AGREEMENT_TRAVERSAL_QUERY="""
MATCH (a:Settlement)-[:HAS_CLAUSE]->(cc:ContractClause)-[:HAS_EXCERPT]-(node) 
RETURN a.identify as agreement_name, a.contract_id as contract_id, cc.sort as clause_type, node.textual content as excerpt
"""

Run a Pure Language Question — A Textual content 2Cypher information retrieval perform

The answer_aggregation_question() perform leverages Neo4j GraphRAG bundle “Text2CypherRetriever” to reply a query in pure language. The Text2CypherRetriever makes use of an LLM to show the consumer query right into a Cypher question and runs it in opposition to the Neo4j database.

The perform leverages OpenAI gpt-4o to generate the required Cypher question. Let’s stroll by the primary parts of this information retrieval perform.

 async def answer_aggregation_question(self, user_question) -> str:
reply = ""
NEO4J_SCHEMA = """
omitted for brevity (see beneath for the total worth)
"""
# Initialize the retriever
retriever = Text2CypherRetriever(
driver=self._driver,
llm=self._llm,
neo4j_schema=NEO4J_SCHEMA
)
# Generate a Cypher question utilizing the LLM, ship it to the Neo4j database, and return the outcomes
retriever_result = retriever.search(query_text=user_question)
for merchandise in retriever_result.objects:
content material = str(merchandise.content material)
if content material:
reply += content material + 'nn'
return reply

This perform leverages Neo4j GraphRAG bundle “Text2CypherRetriever”. It makes use of an LLM, on this case OpenAI LLM is used to show a consumer query (pure language) right into a Cypher question that’s executed in opposition to the database. The results of this question is returned.

A key aspect to make sure that the LLM generates a question that makes use of the nodes, relationships and properties outlined within the database is to supply the LLM with a textual content description of the schema.

In our case, we used the next illustration of the info mannequin is ample.

NEO4J_SCHEMA = """
Node properties:
Settlement {agreement_type: STRING, contract_id: INTEGER,effective_date: STRING,renewal_term: STRING, identify: STRING}
ContractClause {identify: STRING, sort: STRING}
ClauseType {identify: STRING}
Nation {identify: STRING}
Excerpt {textual content: STRING}
Group {identify: STRING}
Relationship properties:
IS_PARTY_TO {position: STRING}
GOVERNED_BY_LAW {state: STRING}
HAS_CLAUSE {sort: STRING}
INCORPORATED_IN {state: STRING}
The relationships:
(:Settlement)-[:HAS_CLAUSE]->(:ContractClause)
(:ContractClause)-[:HAS_EXCERPT]->(:Excerpt)
(:ContractClause)-[:HAS_TYPE]->(:ClauseType)
(:Settlement)-[:GOVERNED_BY_LAW]->(:Nation)
(:Group)-[:IS_PARTY_TO]->(:Settlement)
(:Group)-[:INCORPORATED_IN]->(:Nation)
"""

Armed with our Information Graph information retrieval features, we’re able to construct an agent grounded by GraphRAG 🙂

Let’s units up a chatbot agent able to answering consumer queries about contracts utilizing a mix of OpenAI’s gpt-4o mannequin, our information retrieval features and a Neo4j-powered data graph.

We are going to use Microsoft Semantic Kernel, a framework that permits builders to combine LLM perform calling with current APIs and information retrieval features

The framework makes use of an idea known as Plugins to signify particular performance that the kernel can carry out. In our case, all of our information retrieval features outlined within the “ContractPlugin” can be utilized by the LLM to reply the query.

The framework makes use of the idea of Reminiscence to maintain all interactions between consumer and agent, in addition to features executed and information retrieved.

A very simple Terminal-based agent could be carried out with just a few strains of code. The snippet beneath exhibits the primary elements of the agent (imports and setting vars eliminated).

logging.basicConfig(degree=logging.INFO)
# Initialize the kernel
kernel = Kernel()
# Add the Contract Search plugin to the kernel
contract_search_neo4j = ContractSearchService(NEO4J_URI,NEO4J_USER,NEO4J_PASSWORD)
kernel.add_plugin(ContractPlugin(contract_search_service=contract_search_neo4j),plugin_name="contract_search")
# Add the OpenAI chat completion service to the Kernel
kernel.add_service(OpenAIChatCompletion(ai_model_id="gpt-4o",api_key=OPENAI_KEY, service_id=service_id))
# Allow automated perform calling
settings: OpenAIChatPromptExecutionSettings = kernel.get_prompt_execution_settings_from_service_id(service_id=service_id)
settings.function_choice_behavior = FunctionChoiceBehavior.Auto(filters={"included_plugins": ["contract_search"]})
# Create a historical past of the dialog
historical past = ChatHistory()
async def basic_agent() :
userInput = None
whereas True:
# Accumulate consumer enter
userInput = enter("Consumer > ")
# Terminate the loop if the consumer says "exit"
if userInput == "exit":
break
# Add consumer enter to the historical past
historical past.add_user_message(userInput)
# 3. Get the response from the AI with automated perform calling
chat_completion : OpenAIChatCompletion = kernel.get_service(sort=ChatCompletionClientBase)
outcome = (await chat_completion.get_chat_message_contents(
chat_history=historical past,
settings=settings,
kernel=kernel,
arguments=KernelArguments(),
))[0]
# Print the outcomes
print("Assistant > " + str(outcome))
# Add the message from the agent to the chat historical past
historical past.add_message(outcome)
if __name__ == "__main__":
asyncio.run(basic_agent())

1. Discover me contracts with Value Restrictions however no insurance coverage clauses

Assistant > I discovered the next contract that incorporates a Value Restrictions clause however doesn't comprise an Insurance coverage clause:
- **Title:** Birch First World Investments Inc. and Mount Information Holdings Inc. MARKETING AFFILIATE AGREEMENT
- **Settlement Kind:** MARKETING AFFILIATE AGREEMENT
- **Events:**
- **MOUNT KNOWLEDGE HOLDINGS INC.** (Advertising and marketing Affiliate), included in Nevada, USA
- **BIRCH FIRST GLOBAL INVESTMENTS INC.** (Firm), included within the Virgin Islands (VIR)
Would you wish to know extra particulars about this contract?

Right here, The Semantic Agent planner has reviewed the obtainable features within the Kernel and decided that answering this query requires invoking two features that, on their very own, gained’t present the whole reply.

First, it executes the “Get Contracts with [Price Restriction] Clauses” perform, adopted by the “Get Contracts WITHOUT [Insurance] Clauses” perform. The planner then combines the outcomes from each features to determine the intersection, in the end arriving on the appropriate reply.

If you happen to take a look at the debugging info, you possibly can affirm the sequence of calls carried out by the LLM, together with info on the variety of tokens consumed by the decision. Discover the calls with the two information retrieval features.

2. Sure, please. Present extra particulars

Assistant > Listed below are the detailed details about the contract:
### Contract Particulars:
- **Title:** Birch First World Investments Inc. and Mount Information Holdings Inc. MARKETING AFFILIATE AGREEMENT
- **Settlement Kind:** MARKETING AFFILIATE AGREEMENT
- **Expiration Date:** December 31, 2014
- **Renewal Time period:** One (1) 12 months
### Events Concerned:
1. **MOUNT KNOWLEDGE HOLDINGS INC.** 
- **Function:** Advertising and marketing Affiliate
- **Incorporation Nation:** USA
- **Incorporation State:** Nevada
2. **BIRCH FIRST GLOBAL INVESTMENTS INC.**
- **Function:** Firm
- **Incorporation Nation:** Virgin Islands (VIR)
### Clauses Included:
- No-Solicit Of Workers
- Third Celebration Beneficiary
- Value Restrictions
- Affiliate License-Licensee
- Affiliate License-Licensor
- Cap On Legal responsibility
- Audit Rights
- Guarantee Length
- Submit-Termination Companies
- Non-Transferable License
- License grant
- IP Possession Project
- Minimal Dedication
- Anti-Project
- Non-Disparagement
### Clauses Not Included:
- Insurance coverage
Is there something particular you want to know or any explicit clause particulars you have an interest in?

On this case, behind the scenes, the LLM used the “Discover Contract by ID”. You’ll have seen that the output proven to the consumer didn’t embody the id of the contract. Nonetheless, the Semantic Kernel saved all info returned from the features from earlier calls within the dialog reminiscence!

This highly effective mixture of getting the agent determine which perform(s) to name, the usage of reminiscence and the automated marshalling of the consumer queries into perform calls with the suitable parameters clarify the “Magic” behind an agent constructed with Semantic Kernel!

3. Does any contract point out the power to make worth modifications sometimes

Assistant > Sure, there's a contract that mentions 
the power to make worth modifications sometimes. 
Right here is the related excerpt:
1. **Birch First World Investments Inc. and Mount Information Holdings Inc. MARKETING AFFILIATE AGREEMENT**
- **Contract ID:** 1
- **Clause Kind:** Value Restrictions
- **Excerpt:**
```
Firm reserves the suitable to vary its course of and/or charges, sometimes, in its sole and absolute discretion. Within the occasion of a worth and/or charge change for Know-how, Firm shall notify MA in writing ninety (90) days previous to the efficient date of any such change. All different costs and/or charges could also be adjusted with out prior discover to MA.
```
Would you want extra details about this contract or every other particulars?

To be able to reply this query, Semantic Kernel is executing the get_contract_similar_text information retrieval perform.

Consumer > any contract mentions the power to make worth modifications sometimes
INFO:semantic_kernel.connectors.ai.open_ai.providers.open_ai_chat_completion_base:processing 1 instrument calls in parallel.
INFO:semantic_kernel.kernel:Calling contract_search-get_contracts_similar_text
perform with args: {"clause_text":"worth modifications sometimes"}
Assistant > Sure, there's a contract that mentions the power to make worth modifications sometimes. Right here is the related excerpt:
1. **Birch First World Investments Inc. and Mount Information Holdings Inc. MARKETING AFFILIATE AGREEMENT**
- **Contract ID:** 1
- **Clause Kind:** Value Restrictions
- **Excerpt:**
```
Firm reserves the suitable to vary its course of and/or charges, sometimes, in its sole and absolute discretion. Within the occasion of a worth and/or charge change for Know-how, Firm shall notify MA in writing ninety (90) days previous to the efficient date of any such change. All different costs and/or charges could also be adjusted with out prior discover to MA.
```

As defined earlier, this perform makes use of vector search and a Cypher question to retrieve not simply semantically comparable excerpts but additionally the associated contract and clauses that features them.

Lastly, let’s ask what number of contracts are there on the database

4. What number of contracts are there within the database?

This questions requires the LLM to make use of the Text2Cypher information retriever

Assistant > There are a complete of three contracts within the database.

The debug info confirms that Semantic Kernel ran the Text2Cypher information retrieval perform with “user_question” = “What number of contracts are there within the database’?

Consumer >  what number of contracts are there on the database?
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
INFO:semantic_kernel.connectors.ai.open_ai.providers.open_ai_chat_completion_base:processing 1 instrument calls in parallel.
INFO:semantic_kernel.kernel:Calling contract_search-answer_aggregation_question perform 
with args: {"user_question":"What number of contracts are there within the database?"}
INFO:semantic_kernel.features.kernel_function:Perform accomplished. Length: 0.588805s
INFO:semantic_kernel.connectors.ai.open_ai.providers.open_ai_handler:OpenAI utilization: CompletionUsage(completion_tokens=13, prompt_tokens=3328, total_tokens=3341, completion_tokens_details={'reasoning_tokens': 0})
Assistant > There are a complete of three contracts within the database.

The github repo incorporates a Streamlit app that gives a extra elegant Agent UI. You’re inspired to work together with the agent and make modifications to the ContractPlugin so your agent’s means to deal with extra questions!

On this weblog, we explored a Graph Retrieval Augmented Technology (GraphRAG) strategy to rework labor-intensive duties of economic contract evaluate right into a extra environment friendly, AI-driven course of.

By specializing in focused info extraction utilizing LLMs and prompts, constructing a structured data graph with Neo4j, implementing easy information retrieval features, and in the end creating a Q&A agent, we created an clever resolution that handles advanced questions successfully.

This strategy minimizes inefficiencies present in conventional vector search based mostly RAG, focusing as a substitute on extracting solely related info, lowering the necessity for pointless vector embeddings, and simplifying the general course of. We hope this journey from contract ingestion to an interactive Q&A agent evokes you to leverage GraphRAG in your personal tasks for improved effectivity and smarter AI-driven decision-making.

Begin constructing your personal business contract evaluate agent immediately and expertise the ability of GraphRAG firsthand!

For these desirous to take a deeper dive, please try the sources linked beneath:

Except in any other case famous, all pictures are by the creator