基于 RAG 的方法¶

用于大型文档提取的检索增强生成方法。

LightRAG¶

`hyperextract.methods.rag.Light_RAG` ¶

Bases: AutoGraph[NodeSchema, EdgeSchema]

Light-RAG: Standard Graph-based Retrieval-Augmented Generation

Extracts entity-relationship graphs (nodes and binary edges) from text documents. Optimized for standard Knowledge Graph construction and traversal.

Features: - Two-stage extraction: Entities first, then binary relationships. - Custom Key Extractors: Precise deduplication using name-based node keys and (Source, Target) keys for edges. - Structured Knowledge Representation: Pydantic-based schemas. - Specialized Merging: Custom LLM rules for merging duplicate entities and relationships.

Functions¶

`init(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False)` ¶

Initialize Light_RAG engine.

Parameters:

Name	Type	Description	Default
`llm_client`	`BaseChatModel`	LLM client for text generation.	required
`embedder`	`Embeddings`	Embedding model for text embeddings.	required
`chunk_size`	`int`	Size of text chunks for indexing.	`2048`
`chunk_overlap`	`int`	Overlap between text chunks for indexing.	`256`
`max_workers`	`int`	Maximum number of workers for indexing.	`10`
`verbose`	`bool`	Display detailed execution logs and progress information.	`False`

GraphRAG¶

`hyperextract.methods.rag.Graph_RAG` ¶

Bases: AutoGraph[NodeSchema, EdgeSchema]

Graph-RAG: Graph-based Retrieval-Augmented Generation

Extracts entity-relationships (binary edges) and supports advanced GraphRAG features: - Community Detection (Leiden/Modularity) - Community Reports (Summarization) - Global Search (Map-Reduce over summaries)

Implements the architecture of Microsoft GraphRAG / Nano-GraphRAG.

Attributes¶

`community_reports: Dict[str, CommunityReport] = {}` `instance-attribute` ¶

`_community_graph: Optional[Any] = None` `instance-attribute` ¶

`_community_hierarchy: Dict[int, Dict[str, List[str]]] = {}` `instance-attribute` ¶

`_node_to_community: Dict[str, Dict[str, Any]] = {}` `instance-attribute` ¶

Functions¶

`init(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False)` ¶

Initialize Graph_RAG engine.

Parameters:

Name	Type	Description	Default
`llm_client`	`BaseChatModel`	LLM client for text generation.	required
`embedder`	`Embeddings`	Embedding model for text embeddings.	required
`chunk_size`	`int`	Size of text chunks for indexing.	`2048`
`chunk_overlap`	`int`	Overlap between text chunks for indexing.	`256`
`max_workers`	`int`	Maximum number of workers for indexing.	`10`
`verbose`	`bool`	Display detailed execution logs and progress information.	`False`

`dump(folder_path: str | Path) -> None` ¶

Saves graph state: internal data, community reports, and GraphML for visualization.

`load(folder_path: str | Path) -> None` ¶

Loads graph state from directory.

`_ensure_community_graph()` ¶

Lazily build _community_graph from nodes and edges when needed.

`build_communities(level: int = 0)` ¶

Detects communities in the graph and generates reports for them. Uses Leiden algorithm (via graspologic).

Parameters:

Name	Type	Description	Default
`level`	`int`	The hierarchical level for Leiden algorithm (default 0).	`0`

`search(query: str, top_k_nodes: int = 3, top_k_edges: int = 3, top_k: int | None = None, use_community: bool = False) -> Tuple[List, List, Optional[Dict]]` ¶

Unified graph search interface with optional community enhancement.

Parameters:

Name	Type	Description	Default
`query`	`str`	Search query string.	required
`top_k_nodes`	`int`	Number of node results to return (default: 3).	`3`
`top_k_edges`	`int`	Number of edge results to return (default: 3).	`3`
`top_k`	`int \| None`	If provided, sets both top_k_nodes and top_k_edges to this value.	`None`
`use_community`	`bool`	If True, enables community-aware search. Requires networkx and graspologic. Default: False.	`False`

Returns:

Type	Description
`List`	Tuple of (nodes, edges, community_context).
`List`	community_context is None when use_community=False.

`_global_search_impl(query: str, top_k_nodes: int = 3, top_k_edges: int = 3) -> Tuple[List, List, Dict]` ¶

Internal implementation for community-enhanced search.

`_get_community_context_for_query(query: str) -> Dict` ¶

Get community context related to the query.

HyperRAG¶

`hyperextract.methods.rag.Hyper_RAG` ¶

Bases: AutoHypergraph[NodeSchema, EdgeSchema]

Hyper-RAG: Hypergraph-based Retrieval-Augmented Generation

Extracts multi-entity relationships (hyperedges) from text documents.

Features: - Two-stage extraction: Entities first, then low-order (binary) and high-order (n-ary) relationships. - Custom Key Extractors: Precise deduplication using name-based node keys and sorted participant tuples for edges. - Hyperedge Support: Handles complex n-ary relationships connecting multiple entities simultaneously. - Structured Knowledge Representation: Pydantic-based Node and Edge schemas with comprehensive attributes. - Advanced Indexing: Optimized field-level indexing for efficient semantic search and retrieval.

Functions¶

`init(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False)` ¶

Initialize Hyper_RAG engine.

Parameters:

Name	Type	Description	Default
`llm_client`	`BaseChatModel`	LLM client for text generation.	required
`embedder`	`Embeddings`	Embedding model for text embeddings.	required
`chunk_size`	`int`	Size of text chunks for indexing.	`2048`
`chunk_overlap`	`int`	Overlap between text chunks for indexing.	`256`
`max_workers`	`int`	Maximum number of workers for indexing.	`10`
`verbose`	`bool`	Display detailed execution logs and progress information.	`False`

HyperGraphRAG¶

`hyperextract.methods.rag.HyperGraph_RAG` ¶

Bases: AutoHypergraph[NodeSchema, EdgeSchema]

HyperGraphRAG extractor using semantic knowledge segments as hyperedges.

This class implements the HyperGraphRAG algorithm which models knowledge as a hypergraph where each hyperedge represents a complete "knowledge segment" (atomic semantic unit) that connects multiple related entities. Unlike traditional knowledge graphs that represent pairwise relationships, hypergraphs can naturally express n-ary relationships and maintain the original context and semantic integrity of information.

The extraction process simultaneously identifies both knowledge segments (hyperedges) and the entities involved within each segment, followed by deduplication and merging across multiple text chunks.

Schema Components:

NodeSchema: Represents an entity (node) extracted from the source text within a specific context.
- name (str): The entity name, maintaining original language and capitalization.
- type (str): Entity type such as 'person', 'organization', 'event', 'location', 'concept', etc.
- description (str): Comprehensive description of the entity's attributes and activities within the knowledge segment.
- key_score (int): Importance score (0-100) indicating the entity's significance in context.
EdgeSchema: Represents a knowledge segment (hyper-edge) that connects multiple entities through a specific context or event.
- knowledge_segment (str): The actual text span (sentence or phrase) from source material capturing the relationship, event, or context connecting entities.
- completeness_score (int): Quality score (0-10) indicating how complete and meaningful the segment is as a standalone knowledge unit.
- related_entities (List[str]): Names of all entities involved in or referenced by this knowledge segment.

Functions¶

`init(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False)` ¶

Initialize HyperGraph_RAG extraction engine.

Configures the hypergraph extraction pipeline with LLM-based knowledge segment and entity identification, along with semantic embeddings for vector-based retrieval.

Parameters:

Name	Type	Description	Default
`llm_client`	`BaseChatModel`	Language model client for structured extraction.	required
`embedder`	`Embeddings`	Embedding model for semantic vector representation.	required
`chunk_size`	`int`	Text chunk size for processing (default: 2048 tokens).	`2048`
`chunk_overlap`	`int`	Overlap between consecutive chunks to preserve context (default: 256 tokens).	`256`
`max_workers`	`int`	Maximum parallel workers for batch extraction (default: 10).	`10`
`verbose`	`bool`	Whether to display detailed extraction logs (default: True).	`False`

`_extract_data(text: str) -> AutoHypergraphSchema[NodeSchema, EdgeSchema]` ¶

Extract hypergraph knowledge from text via knowledge segment identification.

Performs simultaneous extraction of knowledge segments (hyperedges) and associated entities from the input text. The extraction process: 1. Segments the text into knowledge-bearing units (sentences/clauses) 2. Identifies all entities referenced within each knowledge segment 3. Merges and deduplicates extracted components across multiple chunks 4. Prunes dangling edges (hyperedges without valid entity references)

Parameters:

Name	Type	Description	Default
`text`	`str`	Source text for hypergraph extraction.	required

Returns:

Type	Description
`AutoHypergraphSchema[NodeSchema, EdgeSchema]`	AutoHypergraphSchema[NodeSchema, EdgeSchema]: Extracted hypergraph containing deduplicated and merged entities (nodes) and knowledge segments (hyperedges).

CogRAG¶

`hyperextract.methods.rag.Cog_RAG` ¶

Cognitive-Inspired Dual-Hypergraph RAG System.

Combines two Hypergraph Layers: 1. Theme Layer (Cog_RAG_ThemeLayer): Captures macro narratives and themes. 2. Detail Layer (Cog_RAG_DetailLayer): Captures micro entity relationships.

Attributes¶

`metadata = {'created_at': datetime.now(), 'updated_at': datetime.now()}` `instance-attribute` ¶

`theme_layer = Cog_RAG_ThemeLayer(llm_client=llm_client, embedder=embedder, chunk_size=chunk_size, chunk_overlap=chunk_overlap, max_workers=max_workers, verbose=verbose)` `instance-attribute` ¶

`detail_layer = Cog_RAG_DetailLayer(llm_client=llm_client, embedder=embedder, chunk_size=chunk_size, chunk_overlap=chunk_overlap, max_workers=max_workers, verbose=verbose)` `instance-attribute` ¶

`llm = llm_client` `instance-attribute` ¶

`verbose = verbose` `instance-attribute` ¶

`nodes` `property` ¶

Return combined unique nodes from both layers (simple aggregation).

`edges` `property` ¶

Return all edges (Themes + Relations).

Functions¶

`init(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False)` ¶

`feed_text(text: str)` ¶

Feed text to both hypergraph layers.

`build_index()` ¶

Build indices for both layers.

`search(query: str, top_k_themes: int = 3, top_k_entities: int = 3)` ¶

Dual-Layer Search. Strategy: 1. Macro: Search for Themes in ThemeLayer (Edge-First). 2. Micro: Search for Entities in DetailLayer (Node-First).

`chat(query: str, top_k_themes: int = 3, top_k_entities: int = 3)` ¶

Generate an answer using context from both layers.

`dump(folder_path: str)` ¶

Save both extracted graphs and metadata.

`load(folder_path: str)` ¶

Load extracted graphs and metadata.

`show(node_label_extractor=None, edge_label_extractor=None)` ¶

Visualize the specified layer interactively.

基于 RAG 的方法¶

LightRAG¶

hyperextract.methods.rag.Light_RAG ¶

Functions¶

__init__(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False) ¶

GraphRAG¶

hyperextract.methods.rag.Graph_RAG ¶

Attributes¶

community_reports: Dict[str, CommunityReport] = {} instance-attribute ¶

_community_graph: Optional[Any] = None instance-attribute ¶

_community_hierarchy: Dict[int, Dict[str, List[str]]] = {} instance-attribute ¶

_node_to_community: Dict[str, Dict[str, Any]] = {} instance-attribute ¶

Functions¶

__init__(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False) ¶

dump(folder_path: str | Path) -> None ¶

load(folder_path: str | Path) -> None ¶

_ensure_community_graph() ¶

build_communities(level: int = 0) ¶

search(query: str, top_k_nodes: int = 3, top_k_edges: int = 3, top_k: int | None = None, use_community: bool = False) -> Tuple[List, List, Optional[Dict]] ¶

_global_search_impl(query: str, top_k_nodes: int = 3, top_k_edges: int = 3) -> Tuple[List, List, Dict] ¶

_get_community_context_for_query(query: str) -> Dict ¶

HyperRAG¶

hyperextract.methods.rag.Hyper_RAG ¶

Functions¶

__init__(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False) ¶

HyperGraphRAG¶

hyperextract.methods.rag.HyperGraph_RAG ¶

Functions¶

__init__(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False) ¶

_extract_data(text: str) -> AutoHypergraphSchema[NodeSchema, EdgeSchema] ¶

CogRAG¶

hyperextract.methods.rag.Cog_RAG ¶

Attributes¶

metadata = {'created_at': datetime.now(), 'updated_at': datetime.now()} instance-attribute ¶

theme_layer = Cog_RAG_ThemeLayer(llm_client=llm_client, embedder=embedder, chunk_size=chunk_size, chunk_overlap=chunk_overlap, max_workers=max_workers, verbose=verbose) instance-attribute ¶

detail_layer = Cog_RAG_DetailLayer(llm_client=llm_client, embedder=embedder, chunk_size=chunk_size, chunk_overlap=chunk_overlap, max_workers=max_workers, verbose=verbose) instance-attribute ¶

llm = llm_client instance-attribute ¶

verbose = verbose instance-attribute ¶

nodes property ¶

edges property ¶

Functions¶

__init__(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False) ¶

feed_text(text: str) ¶

build_index() ¶

search(query: str, top_k_themes: int = 3, top_k_entities: int = 3) ¶

chat(query: str, top_k_themes: int = 3, top_k_entities: int = 3) ¶

dump(folder_path: str) ¶

load(folder_path: str) ¶

show(node_label_extractor=None, edge_label_extractor=None) ¶

`hyperextract.methods.rag.Light_RAG` ¶

`init(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False)` ¶

`hyperextract.methods.rag.Graph_RAG` ¶

`community_reports: Dict[str, CommunityReport] = {}` `instance-attribute` ¶

`_community_graph: Optional[Any] = None` `instance-attribute` ¶

`_community_hierarchy: Dict[int, Dict[str, List[str]]] = {}` `instance-attribute` ¶

`_node_to_community: Dict[str, Dict[str, Any]] = {}` `instance-attribute` ¶

`init(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False)` ¶

`dump(folder_path: str | Path) -> None` ¶

`load(folder_path: str | Path) -> None` ¶

`_ensure_community_graph()` ¶

`build_communities(level: int = 0)` ¶

`search(query: str, top_k_nodes: int = 3, top_k_edges: int = 3, top_k: int | None = None, use_community: bool = False) -> Tuple[List, List, Optional[Dict]]` ¶

`_global_search_impl(query: str, top_k_nodes: int = 3, top_k_edges: int = 3) -> Tuple[List, List, Dict]` ¶

`_get_community_context_for_query(query: str) -> Dict` ¶

`hyperextract.methods.rag.Hyper_RAG` ¶

`init(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False)` ¶

`hyperextract.methods.rag.HyperGraph_RAG` ¶

`init(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False)` ¶

`_extract_data(text: str) -> AutoHypergraphSchema[NodeSchema, EdgeSchema]` ¶

`hyperextract.methods.rag.Cog_RAG` ¶

`metadata = {'created_at': datetime.now(), 'updated_at': datetime.now()}` `instance-attribute` ¶

`theme_layer = Cog_RAG_ThemeLayer(llm_client=llm_client, embedder=embedder, chunk_size=chunk_size, chunk_overlap=chunk_overlap, max_workers=max_workers, verbose=verbose)` `instance-attribute` ¶

`detail_layer = Cog_RAG_DetailLayer(llm_client=llm_client, embedder=embedder, chunk_size=chunk_size, chunk_overlap=chunk_overlap, max_workers=max_workers, verbose=verbose)` `instance-attribute` ¶

`llm = llm_client` `instance-attribute` ¶

`verbose = verbose` `instance-attribute` ¶

`nodes` `property` ¶

`edges` `property` ¶

`init(llm_client: BaseChatModel, embedder: Embeddings, chunk_size: int = 2048, chunk_overlap: int = 256, max_workers: int = 10, verbose: bool = False)` ¶

`feed_text(text: str)` ¶

`build_index()` ¶

`search(query: str, top_k_themes: int = 3, top_k_entities: int = 3)` ¶

`chat(query: str, top_k_themes: int = 3, top_k_entities: int = 3)` ¶

`dump(folder_path: str)` ¶

`load(folder_path: str)` ¶

`show(node_label_extractor=None, edge_label_extractor=None)` ¶