LangChain

Build powerful, context-aware AI applications with modular components.

---

Overview

LangChain is a robust framework engineered to streamline the development of applications powered by large language models (LLMs). This repository showcases a comprehensive collection of modular tools, chains, agents, and retrieval strategies, enabling developers to construct sophisticated, data-aware, and intelligent systems. From orchestrating complex multi-step reasoning to integrating external data sources and defining dynamic agents, LangChain provides the building blocks to unlock the full potential of generative AI in real-world applications.

Features

• Modular Chains: Construct intricate sequences of LLM calls and other components for multi-step tasks.
• Intelligent Agents: Empower LLMs with tools to interact with their environment and make dynamic decisions.
• Robust RAG Implementation: Integrate external data sources through advanced Retrieval-Augmented Generation techniques, ensuring contextually rich responses.
• Flexible Prompt Engineering: Design, manage, and optimize prompts for various LLMs and use cases.
• Structured Output Parsing: Reliably extract structured data from LLM responses, enabling seamless integration with downstream systems.
• Advanced Runnables: Compose and execute complex logic flows with a unified, expressive interface.
• Tool Calling: Define and integrate custom tools that agents can use to perform actions and retrieve information.

Tech Stack

Architecture / Workflow

This repository structures a typical LangChain application workflow, demonstrating how various components interact to form an intelligent system.

1. Document Ingestion & Splitting: Raw documents (e.g., RAG/TextSplitter/demo.pdf, RAG/TextSplitter/demo.txt) are processed by Text Splitters (RAG/TextSplitter/split.py, RAG/TextSplitter/mark.py) into manageable chunks.
2. Embedding & Vector Storage: These chunks are then embedded using EmbeddedModels/gen.py and stored in a vector database (VectorStore/vector_stores.py), such as ChromaDB (RAG/Retreival/chroma_db).
3. Prompt Engineering: User queries and system instructions are crafted into effective prompts using templates defined in Prompt/prompt.py and Prompt/pr.py.
4. LLM/ChatModel Interaction: Prompts are passed to Large Language Models (LLMs/demo.py) or Chat Models (ChatModels/chat.py, ChatModels/hug.py) for generation.
5. Retrieval & Augmentation: For RAG applications, the system retrieves relevant context from the vector store (RAG/Retreival/ret.py) to augment the LLM's input.
6. Chains & Agents: Logic is orchestrated through Chains (Chains/simple.py, Chains/seq.py) or dynamic Agents (AIAgents/aiagentlangchain.py) that can utilize Tools (Tool/tools_creation.py, Tool/toolcalllling.py).
7. Output Parsing: Generated responses are parsed into structured formats using dedicated parsers (Parser/json_parser.py, StructuredOutput/parser.py) for further processing or display.

Project Structure

.
├── AIAgents/                     # Implementations of AI agents for complex task execution
│   └── aiagentlangchain.py       # Core agent logic using LangChain
├── Chains/                       # Modular components for chaining LLM calls and logic
│   ├── cond.py                   # Conditional chain examples
│   ├── para.py                   # Parallel chain execution
│   ├── seq.py                    # Sequential chain examples
│   └── simple.py                 # Basic chain implementations
├── ChatModels/                   # Integrations for various chat-optimized LLMs
│   ├── chat.py                   # Generic chat model interface
│   └── hug.py                    # Hugging Face chat model integration
├── EmbeddedModels/               # Models for generating text embeddings
│   ├── doc.py                    # Document embedding utilities
│   └── gen.py                    # General embedding generation
├── LLMs/                         # Core Large Language Model integrations
│   └── demo.py                   # Example LLM instantiation and usage
├── Parser/                       # Tools for parsing structured output from LLMs
│   ├── json_parser.py            # JSON output parsing
│   ├── py_parser.py              # Python code parsing
│   ├── string_parser.py          # Generic string parsing utilities
│   └── structure_parser.py       # General structured data parsing
├── Prompt/                       # Prompt templating and management utilities
│   ├── chat.py                   # Chat-specific prompt templates
│   ├── msg.py                    # Message-based prompt constructs
│   ├── pr.py                     # Prompt template examples
│   └── prompt.py                 # Core prompt template definitions
├── RAG/                          # Retrieval-Augmented Generation components
│   ├── Retreival/                # Document retrieval mechanisms
│   │   ├── chroma_db/            # ChromaDB vector store for retrieval
│   │   │   └── ...               # ChromaDB data files
│   │   ├── demo.pdf              # Example PDF document for retrieval
│   │   └── ret.py                # Retrieval logic implementation
│   └── TextSplitter/             # Utilities for splitting text into manageable chunks
│       ├── demo.pdf              # Example PDF for text splitting
│       ├── demo.txt              # Example TXT for text splitting
│       ├── mark.py               # Markdown text splitter
│       ├── pycode.py             # Python code splitter
│       ├── sem.py                # Semantic text splitter
│       ├── split.py              # Generic text splitting logic
│       ├── str.py                # String-based text splitter
│       └── vector_stores.py      # Integration with vector stores for text chunks
├── Runnables/                    # Composable and executable units for complex workflows
│   ├── branch.py                 # Conditional branching runnables
│   ├── par.py                    # Parallel execution runnables
│   ├── pass.py                   # Passthrough runnables
│   └── seq.py                    # Sequential runnables
├── StructuredOutput/             # Modules for enforcing and parsing structured LLM output
│   ├── parser.py                 # Structured output parsing logic
│   ├── str.py                    # String-based structured output
│   └── typdic.py                 # Type dictionary definitions for structured output
├── Tool/                         # Definitions and implementations of tools for agents
│   ├── ToolCalllling.ipynb       # Jupyter Notebook demonstrating tool calling
│   ├── toolcalllling.py          # Python script for tool calling examples
│   └── tools_creation.py         # Utilities for creating custom tools
└── VectorStore/                  # Generic interfaces and implementations for vector databases
    └── vector_stores.py          # Core vector store management and operations

Usage

This section guides you through setting up the environment, executing core components, and understanding the workflow of LangChain applications within this repository.

Setup

1. Clone the Repository:

   git clone https://github.com/langchain-ai/langchain.git
   cd langchain

2. Create a Virtual Environment (Recommended):

   python -m venv venv
   source venv/bin/activate  # On Windows: `venv\Scripts\activate`

3. Install Dependencies: Install the necessary libraries. While a requirements.txt is not explicitly provided, a typical LangChain setup involves:

   pip install langchain openai chromadb pypdf unstructured

   • langchain: The core framework.
   • openai: For OpenAI LLMs (if used).
   • chromadb: For the vector store.
   • pypdf, unstructured: For PDF document loading and parsing.
   • You may need additional packages depending on the specific LLM or tool integrations you intend to use (e.g., huggingface_hub for Hugging Face models).

Execute Pipeline

Follow these steps to explore a typical RAG-enabled LangChain application:

1. Prepare Documents: Ensure your input documents are placed in the appropriate directories. For example, RAG/Retreival/demo.pdf and RAG/TextSplitter/demo.txt are provided.

2. Split and Embed Documents into Vector Store: First, process your documents into chunks, then embed them and store them in ChromaDB.

   # Example: Split text (you might adapt this script or run a specific one)
   python RAG/TextSplitter/split.py
   
   # Example: Create and populate the vector store
   python VectorStore/vector_stores.py

   • Expected Output: This step will create and populate the RAG/Retreival/chroma_db directory with embedded document data.

3. Initialize LLM/ChatModel: Configure your desired LLM or Chat Model. You will typically set API keys as environment variables (OPENAI_API_KEY) or directly within the script.

   # Example: Initialize an LLM
   python LLMs/demo.py
   # Example: Initialize a Chat Model
   python ChatModels/chat.py

   • Expected Output: Successful initialization, potentially printing a basic LLM response.

4. Define and Use Prompts: Review and customize the prompt templates. These define how your queries are structured for the LLM.

   # Example: See prompt definition in action
   python Prompt/pr.py

   • Expected Output: Display of a formatted prompt or a sample LLM interaction using the prompt.

5. Run a Retrieval Chain or Agent: Execute a script that demonstrates retrieval from the vector store or an agent utilizing tools.

   # Example: Perform a retrieval query
   python RAG/Retreival/ret.py
   
   # Example: Run an AI Agent
   python AIAgents/aiagentlangchain.py
   
   # Example: Explore tool calling via a Python script
   python Tool/toolcalllling.py

   • Expected Output: Contextualized responses from the LLM based on retrieved documents, or agent actions and final answers.

6. Parse Structured Output: If your application expects structured data from the LLM, run the relevant parser.

   # Example: Parse a JSON output
   python Parser/json_parser.py
   # Example: Use structured output parsing utilities
   python StructuredOutput/parser.py

   • Expected Output: Parsed data displayed in a structured format (e.g., dictionary, object).

7. Explore Jupyter Notebooks: For interactive exploration and experimentation, launch Jupyter Notebook and navigate to the provided notebooks.

   jupyter notebook

   • Navigate your browser to Tool/ToolCalllling.ipynb to interactively experiment with tool calling.

Customization

• LLM Configuration: Modify LLMs/demo.py or ChatModels/chat.py to switch between different LLM providers (e.g., OpenAI, Hugging Face, local models) or adjust model parameters.
• Prompt Templates: Edit files in the Prompt/ directory (e.g., Prompt/pr.py, Prompt/prompt.py) to refine prompts for specific tasks or integrate new variables.
• RAG Sources: Replace RAG/Retreival/demo.pdf and RAG/TextSplitter/demo.txt with your own documents to build a custom knowledge base. Adjust text splitting strategies in RAG/TextSplitter/ for optimal chunking.
• Tools and Agents: Extend Tool/tools_creation.py to define new tools and integrate them into agents in AIAgents/aiagentlangchain.py for enhanced capabilities.

Expected Outputs

Upon successful execution of the pipeline steps, you should observe:

• A populated RAG/Retreival/chroma_db directory containing your embedded documents.
• Console output demonstrating LLM responses, agent thought processes, and retrieved document snippets.
• Structured data printed to the console if parsing steps are executed.
• Interactive results within Jupyter Notebooks for specific examples.

Contributing

We welcome contributions to enhance LangChain. To contribute, please follow these guidelines:

1. Fork the Repository: Start by forking the official LangChain repository to your GitHub account.
2. Create a Feature Branch: Create a new branch for your feature or bug fix:

   git checkout -b feat/your-feature-name

   (e.g., feat/add-new-parser or fix/rag-bug)
3. Implement Your Changes: Make your modifications, ensuring adherence to existing code style and best practices.
4. Write Clear Commit Messages: Craft descriptive commit messages that explain the purpose of your changes.

   feat: Add support for new structured output format
   
   This commit introduces a new parser in `StructuredOutput/` to handle YAML output,
   improving flexibility for data extraction.
   

5. Submit a Pull Request: Push your branch to your fork and open a pull request against the main branch of the original repository. Provide a clear description of your changes and reference any related issues.
6. Report Issues: If you encounter a bug or have a feature request, please open an issue on the GitHub repository, providing as much detail as possible.