Haystack

Haystack is an open-source framework for building production-ready LLM applications, retrieval-augmented generative pipelines, and state-of-the-art search systems. It connects your data to large language models.

Overview

Haystack provides a component-based architecture where you build pipelines by connecting modular components. Each component handles a specific task — document retrieval, text generation, embedding, ranking — and pipelines chain them together into end-to-end workflows.

Key Features

Modular Components: 100+ pre-built components for common NLP and LLM tasks
Pipeline Architecture: Connect components into directed acyclic graphs (DAGs)
Model Agnostic: Support for OpenAI, Anthropic, Cohere, Hugging Face, and local models
Document Stores: Pluggable backends including Elasticsearch, OpenSearch, Weaviate, Pinecone, Qdrant, and pgvector
RAG Pipelines: First-class support for retrieval-augmented generation patterns
Evaluation: Built-in evaluation framework for measuring pipeline quality
Serialization: Pipelines can be serialized to YAML for versioning and deployment

Installation

pip install haystack-ai

Quick Start

from haystack import Pipeline
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders import PromptBuilder

template = """
Given the following context, answer the question.
Context: {{context}}
Question: {{question}}
"""

pipe = Pipeline()
pipe.add_component("prompt_builder", PromptBuilder(template=template))
pipe.add_component("llm", OpenAIGenerator(model="gpt-4"))
pipe.connect("prompt_builder", "llm")

result = pipe.run({
    "prompt_builder": {
        "context": "Haystack is an AI framework by deepset.",
        "question": "What is Haystack?"
    }
})

Pipeline Patterns

RAG (Retrieval-Augmented Generation)

Retrieve relevant documents from a store, then generate answers using an LLM with the retrieved context.

Indexing Pipeline

Process documents through cleaning, splitting, embedding, and storage for later retrieval.

Agent Pipeline

Create agentic workflows with tool use, planning, and iterative reasoning using Haystack components.

Documentation

License

Apache License 2.0