Conductor

Conductor is a platform for orchestrating workflows across microservices. Originally developed at Netflix, Conductor has been battle-tested at scale to orchestrate millions of workflows daily.

Overview

Conductor enables you to define complex workflows as a series of tasks, where each task can be a microservice call, a human approval step, or a decision point. Workflows are defined declaratively using JSON, making them easy to version, review, and deploy.

Key Features

• Declarative Workflows: Define workflows in JSON/YAML without writing orchestration code
• Scalable Execution: Horizontally scalable architecture supporting millions of concurrent workflows
• Task Workers: Language-agnostic task workers in Java, Python, Go, C#, and more
• Visual Workflow Designer: Built-in UI for designing, monitoring, and debugging workflows
• Sub-Workflows: Compose complex workflows from reusable sub-workflow components
• Error Handling: Configurable retry policies, timeouts, and compensation workflows
• Event-Driven: Native support for event-based triggers and webhooks
• Human-in-the-Loop: Built-in support for manual approval and review steps

Quick Start

Using Docker Compose

git clone https://github.com/Conductor-oss/conductor.git
cd conductor
docker compose up -d

The Conductor UI will be available at http://localhost:5000.

Defining a Workflow

{
  "name": "order_processing",
  "version": 1,
  "tasks": [
    {
      "name": "validate_order",
      "taskReferenceName": "validate",
      "type": "SIMPLE"
    },
    {
      "name": "process_payment",
      "taskReferenceName": "payment",
      "type": "SIMPLE"
    },
    {
      "name": "ship_order",
      "taskReferenceName": "shipping",
      "type": "SIMPLE"
    }
  ]
}

Architecture

Conductor uses a server-worker architecture:

• Conductor Server: Manages workflow state, task scheduling, and event processing
• Task Workers: Poll for tasks, execute business logic, and report results
• Persistence: Pluggable storage backends (PostgreSQL, MySQL, Redis, Elasticsearch)

Documentation

Full documentation is available at conductor-oss.github.io/conductor.

License

Apache License 2.0