Skip to main content
EVOKORE// SKILLS / CONTRIBUTING
Sign in
Workspace
promptfoo/promptfoomarkdownsha D934FD7CF614

CONTRIBUTING

contribution guide

Contributing to promptfoo

We love contributions from the community! Here is how to get started.

Development Setup

Prerequisites

  • Node.js 18+
  • npm or yarn

Quick Start

git clone https://github.com/promptfoo/promptfoo.git
cd promptfoo
npm install
npm run build

Running Tests

# Unit tests
npm test

# Watch mode
npm run test:watch

# Integration tests
npm run test:integration

# Linting
npm run lint

Project Structure

src/
  assertions/     # Built-in assertion implementations
  providers/      # LLM provider integrations
  redteam/        # Red teaming and adversarial testing
  evaluator/      # Core evaluation engine
  web/            # Web UI (React)
  cli/            # CLI commands
  types/          # TypeScript type definitions

Contribution Guidelines

What We Accept

  • Bug fixes: Include a failing test case when possible
  • New providers: Implement the ApiProvider interface
  • New assertions: Follow the assertion interface pattern
  • Documentation: Improvements, examples, tutorials
  • Performance: Evaluation speed optimizations
  • Red teaming: New attack strategies and detectors

Pull Request Process

  1. Fork the repository
  2. Create a feature branch from main
  3. Write tests for new functionality
  4. Ensure all tests pass: npm test
  5. Lint your code: npm run lint:fix
  6. Submit a PR with a clear title and description

Commit Convention

type(scope): description

feat(provider): add Mistral AI provider
fix(assertion): handle null responses in contains assertion
docs(readme): update installation instructions
test(evaluator): add edge case tests for parallel evaluation

Code Standards

  • TypeScript strict mode
  • Jest for testing
  • ESLint + Prettier for formatting
  • No any types without explicit justification
  • JSDoc comments on exported functions

Adding a New Provider

  1. Create src/providers/<name>.ts
  2. Implement the ApiProvider interface
  3. Register in src/providers/index.ts
  4. Add tests in test/providers/<name>.test.ts
  5. Add documentation in site/docs/providers/

License

By contributing, you agree that your contributions will be licensed under the MIT License.