Trigger.dev is an open-source platform for building and running reliable background jobs, asynchronous workflows, AI agents, and long-running tasks in Node.js/TypeScript applications. The system provides task orchestration, execution management, retry handling, concurrency control, and real-time observability through a unified platform.
This document provides a high-level overview of the entire Trigger.dev system architecture, major components, and how they interact. For detailed information about specific subsystems:
Sources: package.json1-121 apps/webapp/package.json1-303 packages/trigger-sdk/package.json1-132
Trigger.dev enables developers to define background tasks as typed functions using task() or schemaTask(), trigger them via SDK methods like trigger() or triggerAndWait(), and monitor execution through a web dashboard with real-time updates. The platform handles task queuing, retry logic, checkpointing for long-running operations, concurrency management, and distributed execution across multiple workers.
The system supports multiple deployment targets including cloud-hosted (Trigger.dev Cloud) and self-hosted environments, with containerized task execution via Docker or Kubernetes providers.
Sources: packages/trigger-sdk/CHANGELOG.md1-430 packages/cli-v3/package.json1-163
Architecture Overview: The platform consists of three primary layers: Developer Interface (CLI/SDK for task definition, React hooks for frontend integration), Platform Core (Express/Remix webapp hosting the dashboard and API gateway), and Execution Infrastructure (RunEngine for orchestration, workers for background jobs). Task runtime is handled by provider applications (Coordinator, Docker/Kubernetes providers, Supervisor) that manage containerized execution. Data persistence uses PostgreSQL for relational data, Redis for queues and coordination, ClickHouse for observability, and S3-compatible storage for large payloads.
Sources: apps/webapp/app/root.tsx1-138 apps/webapp/server.ts1-150 internal-packages/run-engine/src/engine/index.ts1-800
| Application | Location | Purpose | Key Technologies |
|---|---|---|---|
| webapp | apps/webapp | Web dashboard, API gateway, task orchestration | Express, Remix, Socket.io, Prisma |
| coordinator | apps/coordinator | Worker coordination via Socket.io | Socket.io server, nanoid |
| docker-provider | apps/docker-provider | Local task execution in Docker | execa, dockerode |
| kubernetes-provider | apps/kubernetes-provider | Cloud task execution in K8s | @kubernetes/client-node |
| supervisor | apps/supervisor | v4 task container lifecycle management | dockerode, Socket.io |
Sources: package.json1-121 apps/webapp/package.json1-303 pnpm-lock.yaml113-212
| Package | NPM Name | Purpose | Exports |
|---|---|---|---|
| SDK | @trigger.dev/sdk | Task definition and triggering | task(), trigger(), batch.trigger(), streams, metadata |
| CLI | trigger.dev | Development and deployment commands | dev, deploy, whoami, env, mcp |
| Core | @trigger.dev/core | Shared types and utilities | Schemas, utilities, OpenTelemetry integration |
| Build | @trigger.dev/build | Build extensions system | prismaExtension, pythonExtension, etc. |
| React Hooks | @trigger.dev/react-hooks | Frontend integration | useRealtimeRun, useRealtimeRunWithStreams |
Sources: packages/trigger-sdk/package.json1-132 packages/cli-v3/package.json1-163 packages/core/package.json1-500 packages/build/package.json1-200 packages/react-hooks/package.json1-76
Internal Package Dependencies: The run-engine package is the core orchestration layer, depending on database for Prisma models, redis for coordination, cache for performance, zod-worker for background job execution, and tracing for observability. The clickhouse package handles analytics queries using the tsql query parser for TRQL (Trigger Query Language).
Sources: pnpm-lock.yaml1020-1280 apps/webapp/package.json58-64
Task Execution State Machine: Tasks flow through validation and idempotency checks via TriggerTaskService, then enter the queue through EnqueueSystem. The DequeueSystem claims tasks from Redis queues when workers are available. The RunAttemptSystem manages execution lifecycle, while WaitpointSystem handles blocking operations and CheckpointSystem manages suspension for long-running tasks. Retries are controlled by retry policies in RunAttemptSystem.
Sources: internal-packages/run-engine/src/engine/index.ts1-800 internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts1-1200 internal-packages/run-engine/src/engine/systems/dequeueSystem.ts1-800 internal-packages/run-engine/src/engine/systems/waitpointSystem.ts1-600
RunEngine System Architecture: The RunEngine class coordinates multiple specialized subsystems. Core systems handle task lifecycle (DequeueSystem, EnqueueSystem, RunAttemptSystem), execution state (ExecutionSnapshotSystem), and advanced features (WaitpointSystem for blocking, CheckpointSystem for suspension, BatchSystem for parallel operations). Infrastructure components provide queueing (RunQueue with fair selection), locking (RunLocker for distributed coordination), and eventing (EventBus). Additional systems handle TTL expiration, delayed runs, debouncing, and version waiting.
Sources: internal-packages/run-engine/src/engine/index.ts82-290 internal-packages/run-engine/src/engine/types.ts1-300
Data Storage Architecture: PostgreSQL serves as the primary relational database with read-write splitting via PrismaClient (writes) and PrismaReplicaClient (reads). Redis is used for multiple purposes with separate instances: run locking, task queuing, caching, pub/sub, and realtime streams. ClickHouse stores observability data (logs, traces, metrics) queryable via TRQL. S3-compatible object storage handles large payloads and artifacts that exceed the TASK_PAYLOAD_OFFLOAD_THRESHOLD.
Sources: apps/webapp/app/env.server.ts50-270 apps/webapp/package.json28-230 internal-packages/run-engine/src/engine/index.ts113-205
Development to Production Pipeline: Developers write tasks in the trigger/ directory and configure build extensions in trigger.config.ts. The trigger dev command provides local execution with hot reload while indexing tasks with the platform. For deployment, trigger deploy executes build extensions (Prisma, Python, Playwright, etc.), bundles code with esbuild, builds a Docker image, and pushes to the registry. The platform deploys to selected environments where the Supervisor manages container execution, orchestrated by RunEngine, with real-time monitoring via the dashboard.
Sources: packages/cli-v3/package.json1-163 packages/build/package.json1-200 apps/supervisor/package.json1-50
The platform uses environment-based configuration validated by Zod schemas in apps/webapp/app/env.server.ts1-900 Key configuration areas:
| Configuration Domain | Environment Variables | Purpose |
|---|---|---|
| Database | DATABASE_URL, DATABASE_READ_REPLICA_URL, DIRECT_URL | PostgreSQL connection strings with read replica support |
| Redis Instances | REDIS_HOST, RATE_LIMIT_REDIS_HOST, CACHE_REDIS_HOST, PUBSUB_REDIS_HOST, REALTIME_STREAMS_REDIS_HOST | Separate Redis instances for different concerns |
| Object Storage | OBJECT_STORE_BASE_URL, OBJECT_STORE_ACCESS_KEY_ID, ARTIFACTS_OBJECT_STORE_BUCKET | S3-compatible storage configuration |
| Deployment | DEPLOY_REGISTRY_HOST, DEPLOY_REGISTRY_NAMESPACE, DEPOT_TOKEN | Container registry and build settings |
| Execution | DEFAULT_ENV_EXECUTION_CONCURRENCY_LIMIT, CHECKPOINT_THRESHOLD_IN_MS, PROD_TASK_HEARTBEAT_INTERVAL_MS | Runtime execution parameters |
| Observability | INTERNAL_OTEL_TRACE_EXPORTER_URL, DEV_OTEL_EXPORTER_OTLP_ENDPOINT | OpenTelemetry export endpoints |
Sources: apps/webapp/app/env.server.ts1-900
Observability Pipeline: Tasks emit OpenTelemetry traces, logs, and metrics via SDK instrumentation. These are exported via OTLP HTTP exporters to the platform's OTLP importer, which parses protobuf data and stores it in ClickHouse. The TRQL query engine provides SQL-like querying with typed results via query.execute(). The dashboard visualizes this data with real-time updates via Electric SQL shape streams. External exporters can also send telemetry to third-party services like Axiom or Datadog.
Sources: packages/trigger-sdk/CHANGELOG.md250-337 packages/core/package.json170-206 internal-packages/otlp-importer/package.json1-50
Build System Architecture: The monorepo uses pnpm workspaces for package management with Turborepo for build orchestration. Turborepo defines a pipeline in turbo.json that schedules tasks with dependency awareness and caching. Packages are built using different tools: tshy for dual ESM/CommonJS builds (SDK, Core, CLI), esbuild for fast bundling (server components), and Remix compiler for the webapp. Dependency patches in patches/ modify upstream packages. The onlyBuiltDependencies configuration in pnpm ensures native modules are built correctly.
Sources: package.json1-121 turbo.json1-100 pnpm-lock.yaml1-100
The platform exposes multiple API surfaces for different use cases:
@trigger.dev/sdk)task(), schemaTask() for defining tasks with typed payloadstrigger(), triggerAndWait(), batchTrigger() for executing tasksstreams.define(), streams.input() for bidirectional communicationquery.execute() for TRQL-based data queriesmetadata.set(), metadata.root, metadata.parent for run metadataotel.tracer, otel.metrics.getMeter() for observabilityai.tool() for LLM integrationtrigger.dev)trigger dev for local execution with hot reloadtrigger deploy for production deploymentstrigger whoami for project/profile informationtrigger env for managing environment variablestrigger mcp for AI coding assistant integration@trigger.dev/react-hooks)useRealtimeRun() for live run updatesuseRealtimeRunWithStreams() for stream eventsuseInputStreamSend() for sending data to tasksuseRealtimeRunWithTags() for filtering by tagsSources: packages/trigger-sdk/package.json24-127 packages/cli-v3/package.json72-82 packages/react-hooks/package.json1-76
The project uses Changesets for version management with automated workflows:
v4-beta and normal release channels@trigger.dev/* packages maintain synchronized versions (current: 4.4.2)The .server-changes system tracks server-side changes separately from SDK changes, ensuring coordinated releases across the platform.
Sources: package.json43-48 packages/trigger-sdk/package.json3-4 packages/cli-v3/package.json3-4
Refresh this wiki
This wiki was recently refreshed. Please wait 4 days to refresh again.