Deploying background jobs to production

An app with Inngest functions is an ordinary ASP.NET Core app with one extra endpoint. It deploys like any other, with three additions: it runs in cloud mode with its keys, Inngest must be able to reach its endpoint, and it has to be synced after each deploy.

Inngest Cloud

or a self-hosted Inngest

signed POST /api/inngest

events, with the event key

functions, on each sync

Your app, in containers

behind a load balancer at https://shop.example.com; any instance can answer

• INNGEST_SIGNING_KEY
• INNGEST_EVENT_KEY
• INNGEST_SERVE_ORIGIN

In production, Inngest calls the app's public endpoint with requests signed by the signing key, and any instance behind the load balancer can answer them. The app sends events with the event key, and reports its functions to Inngest when it syncs after a deploy.

Checklist

• Cloud mode. Do not set INNGEST_DEV in production. Cloud mode is the default.
• Keys. Set INNGEST_SIGNING_KEY and INNGEST_EVENT_KEY from the Inngest dashboard, from your secret store. Without a signing key the endpoint answers Inngest with errors. See Configuration.
• A public endpoint. Inngest calls /api/inngest over HTTPS. Behind a proxy or load balancer, set INNGEST_SERVE_ORIGIN to the public origin so that syncs report the right URL. If the app cannot be reachable, use Connect instead.
• A sync after every deploy. See below.
• A readiness check. Point the probe at a path of its own, and add the check that tells whether the app can work with Inngest. See Health checks.
• Time for long steps. A step runs inside one request from Inngest, so proxy and platform request timeouts apply to it. Keep steps well under them, split long work into several steps, or use Connect.

Syncing

Inngest only knows about the functions the app reported at its last sync. The first time, add the app in the Inngest dashboard with its URL; Inngest syncs it in-band, with a signed request that the SDK answers with its functions. After later deploys, sync again from the deploy pipeline:

curl -X PUT https://shop.example.com/api/inngest

That PUT makes the SDK post its configuration to Inngest, authenticated with the signing key. Inngest’s apps and syncs page describes both kinds and the dashboard view of them.

In a container

A multi-stage build is all it takes. The image below runs as the non-root user of the .NET images and listens on port 8080:

Dockerfile

FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
WORKDIR /src
COPY . .
RUN dotnet publish Shop/Shop.csproj -c Release -o /app

FROM mcr.microsoft.com/dotnet/aspnet:10.0
WORKDIR /app
COPY --from=build /app .
USER $APP_UID
EXPOSE 8080
ENTRYPOINT ["dotnet", "Shop.dll"]

For a Native AOT app, publish with -r linux-x64 and use mcr.microsoft.com/dotnet/runtime-deps:10.0 as the final image.

Run it with the keys as environment variables:

docker run -p 8080:8080 -e INNGEST_SIGNING_KEY=signkey-prod-... -e INNGEST_EVENT_KEY=... -e INNGEST_SERVE_ORIGIN=https://shop.example.com shop:latest

In Azure Functions

An Azure Functions app serves the endpoint from an HTTP-triggered function of the isolated worker, and has its own limit on how long one call can take. See Azure Functions.

Health checks

Point health checks at an endpoint of their own, not at /api/inngest: in cloud mode the SDK answers unsigned requests there with 401 Unauthorized, so a probe of it never passes. A liveness endpoint that only shows the process is up is one line:

app.MapGet("/healthz", () => Results.Ok());

Whether the app can work with Inngest

AddInngest on the health checks builder adds a check for an app that serves its functions with MapInngest. It is meant for the readiness probe: tag it, serve the tagged checks on a path of their own, and keep it out of the liveness probe, because restarting the app does not bring Inngest back.

builder.Services.AddHealthChecks().AddInngest(tags: ["ready"]);

using Microsoft.AspNetCore.Diagnostics.HealthChecks;

app.MapHealthChecks("/healthz/ready", new HealthCheckOptions
{
    Predicate = check => check.Tags.Contains("ready"),
});
app.MapHealthChecks("/healthz/live", new HealthCheckOptions
{
    Predicate = _ => false,
});

ASP.NET Core answers 200 for a healthy or degraded result and 503 for an unhealthy one, which is what a probe reads. In Kubernetes the two paths go into the probes of the container:

readinessProbe:
  httpGet: { path: /healthz/ready, port: 8080 }
  periodSeconds: 10
  timeoutSeconds: 5
livenessProbe:
  httpGet: { path: /healthz/live, port: 8080 }

The check reports what it found as entries in the data of its result, and names the first problem in the description. It checks two things:

Part	Healthy when	Entries in the data
Configuration	In cloud mode, a signing key is set. With RequireEventKey, an event key is set too; an app that only receives calls from Inngest needs none.	signingKey, signingKeyFallback and eventKey: set, not set or missing
Inngest	Inngest answers a request with a 2xx. In cloud mode the request carries the signing key, so a key that Inngest does not know fails it; if Inngest refuses the signing key, the fallback key is tried, as for every call to the API.	mode, apiBaseUrl, api (ok, refused, unreachable, timed out, rate limited or unexpected response), apiStatusCode, acceptedKey (signingKey or signingKeyFallback) and checkedAt

In cloud mode, and against a self-hosted server, the request is GET /v2/apps?limit=1 on the API address (https://api.inngest.com, or INNGEST_BASE_URL). It lists at most one app and changes nothing. A self-hosted server needs Inngest v1.40.0 or later for it. The dev server holds no keys, so in dev mode the request is GET /health, its own health endpoint.

The result never shows a key, the hashed token the SDK sends in its place, or the event key; the data says whether each is set. Nor does the address: only its scheme, host and port.

A missing key fails the check without a request. A failed check reports failureStatus, Unhealthy unless you give another. Choose it for what an outage of Inngest should do to the app. Unhealthy takes the instance out of rotation, which suits an app that exists to run Inngest functions. An app that mostly serves its own users, and sends work to Inngest on the side, should stay in rotation: use HealthStatus.Degraded, which still answers 200 and still shows the problem in the report. When Inngest rate limits the request (429), the result is Degraded whatever failureStatus says, because the request proves nothing about the key, and taking every replica out of rotation because the probes ask too often would be an outage the probes made.

Cost and timeouts

Every probe could send a request to Inngest, and a load balancer, a kubelet and a monitor may probe every few seconds, on every replica. The check keeps Inngest’s answer instead: for CacheDuration, 30 seconds by default, and for FailureCacheDuration, 5 seconds, when the answer was a failure, so that an instance that was not ready is back in rotation soon after Inngest is reachable again. A rate limited answer is kept for the full CacheDuration. Probes that arrive while a request is out wait for it instead of sending their own. Each registration and each replica keeps its own answer, so N replicas send about N requests per interval.

using Microsoft.Extensions.Diagnostics.HealthChecks;

builder.Services.AddHealthChecks().AddInngest(
    options =>
    {
        options.RequireEventKey = true;
        options.CacheDuration = TimeSpan.FromMinutes(1);
    },
    failureStatus: HealthStatus.Degraded,
    tags: ["ready"],
    timeout: TimeSpan.FromSeconds(5));

A probe never waits on Inngest for long. The request is cancelled when the probe is, for example when the kubelet’s timeoutSeconds has passed, and when the timeout of the check has, which is 10 seconds unless you give one. Give the probe itself a few seconds: the first request after a start opens a connection to Inngest.

What it does not check

• Whether Inngest can reach the app. The check runs inside the app and calls Inngest, not the other way round. A firewall, a wrong INNGEST_SERVE_ORIGIN or a proxy that blocks Inngest shows up as failed syncs and calls in the Inngest dashboard.
• Whether the event key is valid. Only that it is set. Finding out whether Inngest accepts it would take a request to the Event API, which the check does not send, and the address events go to is not probed either.
• Whether the app is synced, unless you ask for it, as below.

Whether the app is synced

Inngest keeps the functions an app reported at its last sync, and its Get app endpoint, which Inngest marks Beta, returns that record. With CheckSync the check asks for the app by its id, in addition to the request above, and fails while

• Inngest has no app by that id,
• the app is archived,
• its latest sync failed, by the status of the sync or, as the dev server reports it, by an error,
• Inngest synced another AppVersion than the one this instance has, when both have one, or
• Inngest knows another number of functions than this instance serves.

The data then holds sync (synced, not registered, archived, failed, version mismatch or function count mismatch), functionsServed and, as far as Inngest reports them, functionsInInngest, syncStatus, syncedAt, appVersion and syncedAppVersion. The endpoint is in Inngest v1.27.1 and later.

The check compares counts and a version, not function ids or settings: a renamed function keeps the count. Set AppVersion, to a build number or a commit, and every deploy is compared. A flag kept in the process could not answer this question at all, because Inngest syncs through whichever instance the load balancer picks, so what one instance remembers says nothing about the others; this check asks Inngest.

Do not use it for the readiness of a rollout. The sync usually comes after the new instances serve, as above, so a deploy that changes the functions fails the check until the sync, and a rollout that waits for readiness would wait for a sync that only comes after it. Give the check a registration of its own, with its own name and tag, for monitoring or for a step of the pipeline after the sync:

using Microsoft.Extensions.Diagnostics.HealthChecks;

builder.Services.AddHealthChecks()
    .AddInngest(tags: ["ready"])
    .AddInngest(options => options.CheckSync = true, "inngest-sync", HealthStatus.Degraded, ["sync"]);

Connect workers

A Connect worker has no endpoint for Inngest, but it can still report whether it is connected. The Connect health check is healthy only while the worker holds an active connection, so use it for the readiness probe, not for liveness: a worker that is reconnecting recovers by itself.

builder.Services.AddHealthChecks().AddInngestConnect();

Scaling and restarts

The app holds no state between calls: a run’s state lives in Inngest, and any instance can serve any call, including the next step of a run another instance started. Run as many replicas as you need behind a load balancer. A call that is cut off by a restart or a crash is retried by Inngest, so let the host shut down gracefully but do not worry about losing runs.

A load balancer that closes idle connections cuts off a call that runs longer than its timeout, and Inngest retries the call from the step it was running. Keep each step shorter than the timeout, or turn on streaming, which keeps a long call’s connection busy.

On AWS Lambda

A Lambda function can serve the endpoint, behind an API Gateway REST API or HTTP API or a function URL, as a managed runtime function or a Native AOT executable. Inngest.AwsLambda handles the events of all three, and the timeouts of API Gateway apply to the steps it runs. Running on AWS Lambda covers the setup and the limits.

Self-hosting Inngest

To run Inngest yourself instead of using Inngest Cloud, point the SDK at your server with INNGEST_BASE_URL and use the keys your server is configured with. See self-hosting in the Inngest docs.

Related examples

The Connect worker example includes Dockerfiles for a worker and a web app.