Concurrency, throttling and rate limits

Flow control decides when runs start and how many run at once. You set it per function, in its FunctionOptions or with attributes on its class, the SDK sends it to Inngest when the app syncs, and Inngest enforces it. Nothing in your app has to queue, lock or count.

SyncAccount.cs

using Inngest;

public sealed record AccountUpdated(string AccountId) : IInngestEventData
{
    public static string EventName => "account/updated";
}

public sealed record AccountDeleted(string AccountId) : IInngestEventData
{
    public static string EventName => "account/deleted";
}

public sealed class SyncAccount : InngestFunction<AccountUpdated>
{
    public override Task<object?> ExecuteAsync(FunctionContext<AccountUpdated> context, CancellationToken cancellationToken) =>
        Task.FromResult<object?>(null);
}

Its options, with a concurrency limit, a throttle, a debounce and a cancellation event:

Attributes

[InngestFunction("sync-account")]
[EventTrigger<AccountUpdated>]
[Concurrency(5, Key = "event.data.accountId")]
[Throttle(10, "1m")]
[Debounce("30s", Key = "event.data.accountId")]
[CancelOn<AccountDeleted>(Match = "data.accountId")]

The attributes go on SyncAccount, and inngest.AddFunction<SyncAccount>() registers it without options.

Code first

inngest.AddFunction<SyncAccount>(new FunctionOptions
{
    Id = "sync-account",
    Triggers = [Trigger.Event<AccountUpdated>()],
    Concurrency = [new ConcurrencyLimit(5) { Key = "event.data.accountId" }],
    Throttle = new Throttle(10, TimeSpan.FromMinutes(1)),
    Debounce = new Debounce(TimeSpan.FromSeconds(30)) { Key = "event.data.accountId" },
    CancelOn = [CancelOn.For<AccountDeleted>() with { Match = "data.accountId" }],
});

Each attribute sets the FunctionOptions member of the same name. Durations are TimeSpans in code and time strings in attributes, such as "30s", "5m" or "1h30m", because attribute arguments must be constants. The sections below show both forms. The function options builder shows the JSON the options turn into, and which record each attribute sets.

See it on a timeline

The timeline below sends the same burst of events to a function with each kind of flow control; change the settings to see which runs start at once, which wait, which are skipped and which are merged. The flow control explainer says how to read it.

Flow control

ConcurrencyThrottleRate limitDebounce

Concurrency

At most a number of runs execute at the same time. Others wait for a free slot and start in order. Concurrency in the Inngest docs.

Settings

Limit: 2Run duration: 8s

15 events, 15 runs, 13 delayed by up to 20s.started at oncedelayedskippedmerged into a later run

The numbers

Event arrives	Outcome	Its run starts
0s	started at once	0s
1s	started at once	1s
1.5s	delayed	8s
2s	delayed	9s
3s	delayed	16s
4s	delayed	17s
10s	delayed	24s
11s	delayed	25s
12s	delayed	32s
13s	delayed	33s
25s	delayed	40s
26s	delayed	41s
40s	delayed	48s
41s	delayed	49s
42s	delayed	56s

Throttle

At most a number of runs start per period. Others are queued and start later, spread out evenly. Throttle in the Inngest docs.

Settings

Limit: 3Period: 10sBurst: 0

15 events, 15 runs, 13 delayed by up to 17.0s.started at oncedelayedskippedmerged into a later run

The numbers

Event arrives	Outcome	Its run starts
0s	started at once	0s
1s	delayed	3.3s
1.5s	delayed	6.7s
2s	delayed	10s
3s	delayed	13.3s
4s	delayed	16.7s
10s	delayed	20s
11s	delayed	23.3s
12s	delayed	26.7s
13s	delayed	30.0s
25s	delayed	33.3s
26s	delayed	36.7s
40s	started at once	40s
41s	delayed	43.3s
42s	delayed	46.7s

Rate limit

At most a number of runs start per period. Events beyond that are skipped and never run. Rate limit in the Inngest docs.

Settings

Limit: 3Period: 10s

15 events, 11 runs, 4 skipped.started at oncedelayedskippedmerged into a later run

The numbers

Event arrives	Outcome	Its run starts
0s	started at once	0s
1s	started at once	1s
1.5s	started at once	1.5s
2s	skipped	never
3s	skipped	never
4s	started at once	4s
10s	started at once	10s
11s	started at once	11s
12s	skipped	never
13s	skipped	never
25s	started at once	25s
26s	started at once	26s
40s	started at once	40s
41s	started at once	41s
42s	started at once	42s

Debounce

One run starts once events stop arriving for a period, with the last event. A timeout caps the wait. Debounce in the Inngest docs.

Settings

Period: 5sTimeout (0 for none): 15s

15 events, 4 runs, 11 merged into a later event's run.started at oncedelayedskippedmerged into a later run

The numbers

Event arrives	Outcome	Its run starts
0s	merged into a later run	9s
1s	merged into a later run	9s
1.5s	merged into a later run	9s
2s	merged into a later run	9s
3s	merged into a later run	9s
4s	started at once	9s
10s	merged into a later run	18s
11s	merged into a later run	18s
12s	merged into a later run	18s
13s	started at once	18s
25s	merged into a later run	31s
26s	started at once	31s
40s	merged into a later run	47s
41s	merged into a later run	47s
42s	started at once	47s

Keys

Most options take a Key, an expression evaluated against each event. Runs with the same key value share a limit, and runs with different values do not. A concurrency limit of 1 keyed on event.data.accountId runs one step at a time per account, while all accounts run side by side. A key can combine fields: event.data.tenantId + "-" + event.data.region.

Concurrency

Attributes

[Concurrency(50)]
[Concurrency(1, Key = "event.data.accountId")]

Code first

Concurrency =
[
    new ConcurrencyLimit(50),
    new ConcurrencyLimit(1) { Key = "event.data.accountId" },
],

Limits how many steps of the function execute at the same time. Runs over the limit wait in a queue and start in order. A function can have up to two limits; a step starts when it fits both. Concurrency counts executing steps, not runs: a run that sleeps or waits for an event holds no slot. A limit of 0 means no limit, as in Inngest.

Scope widens a keyed limit beyond the function. ConcurrencyScope.Environment shares it with every function in the environment that uses the same key, and ConcurrencyScope.Account with every environment of the account. Both need a key. See concurrency.

Throttle

Attributes

[Throttle(10, "1m", Burst = 5, Key = "event.data.merchantId")]

Code first

Throttle = new Throttle(10, TimeSpan.FromMinutes(1)) { Burst = 5, Key = "event.data.merchantId" },

Limits how many runs start per period. Runs over the limit are queued, not dropped, and their starts are spread over the period. Burst lets that many extra runs start at once. Use it to protect an API with a rate limit of its own. The period must be at least one second. See throttling.

Rate limit

Attributes

[RateLimit(1, "4h", Key = "event.data.userId")]

Code first

RateLimit = new RateLimit(1, TimeSpan.FromHours(4)) { Key = "event.data.userId" },

Also limits how many runs start per period, but events over the limit are skipped and never run. Use it where running late is worse than not running, such as a “your report is ready” email. The period is between one second and 24 hours. See rate limiting.

Debounce

Attributes

[Debounce("30s", Key = "event.data.productId", Timeout = "5m")]

Code first

Debounce = new Debounce(TimeSpan.FromSeconds(30)) { Key = "event.data.productId", Timeout = TimeSpan.FromMinutes(5) },

Waits until events stop arriving for the period, then runs once with the last event. Each new event restarts the period. Timeout caps the total wait, so a steady stream of events still runs the function every so often. The period is between one second and seven days. See debounce.

Batching

Attributes

[BatchEvents(100, "10s", Key = "event.data.importId")]

Code first

BatchEvents = new BatchEvents(100, TimeSpan.FromSeconds(10)) { Key = "event.data.importId" },

Runs the function with up to MaxSize events at once, or with what has arrived when the timeout passes. The events are in context.Events; context.Event is the first of them. If limits batching to events that match an expression; others run on their own.

var batch = context.Events.Select(e => e.Data).ToList();
await context.Step.RunAsync("write-rows", ct => rows.WriteAsync(batch, ct));

MaxSize must be at least 2 and the timeout at least one second; how long it can be depends on your Inngest plan, which the SDK leaves to Inngest. Inngest does not allow batching together with cancellation, debounce, singletons or priority, and the SDK rejects those combinations at startup. See batching.

Priority

Attributes

[Priority("event.data.plan == 'enterprise' ? 120 : 0")]

Code first

Priority = new Priority("event.data.plan == 'enterprise' ? 120 : 0"),

Run is an expression that returns a number of seconds, from -600 to 600. A run with priority 120 starts ahead of runs that have been waiting for up to two minutes; a negative value holds a run back. Priority only matters when runs queue up, so it goes together with concurrency or throttling. See priority.

Idempotency

Attributes

[InngestFunction("record-payment", Idempotency = "event.data.paymentId")]

Code first

Idempotency = "event.data.paymentId",

Starts at most one run per value of the expression in 24 hours, for this function. Other functions triggered by the same event are not affected. Inngest implements it as a rate limit of 1 per 24 hours, so it replaces any RateLimit. To deduplicate an event for every function, give the event an Id instead. See idempotency.

Singleton

Attributes

[Singleton(SingletonMode.Skip, Key = "event.data.accountId")]

Code first

Singleton = new Singleton(SingletonMode.Skip) { Key = "event.data.accountId" },

Allows one active run per key. With SingletonMode.Skip a new run is skipped while another is active; with SingletonMode.Cancel the new run cancels the active one. Unlike a concurrency limit of 1, this covers the whole run, sleeps and waits included. See singleton.

Cancel on events

Attributes

[CancelOn<AccountDeleted>(Match = "data.accountId")]
[CancelOn("subscription/changed", If = "async.data.plan == 'free'", Timeout = "7d")]

Code first

CancelOn =
[
    CancelOn.For<AccountDeleted>() with { Match = "data.accountId" },
    new CancelOn("subscription/changed") { If = "async.data.plan == 'free'", Timeout = TimeSpan.FromDays(7) },
],

Cancels a run when a matching event arrives. Match compares a property of the triggering event with the same property of the cancelling one; If is a full expression where event is the triggering event and async the cancelling one; when both are set, both apply. CancelOn.For<T>() and [CancelOn<T>] take the event’s name from a typed event. Timeout limits how long after the run is scheduled an event can still cancel it. A function can have up to five.

Cancellation takes effect between steps: a step that is running finishes, and the run stops before the next one. See cancellation.

Timeouts

Attributes

[Timeouts(Start = "10m", Finish = "1h")]

Code first

Timeouts = new FunctionTimeouts { Start = TimeSpan.FromMinutes(10), Finish = TimeSpan.FromHours(1) },

Start cancels a run that waited in the queue longer than that before its first step. Finish cancels a run that takes longer than that from its first step to the end. Neither interrupts a step that is running; give the calls inside your steps timeouts of their own. See timeouts.

Validation

The SDK checks these options when the app starts, against the limits Inngest enforces, whether they come from code or attributes. It throws one InvalidOperationException that lists every problem of every function, each naming its function, for example Function "import-rows": BatchEvents cannot be combined with Debounce. A duration in an attribute that is not a time string fails as well, when AddFunction reads it. A mistake fails the deploy instead of the sync.

The flow control overview in the Inngest docs explains how the options interact and in which order Inngest applies them.

Related examples

The webhook pipeline uses idempotency and a throttle, nightly billing concurrency and a singleton, and order fulfillment cancellation.