Sandbox errors and retries

Sandbox operations can change live external state. Retry decisions must account for whether a mutation was definitely rejected, definitely completed, or may have happened without a confirmed response.

REST error responses

REST errors use an errors array:

{
  "errors": [
    {
      "code": "operation_ambiguous",
      "message": "Sandbox process operation may have executed and must not be retried automatically"
    }
  ]
}

Use code for control flow. Messages are written for humans and can change.

SDK errors

The TypeScript SDK converts API and transport failures to SandboxError:

class SandboxError extends Error {
  action: SandboxAction;
  code: SandboxErrorCode;
  status?: number;
  sandboxId?: string;
  processId?: string;
  ambiguous: boolean;
  retryable: boolean;
  requestId?: string;
  details: readonly Record<string, unknown>[];
  cause?: unknown;
}

Invalid local input and malformed server responses throw SandboxValidationError.

The direct client never retries automatically, even when retryable is true.

Inside step.sandbox:

  • SandboxError with retryable: true becomes a retriable step error;
  • non-retryable SandboxError becomes NonRetriableError; and
  • SandboxValidationError becomes NonRetriableError.

This uses the function's ordinary step retry behavior. It does not provide exactly-once dispatch.

Error reference

HTTPCodeTypical conditionGuidance
400invalid_requestInvalid JSON or unknown fieldsFix the request
400missing_fieldRequired field omittedFix the request
400invalid_field_formatInvalid UUID, command, signal, timeout, cursor, path, mode, or tail sizeFix the request
401authorization_header_missingMissing authorizationFix credentials
401invalid_api_keyInvalid API keyRotate or replace credentials
403access_deniedSandbox access is not enabledRequest access
404sandbox_not_foundSandbox missing or hidden by workspace scopeRe-check the target
404sandbox_file_not_foundSandbox or regular file missingRe-check the target
404sandbox_process_not_foundProcess missingRe-check the target
404sandbox_process_output_not_retainedProcess exists but output was evictedOutput cannot be recovered through this API
409sandbox_name_takenActive sandbox name is already usedChoose another name or use the existing sandbox
409invalid_requestSandbox is not in the required stateGet and inspect current state
409operation_ambiguousMutation may have happenedNever retry automatically
413sandbox_exec_output_too_largeDirect output exceeded 4 MiBCommand may have run; do not retry blindly
413sandbox_file_too_largeFile exceeds 100 MiBReduce or split the file
429rate_limitedRequest rejected by rate limitRetry with bounded backoff
500internal_errorUnexpected failureRetry only when the operation is proven safe
503compute_unavailableNode or compute unavailableSafe reads and confirmed pre-dispatch mutations can retry
504sandbox_exec_timed_outExec observation timed outCommand may have run; do not retry blindly
504sandbox_process_wait_timed_outWait observation timed outProcess continues; waiting again is safe

sandbox_exec_output_too_large, sandbox_exec_timed_out, and operation_ambiguous all produce SandboxError.ambiguous === true and retryable === false.

Reads and mutations

Safe reads do not modify sandbox state:

  • List and Get sandbox;
  • List, Get, and Wait process;
  • retained process output;
  • sandbox and process streams; and
  • file download.

Mutations can have external effects:

  • Create sandbox;
  • captured Exec;
  • Destroy sandbox;
  • Start process;
  • Signal process; and
  • file upload.

Retry safe reads

For 429 rate_limited and 503 compute_unavailable, retry a direct read with bounded exponential backoff and jitter:

This example is for the direct inngest.sandboxes client. Do not add this loop around step.sandbox; Inngest already retries its retryable step errors.

import { SandboxError } from "inngest";

const sleep = (milliseconds: number) =>
  new Promise((resolve) => setTimeout(resolve, milliseconds));

async function getSandboxWithBackoff(id: string) {
  for (let attempt = 0; attempt < 4; attempt++) {
    try {
      return await inngest.sandboxes.get(id);
    } catch (error) {
      if (!(error instanceof SandboxError) || !error.retryable) {
        throw error;
      }

      const delay = Math.min(250 * 2 ** attempt, 2_000);
      await sleep(delay + Math.floor(Math.random() * 100));
    }
  }

  throw new Error("Sandbox is still unavailable");
}

The direct client does not reconnect streams. A manual reconnect can replay retained chunks, so consumers must tolerate duplicates.

Retry mutations only before dispatch

A mutation can retry only when the service confirms that it failed before dispatch. Examples include a 429 rate-limit response or a 503 response produced before a node session was obtained.

Do not infer safety from a missing HTTP response. The direct SDK conservatively maps a transport failure during a mutation to operation_ambiguous.

Handle ambiguous operations

Ambiguity means the platform cannot prove whether the mutation happened:

Do not catch an ambiguous error just to log it or call the operation again. Inside an Inngest function, let the error escape; it is non-retryable. With the direct client, let it escape unless you are implementing an explicit reconciliation or operator-review path.

Possible reconciliation strategies:

  • Create: list sandboxes and inspect the intended unique name.
  • Destroy: get and inspect TERMINATING or terminal state.
  • Start: list processes if the UUID is known. Direct Start does not expose its generated UUID before dispatch, so strong reconciliation is not always possible.
  • Signal: get or wait, remembering that not every signal terminates a process.
  • File upload: download or inspect the destination.
  • Captured Exec: design the command to be idempotent or write a completion marker.

Reconciliation is application policy. The SDK does not guess.

Understand step.sandbox replay

step.sandbox uses ordinary step.run memoization:

  1. The step handler sends the REST request.
  2. The SDK converts the response to JSON-safe data.
  3. Inngest persists the result.
  4. Replay reconstructs the facade from that result.

If a REST mutation commits and the function process stops before the result is persisted, Inngest can run the handler again. There is no sandbox-specific executor fence and no HTTP Idempotency-Key.

An observed operation_ambiguous is non-retriable. A process crash cannot report that error, so mutation commands should tolerate the ordinary at-least-once step window.

Validation and size limits

Sandbox

ValueLimit
Name1–63 lowercase letters, digits, _, or -
vCPUPositive unsigned 32-bit integer
MemoryPositive unsigned 32-bit integer in MiB
List pageDefault 50, maximum 250
Create JSON body1 MiB

Entitlements or capacity can impose lower effective resource limits.

Commands and process Start

ValueLimit
Argument count1–128
Sum of argument UTF-8 bytes32 KiB
Environment entries256
Sum of environment KEY=value UTF-8 bytes64 KiB
Working-directory UTF-8 bytes4096
Encoded process specification96 KiB
JSON body1 MiB

Additional rules:

  • command[0] must be absolute;
  • arguments, keys, values, and cwd cannot contain NUL;
  • environment keys must be non-empty and cannot contain =;
  • the SDK rejects invalid Unicode surrogate sequences; and
  • environment replaces rather than merges with the guest environment.

Captured Exec

ValueLimit
Default timeout30 seconds
Maximum timeout5 minutes
Direct REST combined stdout and stderr4 MiB
step.sandbox retained stdout and stderr2 MiB

The middleware applies the 2 MiB durable limit after a successful REST response. Larger results succeed with deterministic tail truncation and report original and retained byte counts.

Managed processes

ValueLimit
Process List pageDefault 50, maximum 250
SignalInteger 1–64
Wait default timeout30 seconds
Wait maximum timeout5 minutes
Retained output per processApproximately 512 KiB
Output rings retainedNewest 32
tailBytes0–524,288

There is no process runtime timeout. Stop a process with a signal.

Files

ValueLimit
File size100 MiB
PathAbsolute, no NUL, at most 4096 bytes
Upload modeOctal 00010777; default 0644
File typeRegular files only

Stream errors after HTTP 200

An NDJSON stream can fail after response headers are committed. The API sends a terminal frame:

{
  "type": "error",
  "errors": [
    {
      "code": "compute_unavailable",
      "message": "Compute is temporarily unavailable"
    }
  ]
}

Treat the frame as a failed stream. The TypeScript SDK turns it into SandboxError.

A binary file download cannot append a JSON frame after HTTP 200. Verify that the body length matches Content-Length; a short body is a failed download.