AWS Step Functions Basics

Introduction¶

Step Functions runs workflows defined as state machines in the Amazon States Language (ASL), a JSON DSL. It handles the parts of orchestration you'd otherwise hand-code: retries with backoff, error branching, parallel fan-out, timeouts, and a full visual execution history.

Standard vs Express¶

The states¶

{
  "Comment": "Process an uploaded image",
  "StartAt": "Validate",
  "States": {
    "Validate": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:...:function:validate",
      "Retry": [{ "ErrorEquals": ["Lambda.TooManyRequestsException"], "IntervalSeconds": 2, "MaxAttempts": 5, "BackoffRate": 2.0 }],
      "Catch": [{ "ErrorEquals": ["ValidationError"], "Next": "RejectUpload" }],
      "Next": "ProcessSizes"
    },
    "ProcessSizes": {
      "Type": "Map",
      "ItemsPath": "$.sizes",
      "MaxConcurrency": 4,
      "Iterator": {
        "StartAt": "Resize",
        "States": {
          "Resize": { "Type": "Task", "Resource": "arn:aws:lambda:...:function:resize", "End": true }
        }
      },
      "Next": "Notify"
    },
    "Notify": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sns:publish",
      "Parameters": { "TopicArn": "arn:aws:sns:...:uploads", "Message.$": "$.result" },
      "End": true
    },
    "RejectUpload": { "Type": "Fail", "Error": "ValidationError", "Cause": "Bad upload" }
  }
}

• Task — do work: invoke a Lambda, or call a service directly (SNS, SQS, DynamoDB, ECS RunTask, Batch, Glue, another state machine, ~200 integrations). .sync variants wait for the job (ECS task, Glue job) to finish.
• Choice — branch on the input ("Variable": "$.status", "StringEquals": "APPROVED").
• Parallel — run several branches concurrently, wait for all.
• Map — run the same sub-workflow over each item of an array (ItemsPath), with MaxConcurrency. Distributed Map handles millions of items from S3.
• Wait — pause (seconds, or until a timestamp).
• Pass / Succeed / Fail — shape data / terminal states.

Error handling¶

• Retry — per error type, with IntervalSeconds, MaxAttempts, BackoffRate, MaxDelaySeconds, JitterStrategy. This replaces hand-rolled retry loops.
• Catch — route to a recovery state on an error, with the error passed in $.error.
• ErrorEquals: ["States.ALL"] — catch-all.
• TimeoutSeconds / HeartbeatSeconds on a Task — fail if it runs too long / stops sending heartbeats (for waitForTaskToken patterns).

Human-in-the-loop / callbacks¶

The .waitForTaskToken pattern: a Task hands out a token, the workflow pauses, and something external (a human clicking "approve", another system) calls SendTaskSuccess/SendTaskFailure with the token to resume. Standard workflows only (they can wait a year).

When a state machine beats glue code¶

• The workflow has branching, retries, parallelism, and long waits — all of which you'd otherwise build and test yourself.
• You want a visual audit trail of every execution (which step failed, with what input).
• Steps are already AWS service calls — Step Functions calls them directly, no Lambda needed as glue.
• Don't use it for: a linear 2-step process (just chain the calls), tight loops with thousands of iterations at sub-second latency (state transition cost/latency — use a Lambda or Express), or complex logic better expressed in code (put that in one Lambda).

Verification and troubleshooting¶

aws stepfunctions start-execution --state-machine-arn ... --input '{"...": "..."}'
aws stepfunctions describe-execution --execution-arn ...
aws stepfunctions get-execution-history --execution-arn ... --max-items 50

The console's visual graph highlights the failed state — start there.

• States.TaskFailed with no useful cause — the Lambda threw; the error type is the exception class, the cause is the message + stack. Add a Catch for States.ALL to route failures somewhere you can inspect.
• States.Runtime / States.DataLimitExceeded — the state input/output exceeded 256KB. Pass references (an S3 key), not payloads. Use ResultPath / OutputPath to trim what flows between states.
• The role defined for the function cannot be assumed / AccessDenied — the state machine's execution role lacks permission to invoke the Lambda / call the service. Each integration needs its own IAM action.
• Map state slow / throttled — MaxConcurrency too high hitting downstream limits, or too low leaving it serial. Tune it; Distributed Map for huge item counts.
• Standard execution cost surprise — per state transition billing; a tight Choice+Wait polling loop racks up transitions. Use Wait with a longer interval, or Express, or waitForTaskToken (event-driven, no polling).
• Express execution "succeeded" but did nothing — at-least-once semantics + a non-idempotent step that was retried, or async invocation where you didn't check the result. Check CloudWatch Logs (Express has no console history).
• Can't tell what a past execution did (Express) — enable full logging (ALL level, include execution data) on the state machine; Express keeps nothing by default.

Related tools and reading¶

• On-site: JSON formatter.
• Related posts: SQS vs SNS vs EventBridge, Lambda cold starts and how to cut them.