Tests & Steps

Structure test cases with sequential step groups and parallel steps.

Test structure

Tests are defined in the top-level tests array. Each test is independent and contains step groups.

{
  "name": "my-test-suite",
  "items": [ ... ],
  "tests": [
    {
      "name": "Create and fetch a user",
      "variables": { "email": "test@example.com" },
      "steps": [
        [{ "name": "Step 1a", ... }, { "name": "Step 1b", ... }],
        [{ "name": "Step 2", ... }]
      ]
    }
  ]
}

Step groups: parallel and sequential

Steps are organized as an array of arrays:

Outer array = sequential groups (run one after another)
Inner arrays = parallel steps (run simultaneously within a group)

"steps": [
  // Group 1: these two steps run in parallel
  [
    { "name": "Create user A", ... },
    { "name": "Create user B", ... }
  ],
  // Group 2: runs after group 1 completes
  [
    { "name": "List all users", ... }
  ]
]

TestDefinition fields

Field	Type	Required	Default	Description
`name`	string	Yes	—	Name of the test
`description`	string	No	—	What the test verifies
`timeoutSeconds`	integer	No	—	Timeout for entire test
`stopOnFailure`	boolean	No	`true`	Stop subsequent groups when any assertion fails
`variables`	object	No	—	Test-level variables (overrides definition-level). Supports `$ref`.
`steps`	array of arrays	Yes	—	Step groups

TestStep fields

Field	Type	Required	Default	Description
`name`	string	No	—	Step name
`description`	string	No	—	Step description
`action`	object	Yes	—	Action to execute (see Actions)
`extract`	object	No	—	Extract variables from response (see Variables)
`assertions`	array	No	—	Assertion blocks (see Assertions)
`stopOnFailure`	boolean	No	`true`	Stop test when assertions in this step fail

Definition-level variables

The definition file itself can have a top-level variables field alongside name, items, and tests. These are shared across all tests in the definition. Test-level variables override definition-level variables with the same key.

{
  "name": "my-suite",
  "variables": {
    "baseEmail": "test@example.com"
  },
  "items": [ ... ],
  "tests": [
    {
      "name": "Test 1",
      "variables": {
        "baseEmail": "override@example.com"
      },
      "steps": [ ... ]
    }
  ]
}

See Variables for full details on interpolation, extraction, and scoping.

← References ($ref) Actions →