Callman — Zero to Hero#

A complete user guide to Callman: from sending your first HTTP request to running scheduled, multi-system validation pipelines from CI.

This document covers only what ships in the current Callman build. If something is not on these pages, it is not in the product yet.

Table of contents#

1. Introduction
2. Quick start: ten minutes from install to passing test
3. Core concepts
4. Request builder reference
5. Variables, environments, and dynamic data
6. Scripting: the pm.* API
7. Assertions
8. Database testing — Oracle, MongoDB, PostgreSQL, MySQL
9. Redis testing
10. Kafka testing
11. Scenario builder — node by node
12. Scenario data flow
13. Real-world workflows
14. Data-driven runs
15. Callme CLI
16. Scheduling
17. Reports and analytics
18. Tools
19. Workspace collaboration
20. Best practices
21. Mocks
22. Performance testing
23. Pipeline gating & scenario priority
24. Query cookbook — every connection type, every syntax
25. Appendix A — pm.* API index
26. Appendix B — Glossary

1. Introduction#

1.1 What Callman is#

Callman is a desktop application, a CLI, and a backend service that together let you build, run, and schedule HTTP-driven test pipelines against systems that involve more than just HTTP — databases, Kafka topics, and Redis caches.

A request in Callman is not just an HTTP call. It is an HTTP call plus the database state checks that confirm the call had the right side-effects, plus the Kafka events the call was supposed to emit, plus the Redis keys it was supposed to populate, plus the contract the response payload must satisfy. A scenario is a directed graph of those requests with branching, retries, and notifications. A schedule turns a scenario into a recurring server-side job. The CLI lets the same scenario run from a build pipeline.

You can use Callman as a plain request client and never touch the rest. You can also drive an entire post-deploy verification pipeline through it without writing a separate test framework.

1.2 Why it exists#

Most API clients treat the request as the unit of work. That maps well to public REST APIs and stops mapping the moment your service does anything asynchronous. A "POST /orders" call returns 202 and then the truth lives in three places: the orders table, a Kafka orders.events.v1 topic, and a Redis pending-payment hash. Verifying that flow with a generic API client requires three separate tools and a script that glues them together.

Callman folds all four checks into one request. The HTTP call, the database row assertion, the Kafka event match, the Redis state read — all live in the same request tab, all run together, all show up in the same test report. When the flow grows past a single request, the scenario builder turns it into a graph without leaving the app.

1.3 Positioning#

Callman sits in a niche that the generic-API-client category leaves empty: it treats the full side-effect surface of a request — HTTP response, database write, message-broker event, cache mutation — as one atomic test, then composes those atomic tests into directed scenario graphs with retries, branching, scheduling, and CI hooks.

Callman is not trying to be a load tester (use k6 or Gatling). It is not a contract-first API designer (use Stoplight or OpenAPI tooling). It is a workflow validation tool for systems that already exist — specifically the systems that fail in subtle ways because the HTTP response was fine and the database row never landed.

What you give up by picking Callman: a public marketplace of shared collections. What you get: assertions across HTTP + DB + Kafka + Redis in one tab; a visual scenario builder; a server-side scheduler; a CLI that mirrors the same execution engine; and a Slack-wired notification path that closes the loop on a failure.

1.4 Who it's for#

• Backend engineers writing services that touch databases and message brokers, who need to verify a deploy actually works.
• QA engineers building regression suites that go beyond request/response and into "did the side-effects happen."
• DevOps and SRE teams wiring smoke tests into the deploy pipeline and scheduling overnight health checks.
• Platform and integration teams validating event-driven contracts between services in a microservice graph.
• Automation engineers replacing bespoke verification scripts with declarative scenarios that have UI, history, and reports.

1.5 Capability matrix#

1.6 What you can do after reading this guide#

• Build a request that calls an HTTP endpoint, verifies a row landed in MongoDB, and confirms a Kafka event was published — in one tab.
• Write a pre-request script that refreshes an OAuth2 token before every call without modifying every request.
• Compose a scenario graph that handles a payment failure with a compensating request and a Slack alert.
• Run that scenario nightly at 02:00 in your timezone, with retries, and have the JSON report archived to your CI artifact store.
• Drive the same scenario from GitHub Actions on every pull request.

1.7 Moving from Postman? One click and you're in#

If your team is on Postman today, you do not need to rebuild anything. Callman imports Postman collections and Postman environments in their original native format — no schema conversion, no script rewriting, no manual mapping.

• Postman collections — sidebar → Import → pick the exported Postman Collection v2.1 JSON. Folders, request order, request bodies, headers, query parameters, auth, pre-request scripts, and tests scripts all come across exactly as they were authored. Variable references like {{baseUrl}} keep their original syntax.
• Postman environments — Manage Environments drawer → Import. Each variable, its initial value, and (where the export carries it) its secret flag all round-trip.
• What you don't have to rewrite: every pm.test, pm.expect, pm.environment.set, pm.sendRequest, pm.response.to.have.status, HMAC-SHA256 signing with CryptoJS, _ lodash transforms, and the full {{$randomUUID}} / {{$timestamp}} / {{$random*}} token family all run unmodified — Callman's script sandbox accepts them as-is. See §5.4.1 for the full token list and §6 for the pm.* surface.

In practical terms: export your Postman collection, open Callman, click Import, pick the file, and your tests run on the next Send. The same is true for the CLI — callme run collection.json --env env.json accepts the Postman export directly.

Where to next: §2 Quick start.

2. Quick start: ten minutes from install to passing test#

This walkthrough takes you from a clean install to a green test in ten minutes. No previous Callman experience required.

2.1 Install the desktop app#

Download the build for your platform from the Callman release page. The desktop app ships as a notarized DMG (macOS), an MSI installer (Windows), and an AppImage (Linux). Open the installer and launch Callman.

On first launch the app prompts you to sign in or create an account. Sign in if you have one. Otherwise create one — the app will create a personal workspace for you automatically.

2.2 Send your first request#

1. Click the + button at the top of the sidebar and choose Create Collection. Name it Smoke tests.
2. With the new collection selected, click the + beside its name and choose Create Request. Name it Get post.
3. In the URL bar set the method to GET and the URL to https://jsonplaceholder.typicode.com/posts/1.
4. Press Send (or Cmd+Enter/Ctrl+Enter).

The response panel opens with a 200 status, the JSON body, the response headers, and the duration in milliseconds.

2.3 Write your first test#

Switch to the Tests tab in the request editor and paste:

pm.test("response is 200", () => {
  pm.expect(pm.response.status).toBe(200);
});

pm.test("post has a title", () => {
  const body = pm.response.json();
  pm.expect(body.title).to.be.a("string");
  pm.expect(body.title.length).to.be.greaterThan(0);
});

Send again. The Tests tab in the response panel now shows two green checkmarks. If you change toBe(200) to toBe(201), the first test goes red and the failure message names the actual vs expected status.

2.4 Capture a value for the next request#

Add to the Tests script:

pm.environment.set("lastPostId", pm.response.json().id);

The value is now stored in the active environment and reusable as {{lastPostId}} in any other request's URL, headers, query, or body. You can confirm by opening the environment drawer with Cmd+E.

2.5 Run the collection from the CLI#

The same collection works from the command line. Export it via the sidebar (right-click the collection → Export) into a file named smoke.json, then:

npm install -g @callman/cli
callme run smoke.json --report json --output smoke-report.json

The CLI prints a per-request line, a green summary, and writes the machine-readable report. Exit code is 0 on success, 1 on failures, 2 on fatal/invalid input. That is enough for CI to gate a deploy.

Where to next: §3 Core concepts.

3. Core concepts#

Callman has a small vocabulary. Internalising it now will save you re-reading later sections.

3.1 The eight nouns#

Workspace
   └── Environments      (named variable sets — dev / staging / prod)
   └── Collections       (folders of related requests)
        └── Folders
             └── Requests
   └── Scenarios         (graphs that orchestrate requests + integrations)
        └── Schedules    (cron-style triggers for scenarios)
   └── Mocks             (short-lived fake HTTP APIs — see §21)
   └── Runs              (a single execution of a collection or scenario)
        └── Reports      (JSON/HTML output of a run)

• Workspace — the unit of collaboration. Members and shared environments live at this level. You can be in more than one.
• Environment — a key-value bag of variables (typically per deploy target). The active environment fills in {{...}} tokens.
• Collection — a folder of requests. Collections are the unit of import/export and CLI file-mode execution.
• Folder — a sub-organiser inside a collection. Purely hierarchical; behaves the same as a collection for runs.
• Request — the canonical unit of work. Holds the HTTP call plus scripts, contract rules, DB / Kafka / Redis hooks.
• Scenario — a visual graph of typed nodes (requests, DB, Kafka, conditions, notifications, etc.) with edges and retry policies.
• Mock — a short-lived fake HTTP API (/mock/<id>/...), owned by the workspace and auto-expiring on its TTL. Useful when the real service is not yet built or you want to inject failure modes. Full reference: §21.
• Run — one execution of a collection, folder, or scenario. Each run produces a report. Reports are stored server-side for scenarios; collection runs stay local unless exported.

3.2 Where things execute#

This split matters: scripts in scenario nodes run on the server, so they cannot touch your laptop's filesystem. Local desktop runs can do more (open files, interact with the Electron host) but only on the machine that has the request open.

3.3 Variable precedence#

When the engine encounters {{userId}} it resolves it in this order, stopping at the first match:

1. Per-iteration row column (Data-Driven Runner)
2. Active environment (pm.environment)
3. Workspace globals (pm.globals)
4. Built-in dynamic tokens (random.*, counter)
5. Unresolved → left as {{userId}} literally (so you notice)

A {{...}} that fails to resolve does not stop the request — it is sent literally so you can spot the missing variable in the trace.

Where to next: §4 Request builder reference.

4. Request builder reference#

Every Callman request has the same set of tabs. This section walks through each one with the cases it solves.

4.1 The URL bar#

The URL bar accepts a method (left dropdown) and a URL with {{var}} interpolation. Three kinds of substitution work in the URL:

• Environment variables: https://{{host}}/orders
• Path tokens: /orders/{{orderId}}/refunds — orderId resolves like any variable.
• Dynamic tokens: /sessions/{{random.uuid}} — a fresh UUID per send.

Pasting a curl command into the URL bar auto-populates the entire request: method, URL, headers, and body. Useful when teammates share reproductions in chat.

Copying the URL bar with Cmd+C while the focus is in it copies the fully-resolved URL (every {{...}} is replaced). Copying with the sidebar focused (after a right-click → "Copy as curl") copies a curl command instead.

4.2 Params tab#

The Params tab is a key/value editor for the query string. Each row has a checkbox to enable or disable it without deleting it — useful when you want to keep a "debug=1" toggle around but off by default.

Path variables also surface here. If your URL is /orders/{{orderId}}/items/{{itemId}}, the Params tab shows orderId and itemId rows you can set without editing the URL string. Query and path values both go through {{...}} interpolation.

4.3 Headers tab#

Standard key/value editor. Two extra capabilities:

• Derived Headers panel at the bottom shows what your auth tab, body type, and scripts will actually add to the outgoing request. This is the source of truth — if Authorization is missing here, it will be missing on the wire.
• Re-ordering does not matter (HTTP headers are unordered).

Common-mistake callout: setting Content-Type manually when the Body tab also implies one. The Body tab wins. If you need to override (say, send application/vnd.acme.v2+json), do it from the Headers tab and switch the body to raw.

4.4 Auth tab#

Three first-class auth modes ship today:

• Bearer token — fills Authorization: Bearer <value>. The value field interpolates {{...}}, so Bearer {{accessToken}} is the normal pattern.
• Basic auth — username + password fields. The engine base64-encodes the pair.
• API key — pick a name (e.g. X-API-Key), a value, and whether to inject as a header or as a query parameter.

OAuth2 with refresh, AWS Signature, NTLM, and Digest are not first-class. The pattern for OAuth2 is a pre-request script that checks expiry and uses pm.sendRequest to refresh — see §6.10 pm.sendRequest for the full pattern.

4.5 Body tab#

Supported modes:

• none — no body. Forces no Content-Type.
• raw — free-form text. Sub-modes for JSON, XML, plain text. The editor switches highlighting accordingly. Auto-beautify on paste.
• form-data — multipart. Each row can be a text field or a file upload (click the row type to switch). File uploads send the actual file from disk at send time.
• x-www-form-urlencoded — key/value pairs URL-encoded into the body. Used by older form APIs.

For GraphQL, use raw / JSON and send {"query": "...", "variables": {...}}. Callman does not ship a separate GraphQL mode — it doesn't need one.

4.6 Pre-request script tab#

A Monaco editor (full IntelliSense for the pm.* API) that runs before the request is sent. Three typical jobs:

• Mint or refresh tokens.
• Derive a value from another value (pm.environment.set("hmac", computeHmac(body))).
• Validate inputs before sending, failing fast with pm.test.

The script has access to pm.request (mutable) and the full variable namespace. You can change the URL, headers, and body before they go on the wire.

Example: HMAC-signed request.

const body = pm.request.body || "";
const secret = pm.environment.get("apiSecret");

// Compute HMAC-SHA256, base64-encode, set header.
const enc = new TextEncoder();
const keyBytes = enc.encode(secret);
const msgBytes = enc.encode(body);
const cryptoKey = await crypto.subtle.importKey(
  "raw", keyBytes,
  { name: "HMAC", hash: "SHA-256" },
  false, ["sign"],
);
const sig = await crypto.subtle.sign("HMAC", cryptoKey, msgBytes);
const sigB64 = btoa(String.fromCharCode(...new Uint8Array(sig)));

pm.request.setHeader("X-Signature", sigB64);

4.7 Tests tab#

The Tests tab runs after the response arrives. Use it for everything the response itself proves — status, body shape, business invariants.

pm.test("order created", () => {
  pm.expect(pm.response.status).toBe(201);
  const body = pm.response.json();
  pm.expect(body.id).to.match(/^ord_[0-9a-f]{16}$/);
  pm.expect(body.total).to.equal(99.99);
});

Tests can be async — pm.sendRequest and pm.db.query both return promises, and pm.test accepts an async callback.

4.8 Kafka tab#

Configure a Kafka connection (broker list, security, group id), the topic to listen on, the match mode (regex or rules), and the timeout window. When the request runs, the listener subscribes before the HTTP call goes out, captures messages during the window, and the matched event becomes available as pm.kafka.event() inside the Tests script.

See §10 Kafka testing for the full mechanics.

4.9 DB Assertions tab#

Pick a saved database connection (Oracle, MongoDB, PostgreSQL, or MySQL), choose Pre-request or Post-response stage, write the query, and define declarative rules against the returned rows. Rules support equals, not_equals, exists, gt, lt. Field expressions use dot notation: db_post.firstRow.status, db_post.rowCount, db_post.rows.0.amount.

For ad-hoc queries from inside a test script, use pm.db.query(...) — see §8 Database testing.

4.10 Redis tab#

Like the DB tab: pick a saved Redis connection, choose pre or post stage, write commands (one per line, e.g. GET cart:{{userId}}), and add assertions against the captured results. Use pm.redis.get(key) inside scripts for ad-hoc reads.

4.11 Contract Validation tab#

A declarative JSON-schema-style checker. You point at a path in the response body and pick rules: required, type, notEmpty, min/max (numbers), minLength/maxLength (strings), minItems/maxItems (arrays), pattern (regex string), enum, contains.

path: data.orders[*].id
rules: required, type=string, pattern=^ord_[0-9a-f]{16}$

Path syntax uses [*] for "every item in this array." When a rule fails, the report names the path that failed and the actual value.

Use Contract rules for schema-shape checks (the field exists, the type matches, the format is right) and the Tests script for business-logic invariants (the total equals quantity × price, the status transitioned from "pending" to "paid").

Where to next: §5 Variables and environments.

5. Variables, environments, and dynamic data#

This is the chapter that turns Callman from a request tool into an automation tool. Everything chainable, everything dynamic, lives here.

5.1 Three scopes#

• Environment — variables tied to a named environment. Each workspace has at least one (typically dev, staging, prod). Switch with Cmd+E.
• Globals — variables visible from every environment. Use for truly cross-cutting values (a team-wide test user, a vendor key).
• Per-iteration (Data-Driven Runner only) — each row of your CSV/JSON/XLSX file becomes a temporary variable map for that iteration. See §14 Data-driven runs.

Precedence: per-iteration > environment > globals > dynamic > literal.

5.2 Interpolation syntax#

Three forms are accepted everywhere {{...}} works (URL, headers, query, body, scripts via pm.variables.resolve):

Resolution runs up to four passes, so nested templates work: {{env.{{counter}}}} will first resolve {{counter}} to (say) 7, then look up the env variable named 7.

5.3 Setting and reading from scripts#

pm.environment.set("orderId", pm.response.json().id);
pm.environment.get("orderId");        // "ord_abc..."
pm.environment.has("orderId");        // true
pm.environment.unset("orderId");

pm.globals.set("vendorKey", "xxx");   // same shape on globals

A set from a pre-request script is visible to that request and every subsequent request in the run. A set from a Tests script is visible to the next request. Setting writes go to the active environment — they do not silently leak across environments.

5.4 Built-in dynamic tokens#

These do not need to be defined anywhere. They resolve at send time.

Defaults: random.int defaults to [0, 999_999], random.string to length 12, random.date to YYYY-MM-DD, random.bigdecimal to scale 6 over [0, 999_999_999].

5.4.1 $xxx dynamic tokens (60+ aliases)#

Alongside the random.* syntax above, every {{$xxx}} token below also resolves at send time. These are dollar-prefixed identifiers (no parentheses, no arguments) and are convenient when a script or collection imported from a different tool already uses the form.

IDs / time:

People / identity:

Internet:

Phone / Location:

Business:

Finance:

Date / time:

Colors / lorem:

Files / images:

Autocomplete in the URL bar, headers, body, and Monaco script editors opens the moment you type {{$.

5.5 Request chaining#

The two-request "create then fetch" pattern is the simplest example.

Request A (Tests script):

pm.test("order created", () => {
  pm.expect(pm.response.status).toBe(201);
});
pm.environment.set("orderId", pm.response.json().id);

Request B (URL):

GET https://{{host}}/orders/{{orderId}}

Request B inherits the value as long as it runs after A in the same collection run (or in the same scenario branch).

5.6 Extracting deep values#

pm.response.json() returns a parsed JSON object. Walk it like any JS object. For deep extraction, write a small helper inline:

const body = pm.response.json();
const firstItemId = body.data.items?.[0]?.id;
pm.expect(firstItemId).to.be.a("string");
pm.environment.set("itemId", firstItemId);

5.7 When to use which scope#

Where to next: §6 Scripting.

6. Scripting: the pm.* API#

Scripts run in a sandbox. They have access to the pm namespace and to the JavaScript runtime's standard globals (Date, JSON, Math, crypto.subtle, crypto.randomUUID, TextEncoder, TextDecoder, console, setTimeout / clearTimeout / setInterval / clearInterval, Promise, URL, URLSearchParams, atob, btoa, typed arrays). They do not have access to require, process, the filesystem, fetch, or arbitrary npm packages.

Two vendor libraries are bundled and bound at the top level — no import needed:

• CryptoJS — the full crypto-js library: CryptoJS.HmacSHA256, CryptoJS.SHA256/SHA1/SHA384/SHA512/MD5/RIPEMD160, CryptoJS.AES, CryptoJS.DES, CryptoJS.TripleDES, CryptoJS.PBKDF2, CryptoJS.enc.Utf8/Base64/Hex/Latin1, CryptoJS.lib.WordArray.
• _ (lodash) — _.filter, _.map, _.groupBy, _.keyBy, _.pick, _.omit, _.get, _.set, _.has, _.cloneDeep, _.merge, _.uniq, _.flatten, _.chunk, _.sortBy, _.orderBy, _.isEqual, _.debounce, _.throttle, and the full lodash surface.

setTimeout and friends are throttled (max 60 s per timer) and any timer still pending when the script's promise settles is cancelled automatically — a stray setInterval cannot outlive the run.

This chapter covers every pm.* namespace. Use Appendix A for a flat alphabetical reference.

6.1 pm.request#

Read and mutate the outgoing request. All mutations from a pre-request script take effect; mutations from a Tests script affect nothing on the wire (the response is already back) but are visible to later assertions.

pm.request.method;                     // "POST"
pm.request.url;                        // resolved URL string
pm.request.headers;                    // HeaderList (see below)
pm.request.query;                      // HeaderList of query params
pm.request.body;                       // { mode, raw, formEntries }

pm.request.setMethod("PATCH");
pm.request.setUrl("https://api.example.com/v2/orders");
pm.request.setHeader("X-Trace-Id", crypto.randomUUID());
pm.request.unsetHeader("X-Trace-Id");
pm.request.getHeader("Authorization");
pm.request.setQueryParam("expand", "items");
pm.request.unsetQueryParam("expand");
pm.request.getQueryParam("expand");
pm.request.setRawBody(JSON.stringify({ ok: true }));

HeaderList API#

pm.request.headers and pm.request.query are both HeaderList instances. They behave as a case-sensitive record for bracket access AND expose case-insensitive helpers:

// Case-sensitive bracket access (record-like)
pm.request.headers["X-Custom"] = "value";
const v = pm.request.headers["X-Custom"];

// Case-insensitive lookup
pm.request.headers.get("content-type");      // "application/json"
pm.request.headers.has("Authorization");     // true

// Mutation helpers
pm.request.headers.add({ key: "X-Run", value: "smoke" });    // append
pm.request.headers.upsert({ key: "X-Trace", value: crypto.randomUUID() });
pm.request.headers.remove("X-Old-Header");

// Inspection
pm.request.headers.idx(0);                   // first entry { key, value, description? }
pm.request.headers.all();                    // [{ key, value, description? }, ...]

// Iterable
for (const h of pm.request.headers) {
  console.log(h.key, "=", h.value);
}

6.2 pm.response#

Read the response. Available in Tests scripts and any callback that runs after send.

pm.response.status;              // 200
pm.response.code;                // 200 (alias)
pm.response.statusText;          // "OK"
pm.response.headers;             // HeaderList (read-only)
pm.response.body;                // parsed body when JSON, else raw string
pm.response.data;                // alias of .body
pm.response.rawBody;             // raw string
pm.response.json();              // parse on demand, throws on non-JSON
pm.response.text();              // raw string
pm.response.responseTime;        // ms (alias of duration)
pm.response.duration;            // ms
pm.response.responseSize;        // body bytes
pm.response.contentType;         // "application/json; charset=utf-8"

pm.response.headers is a read-only HeaderList:

pm.response.headers.get("content-type");     // case-insensitive
pm.response.headers.has("set-cookie");
for (const h of pm.response.headers) {
  // { key, value, description? }
}

6.2.1 Response assertion shortcuts#

pm.response.to, pm.response.be, and pm.response.have all point at a response-aware assertion chain. The same chain is reachable via each entry point — read whichever phrasing reads best at the call site.

Status range (HTTP class):

pm.response.to.be.success;          // 2xx
pm.response.to.be.ok;               // 2xx (alias)
pm.response.to.be.info;             // 1xx
pm.response.to.be.redirection;      // 3xx
pm.response.to.be.clientError;      // 4xx
pm.response.to.be.serverError;      // 5xx
pm.response.to.be.error;            // 4xx OR 5xx

Status exact:

pm.response.to.be.notFound;         // 404
pm.response.to.be.unauthorized;     // 401
pm.response.to.be.forbidden;        // 403
pm.response.to.be.badRequest;       // 400
pm.response.to.be.rateLimited;      // 429
pm.response.to.be.accepted;         // 202
pm.response.to.have.status(201);    // exact, any code

Content type (read from Content-Type header):

pm.response.to.be.json;             // application/json or +json
pm.response.to.be.xml;              // application/xml / text/xml / +xml
pm.response.to.be.html;             // text/html

Headers:

pm.response.to.have.header("X-Trace-Id");
pm.response.to.have.header("Content-Type", "application/json; charset=utf-8");

Body (raw + JSON dot-path):

pm.response.to.have.body();                       // body is non-empty
pm.response.to.have.body("hello");                // substring match
pm.response.to.have.jsonBody();                   // is valid JSON
pm.response.to.have.jsonBody("data.id");          // path exists
pm.response.to.have.jsonBody("data.status", "PAID"); // path equals

Negation: every predicate above accepts .not:

pm.response.to.not.be.serverError;
pm.response.to.not.have.header("X-Internal");

6.3 pm.environment (alias pm.env)#

Read and mutate the active environment. pm.env is provided as a short alias.

pm.environment.get("host");
pm.environment.set("token", "abc");
pm.environment.unset("token");
pm.environment.has("token");
pm.environment.toObject();       // snapshot as plain object
pm.environment.values;            // same as toObject()
pm.environment.name;             // "Staging" — read-only label
pm.environment.clear();          // wipe every key in this scope
pm.environment.replaceIn("https://{{host}}/orders/{{id}}");
                                 // resolve {{...}} against THIS scope only

A common mistake: setting an env variable from a Tests script and expecting the currently-running request to see it. It can't — the URL was already resolved by the time the Tests script runs. The next request gets it.

6.4 pm.globals#

Same shape as pm.environment but writes to workspace globals. Use for cross-environment state (a shared correlation id during one run). Exposes .has/.get/.set/.unset/.has/.clear/.toObject/.replaceIn and the read-only .name label.

6.5 pm.variables#

Read / write / resolve against the merged scope (per-iteration row > collection > environment > globals).

pm.variables.has("traceId");                 // boolean across scopes
pm.variables.get("traceId");                 // first match
pm.variables.set("traceId", crypto.randomUUID());
                                             // request-local write
pm.variables.unset("traceId");
pm.variables.toObject();                     // merged snapshot

pm.variables.resolve("https://{{host}}/orders/{{orderId}}");
pm.variables.replaceIn("Hello {{userName}}"); // alias of resolve

Writes via pm.variables.set are request-scoped — they survive across scripts in the same request (pre-request → Tests) but do not persist to the environment / globals. Use pm.environment.set or pm.collectionVariables.set if you need persistence.

6.5.1 pm.collectionVariables#

A variable scope that survives across requests within a single collection run but does not persist to the workspace. Same shape as pm.environment:

pm.collectionVariables.has("runId");
pm.collectionVariables.get("runId");
pm.collectionVariables.set("runId", crypto.randomUUID());
pm.collectionVariables.unset("runId");
pm.collectionVariables.clear();
pm.collectionVariables.toObject();
pm.collectionVariables.replaceIn("trace={{runId}}");
pm.collectionVariables.name;                 // "Collection"

Use case: write a correlation id in the first request's pre-request script, read it from every later request in the same run without polluting the environment.

6.5.2 pm.info#

Read-only snapshot of "where am I" in the run. Driven by the runner / scenario worker context.

pm.info.eventName;       // "prerequest" | "test" | "scenario-script" | ...
pm.info.iteration;       // 1-based row index in Data-Driven Run; 1 outside
pm.info.iterationCount;  // total iterations; 1 outside DDR
pm.info.requestName;     // "Create order"
pm.info.requestId;       // internal id

6.5.3 pm.execution#

Flow-control surface used by the collection runner and scenario worker. No-op on an ad-hoc single Send.

pm.execution.setNextRequest("Retry");        // jump to a named request
pm.execution.setNextRequest(null);           // stop the branch

// Pre-request only. Tells the runner to skip the HTTP call AND the
// Tests script for the current request.
pm.execution.skipRequest();

// Hierarchical location:
pm.execution.location;                       // ["Smoke", "Auth", "Login"]
pm.execution.location.current;               // "Login"

6.6 pm.test#

Define a named test. Sync or async.

pm.test("status is 200", () => {
  pm.expect(pm.response.status).toBe(200);
});

pm.test("user is present in DB", async () => {
  const row = await pm.db.query({
    connectionId: "primary",
    query: `SELECT id FROM users WHERE email = '${pm.environment.get("email")}'`,
  });
  pm.expect(row.rowCount).toBe(1);
});

A test that throws is recorded as a failure with the error message. A test that completes without throwing is a pass. There is no "skipped" state inside pm.test itself — to skip, simply don't call it.

6.7 pm.expect#

A Chai-flavoured assertion chain. Both BDD-style getters and toBe/toEqual-style methods are accepted.

Type and existence:

pm.expect(value).to.be.true;
pm.expect(value).to.be.false;
pm.expect(value).to.exist;
pm.expect(value).toBeDefined();
pm.expect(value).to.be.a("string");
pm.expect(value).to.be.an("array");

Equality:

pm.expect(7).to.equal(7);
pm.expect(7).toBe(7);
pm.expect({ a: 1 }).to.eql({ a: 1 });   // deep
pm.expect({ a: 1 }).toEqual({ a: 1 });  // deep

Containment and comparison:

pm.expect("hello world").to.contain("hello");
pm.expect([1, 2, 3]).to.include(2);
pm.expect(10).to.be.greaterThan(5);
pm.expect(10).toBeGreaterThan(5);
pm.expect(10).to.be.lessThan(20);
pm.expect(10).toBeLessThan(20);

Truthiness, patterns, navigation:

pm.expect("nonempty").toBeTruthy();
pm.expect("").toBeFalsy();
pm.expect("abc-123").to.match(/^[a-z]+-\d+$/);
pm.expect(body).to.have.property("id");
pm.expect(body).to.have.property("status", "PAID");
pm.response.to.have.header("Content-Type");
pm.response.to.have.status(200);

Fail explicitly:

pm.expect.fail("Unexpected branch reached");

The chain getters to, be, been, is, that, which, and, has, have, with, same, deep are no-ops for fluency — pick the form that reads best.

6.8 pm.random#

Same generators as the {{random.*}} tokens, callable from script. Useful when you need a value twice in one script (once in a request body, once in an assertion).

pm.random.uuid();
pm.random.int(1000, 9999);
pm.random.string(20);
pm.random.email();
pm.random.phone();
pm.random.name();
pm.random.date("YYYY-MM-DD HH:mm");
pm.random.boolean();
pm.random.bigdecimal(0, 1000, 2);

6.9 pm.counter#

A monotonically increasing integer scoped to the run. Useful for ordering, idempotency keys, or batch numbering.

pm.counter.next();        // advance the default counter, return new value
pm.counter.start(100);    // advance a separate counter keyed by start value,
                          // returning 100, 101, 102, ... on subsequent calls

Each unique start argument is its own counter, so pm.counter.start(100) and pm.counter.start(500) are independent sequences. The default counter (pm.counter.next()) is the same one the {{counter}} template token reads.

6.10 pm.sendRequest#

Issue an HTTP request from inside a script. Three call signatures are accepted so existing snippets port over without rewriting: URL + callback, options + callback, or options as a Promise.

// Form 1: URL + callback
pm.sendRequest("https://api.example.com/health", (err, res) => {
  if (err) return console.error(err);
  pm.expect(res.code).toBe(200);
});

// Form 2: options + callback
pm.sendRequest(
  {
    url: "https://auth.example.com/oauth/token",
    method: "POST",
    header: { "Content-Type": "application/x-www-form-urlencoded" },
    body: {
      mode: "urlencoded",
      urlencoded: [
        { key: "grant_type", value: "client_credentials" },
        { key: "client_id", value: pm.environment.get("clientId") },
        { key: "client_secret", value: pm.environment.get("clientSecret") },
      ],
    },
  },
  (err, res) => {
    if (err) throw err;
    const json = JSON.parse(res.body);
    pm.environment.set("accessToken", json.access_token);
    pm.environment.set("tokenExpiresAt", String(Date.now() + json.expires_in * 1000));
  },
);

// Form 3: Promise (no callback)
const res = await pm.sendRequest({ url: "https://api.example.com/me", method: "GET" });
const me = JSON.parse(res.body);

The token-refresh pattern lives here. Put it in a pre-request script shared by every authenticated request (collection-level pre-request scripts apply to every request inside).

const expiresAt = Number(pm.environment.get("tokenExpiresAt") || 0);
if (Date.now() > expiresAt - 30_000) {
  await pm.sendRequest(/* ... refresh ... */);
}
pm.request.setHeader("Authorization", `Bearer ${pm.environment.get("accessToken")}`);

The response object exposes code, status, headers, body, json(), text(), responseTime.

6.11 pm.kafka#

Available inside the Tests script of a request that has the Kafka tab configured (or inside a Kafka scenario node). The listener is opened before the HTTP call goes out and stays open for the configured timeout window.

pm.kafka.event();          // matched message (parsed JSON) or null
pm.kafka.messages();       // every captured message, raw + parsed

pm.test("OrderCreated was published", () => {
  const evt = pm.kafka.event();
  pm.expect(evt).to.exist;
  pm.expect(evt.orderId).to.equal(pm.environment.get("orderId"));
});

If the listener times out without a match, pm.kafka.event() returns null. The test fails with a clear "no matching Kafka event" error.

6.12 pm.redis#

Available when the Redis tab is configured on the request (or in a Redis scenario node).

pm.redis.get("cart:{{userId}}");     // synchronous read of last cached value
pm.redis.results.pre;                // array of pre-stage command results
pm.redis.results.post;               // array of post-stage command results

For ad-hoc commands not in the declarative list, use a scenario Redis node or write the command into the pre/post-stage area.

6.13 pm.db#

Same shape as Redis. Pre and post results come from the rules defined in the DB Assertions tab; pm.db.query is the ad-hoc escape hatch.

pm.db.pre;                 // { rows, rowCount, firstRow(), ... }
pm.db.post;                // same shape after the HTTP call
const r = await pm.db.query({
  connectionId: "ordersPrimary",
  query: "SELECT status, total FROM orders WHERE id = ?",
});
pm.expect(r.rowCount).toBe(1);
pm.expect(r.firstRow().status).to.equal("PAID");

Notice pm.db.query is async (returns a Promise). The Tests script can await it directly.

6.14 pm.node#

Only meaningful in a scenario. Use to read the current node's label and to redirect flow.

pm.node.nodeLabel;                  // "verify-payment"
pm.node.setNextRequest("retry");    // jump to another node by name
pm.node.setNextRequest(null);       // stop the current branch

pm.execution.setNextRequest(...) (see §6.5.3) is the preferred entry point going forward; pm.node.setNextRequest remains for back-compatibility with older scripts.

6.15 pm.log#

Append a line to the request console. Visible in the Console drawer of the desktop app and in CLI --verbose output.

pm.log("computed signature", sigB64);
pm.log({ orderId: pm.environment.get("orderId"), retry: 3 });

6.16 CryptoJS (top-level global, not under pm.*)#

CryptoJS is bound as a top-level identifier — no import needed. Useful for request signing, hashing, encryption.

// HMAC-SHA256 signed request (pre-request script)
const body = pm.request.body?.raw ?? "";
const secret = pm.environment.get("apiSecret");
const signature = CryptoJS.HmacSHA256(body, secret).toString();   // hex
pm.request.headers.upsert({ key: "X-Signature", value: signature });

// Base64 wrap for binary signatures
const b64 = CryptoJS.enc.Base64.stringify(
  CryptoJS.HmacSHA256(body, secret),
);

// SHA hashes
CryptoJS.SHA256("hello").toString();          // hex
CryptoJS.MD5("hello").toString();
CryptoJS.SHA512("hello").toString();

// AES round-trip
const cipher = CryptoJS.AES.encrypt("secret payload", "passphrase").toString();
const plain = CryptoJS.AES.decrypt(cipher, "passphrase").toString(CryptoJS.enc.Utf8);

// PBKDF2
const key = CryptoJS.PBKDF2("password", "salt", { keySize: 8, iterations: 1000 });

// Encoders
CryptoJS.enc.Hex.stringify(CryptoJS.enc.Utf8.parse("hello")); // "68656c6c6f"
CryptoJS.enc.Base64.parse("aGVsbG8=");                        // WordArray

The full crypto-js surface ships with the sandbox — every hash, HMAC variant, symmetric cipher, encoder, and lib.WordArray / lib.CipherParams helper from the upstream library is callable.

6.17 _ (lodash, top-level global)#

Lodash is bound as the global identifier _:

const body = pm.response.json();

// Filter / map / group response collections
const active = _.filter(body.users, { status: "active" });
const byRegion = _.groupBy(active, "region");
const totals = _.map(byRegion, (rows, region) => ({
  region,
  count: rows.length,
}));

// Deep extraction with a default
const orderId = _.get(body, "data.order.id", null);

// Schema shape: assert nothing extra leaked into the payload
const expected = ["id", "email", "createdAt"];
pm.test("no extra user fields", () => {
  pm.expect(_.isEqual(_.keys(body.user).sort(), expected.sort())).to.be.true;
});

// Cloning before mutating
const draft = _.cloneDeep(body);
draft.user.email = "test@example.com";

Common entries with autocomplete in the editor: _.filter, _.map, _.forEach, _.find, _.groupBy, _.keyBy, _.countBy, _.sortBy, _.orderBy, _.uniq, _.uniqBy, _.flatten, _.flattenDeep, _.chunk, _.zip, _.range, _.pick, _.omit, _.merge, _.cloneDeep, _.isEqual, _.get, _.set, _.has, _.keys, _.values, _.entries, _.isEmpty, _.isArray, _.isObject, _.shuffle, _.sample, _.debounce, _.throttle.

6.18 setTimeout / retry loops#

The sandbox exposes setTimeout, clearTimeout, setInterval, clearInterval. Use them for poll loops and async waits:

// Pre-request: refresh token if near expiry
const expiresAt = Number(pm.environment.get("tokenExpiresAt") || 0);
if (Date.now() > expiresAt - 30_000) {
  // ... refresh ...
}

// Tests: poll for eventual consistency
let attempts = 0;
let row = null;
while (attempts < 10) {
  const r = await pm.db.query({
    connectionId: "ordersPrimary",
    query: "SELECT status FROM orders WHERE id = '{{orderId}}'",
  });
  if (r.firstRow()?.status === "PAID") {
    row = r.firstRow();
    break;
  }
  await new Promise((resolve) => setTimeout(resolve, 500));
  attempts += 1;
}
pm.expect(row, "order eventually PAID").to.exist;

Two safety rails worth knowing:

• Each timer's delay is capped at 60 seconds.
• Any timer still pending when the script's promise settles is cancelled automatically. A stray setInterval(fn, 100) cannot outlive the run.

6.19 Anti-patterns#

• Putting business logic in URL strings. If a URL needs an if-then, build it in a pre-request script and setUrl.
• Setting env from Tests and expecting the same request to see it. Use a pre-request script instead.
• Re-implementing assertions with if (...) pm.expect.fail(...). Just pm.expect(value).toBe(expected) — the error message is better and the report shape is uniform.
• Long sync loops. The sandbox has a per-script timeout. Use scenario Wait nodes for delays, not while (Date.now() < ...).

Where to next: §7 Assertions.

7. Assertions#

Assertions in Callman come in two flavours: scripted (you write the check) and declarative (you describe the shape, the engine checks it). Both run as part of the same test report. Use scripted when the invariant is business logic; use declarative when the invariant is "this field exists, is a string, and matches this regex."

7.1 Scripted assertions — pm.expect#

Every operator was listed in §6.7 pm.expect. The mental model: pm.expect(actual).<chain>.<operator>(expected). The chain words are decoration; only the operator matters semantically.

Convert sloppy if-throws into clean assertions:

// Bad
if (pm.response.status !== 200) {
  throw new Error("not 200");
}

// Good
pm.expect(pm.response.status, "HTTP status").toBe(200);

The optional second argument to pm.expect (the message) shows up verbatim in the failure report.

7.2 Contract Validation — declarative response schema#

The Contract Validation tab is path-based JSON-schema. Each row:

• Path — dotted path, with [*] for "every array element." Examples: data.id, data.orders[*].id, data.orders[*].items[*].sku.
• Rules — any combination of:
  • required (true/false) — path must be present
  • type — string / number / boolean / object / array / null
  • notEmpty — string length ≥ 1 or array length ≥ 1
  • min, max — numeric bounds
  • minLength, maxLength — string length bounds
  • minItems, maxItems — array size bounds
  • pattern — regex (string form)
  • enumValues — list of allowed strings
  • contains — substring for strings

Contract rules run after the Tests script. Their failures show up as separate entries in the report — you can filter "contract failures only" in the analytics view.

When to use Contract vs Tests:

7.3 DB / Kafka / Redis declarative rules#

Each integration tab supports the same five operators: equals, not_equals, exists, gt, lt. Field expressions support dot notation and array indexing. Value expressions support {{...}} substitution, so you can assert db_post.firstRow.userId equals {{response.userId}} without writing a line of JS.

This declarative style is the right choice when:

• The check is "this field equals that field." Don't write a script.
• The checker should be visible to non-engineers (PMs reading reports).
• The same check repeats across many requests — declarative scales.

Switch to a script the moment you need anything conditional ("only check refundedAt if status is REFUNDED").

Where to next: §8 Database testing.

8. Database testing — Oracle, MongoDB, PostgreSQL, MySQL#

Four database engines are wired into Callman: Oracle, MongoDB, PostgreSQL, and MySQL. All four are accessed the same way from the UI, from pm.db.*, and from scenario DB nodes — and the query syntax is identical whether the query runs on your desktop or inside a scheduled server-side scenario run. For a per-engine syntax guide with worked examples, see §24 Query cookbook.

8.1 Creating a connection#

Sidebar → Connections → + Database. Two configuration modes:

1. Detail mode — host, port, database (Oracle service name / Mongo database), user, password.
2. Connection-string mode — paste a full URI:
   • mongodb://user:pass@host:27017/db
   • oracle://user:pass@host:1521/SERVICE
   • postgresql://user:pass@host:5432/db
   • mysql://user:pass@host:3306/db

Connections are saved per workspace. The connection ID is what you reference from scripts and scenario nodes.

Passwords are encrypted at rest (a separate workspace encryption key, not stored alongside the data). Connection definitions are pulled fresh each run — rotating a password doesn't require touching every request.

8.2 Per-request pre / post queries#

In the DB Assertions tab of a request, pick a stage:

• Pre-request — runs before the HTTP call. Useful for capturing baseline state ("how many orders does this user have").
• Post-response — runs after. Useful for the typical "did the side effect happen" check.

Write the query (parametrised with {{...}} substitutions), and define assertion rows against the returned rows.

-- Oracle
SELECT id, status, total
FROM orders
WHERE user_id = '{{userId}}'
  AND created_at > SYSTIMESTAMP - INTERVAL '5' MINUTE

{
  "collection": "orders",
  "operation": "find",
  "filter": { "userId": "{{userId}}" },
  "sort": { "createdAt": -1 },
  "limit": 10
}

MongoDB queries are not written in shell syntax (db.orders.find(...) is rejected). The query is a strict JSON object with collection and operation fields — see §24.6 for every operation and its envelope.

Sample rule rows:

8.3 pm.db.query — ad-hoc queries#

When you need a query you'd rather not declare on the tab — debug lookups, conditional checks — use the script API:

const res = await pm.db.query({
  connectionId: "ordersPrimary",
  query: `
    SELECT id, status, paid_at
    FROM orders
    WHERE user_id = '${pm.environment.get("userId")}'
    ORDER BY created_at DESC
    FETCH FIRST 1 ROWS ONLY
  `,
});

pm.test("most recent order is paid", () => {
  pm.expect(res.rowCount).toBe(1);
  pm.expect(res.firstRow().status).toBe("PAID");
  pm.expect(res.firstRow().paid_at).to.exist;
});

The returned object: { success, error, rowCount, rows, firstRow(), connectionId, connectionName, connectionType, durationMs, executedAt }.

8.4 Polling for async state#

A common asynchronous pattern: the HTTP call schedules a job, the job writes to the DB seconds later. Poll inside a Tests script:

const target = pm.environment.get("orderId");
let row = null;
for (let i = 0; i < 10; i++) {
  const r = await pm.db.query({
    connectionId: "ordersPrimary",
    query: `SELECT status FROM orders WHERE id = '${target}'`,
  });
  if (r.firstRow()?.status === "PAID") {
    row = r.firstRow();
    break;
  }
  await new Promise((r) => setTimeout(r, 500));
}

pm.test("order eventually transitioned to PAID", () => {
  pm.expect(row).to.exist;
  pm.expect(row.status).toBe("PAID");
});

For longer waits, prefer a Wait node in a scenario rather than busy- looping in a script.

8.5 What is not supported#

• MSSQL and SQLite are not wired today. The drivers aren't shipped and the UI does not expose them.
• DDL changes from scripts. The tooling assumes queries are reads or bounded writes — don't run DROP TABLE from a test.
• Transactions across multiple requests. Each query runs in its own connection acquisition.
• Multiple statements per query. One statement per query box — no semicolon-separated batches.

Where to next: §9 Redis testing.

9. Redis testing#

Redis support has the same shape as DB support: saved connections, pre/post stages per request, declarative assertions, and a script API for ad-hoc reads.

9.1 Connection#

Sidebar → Connections → + Redis. Host, port, password, database index (0 by default), TLS toggle. Saved per workspace.

9.2 Supported commands#

Exactly five commands are supported — the same five on desktop and in server-side scenario runs. Anything else is rejected with "Supported Redis commands are GET, SET, DEL, EXISTS, and TTL".

Values with spaces go in quotes: SET greeting "hello world". Single or double quotes both work, and \" escapes a quote inside a quoted value. GET/DEL/EXISTS/TTL take exactly one key — multi-key forms (DEL a b, MGET) are rejected. See §24.7 for worked examples.

9.3 Pre/post commands#

In the Redis tab, one command per line. Each line gets a result row accessible as pm.redis.results.pre[i] or .post[i] in scripts.

SET test-trace-{{counter}} "{{random.uuid}}"
GET cart:{{userId}}
TTL session:{{userId}}

Assertion rows reference results by index or by key.

9.4 Worked example — cache validation#

Scenario: HTTP POST /signup creates a user and the service writes a session entry to Redis. We want to assert the right key landed with an expiry attached.

Request configuration:

• Body (raw JSON): {"type":"USER_SIGNED_UP","userId":"{{userId}}"}
• Redis tab → Post-response:

  GET session:{{userId}}
  TTL session:{{userId}}
  

• Assertions: redis.post.0 contains {{userId}}, and redis.post.1 gt 0 (a TTL of -1 means the service forgot to set an expiry; -2 means the key was never written).

9.5 Scripted reads#

const cached = pm.redis.get(`cart:${pm.environment.get("userId")}`);
pm.test("cart cache populated", () => {
  pm.expect(cached).to.exist;
});

const lastCommand = pm.redis.results.pre[0];
pm.log("GET returned", lastCommand);

pm.redis.get reads the result already captured in the pre/post stage. For ad-hoc commands not declared in the tab, prefer adding them to the pre/post list rather than synthesising a new connection in a script — the pre/post stages are part of the run report.

Where to next: §10 Kafka testing.

10. Kafka testing#

Callman ships a Kafka consumer. The producer side is your service under test — you POST to it and assert the event came out. There is no pm.kafka.publish (and there shouldn't be — tests should not be the only producer of an event).

10.1 Connection#

Sidebar → Connections → + Kafka. Broker list (comma-separated), client id, group id, security (PLAINTEXT, SASL_PLAINTEXT, SASL_SSL, SSL), username/password if SASL. Saved per workspace.

10.2 Listener mechanics#

In the Kafka tab of a request:

• Connection — saved connection ID.
• Topic — exact topic name (no wildcards).
• Match mode — regex or rules (see below).
• Timeout — milliseconds to listen before giving up. Default 5000.

When the request runs, the listener subscribes before the HTTP call goes out. Every message that arrives during the window is captured. The first one that matches becomes pm.kafka.event(). If the window ends with no match, pm.kafka.event() is null and the test fails with "no matching Kafka event."

There are no automatic retries. If you need to listen longer, raise the timeout. If you need to listen on multiple topics for the same request, use a scenario with a Kafka node per topic and an aggregator.

10.3 Regex match mode#

The captured message is serialised as a string and matched against your regex. Useful when the message has a stable token you can spot quickly:

regex: /"orderId":"ord_[0-9a-f]{16}"/

This mode is fast and forgiving but doesn't let you assert against specific fields. Good for "did anything that looks like an order event come through."

10.4 Rules match mode#

Add one or more field rules. Each rule is <path> <op> <value>.

All rules must match for the message to count. Path expressions support dot notation and numeric array indices (kafkaEvent.items.0.sku).

10.5 Asserting against the matched event#

pm.test("order event published", () => {
  const evt = pm.kafka.event();
  pm.expect(evt).to.exist;
  pm.expect(evt.type).to.equal("OrderCreated");
  pm.expect(evt.orderId).to.equal(pm.environment.get("orderId"));
});

pm.test("no errant duplicate events", () => {
  const all = pm.kafka.messages();
  const ours = all.filter((m) => m.value?.orderId === pm.environment.get("orderId"));
  pm.expect(ours.length).toBe(1);
});

10.6 Correlation-id pattern#

The reliable way to match the right event in a busy topic: send a correlation id with the HTTP call, then match on it.

Pre-request script:

pm.environment.set("corr", crypto.randomUUID());

Request body:

{ "userId": "{{userId}}", "correlationId": "{{corr}}" }

Kafka tab rules:

This way two parallel test runs against the same service never collide.

10.7 Worked example — event-driven order pipeline#

Goal: when we POST an order, three things should happen:

1. orders.events.v1 should receive an OrderCreated.
2. The orders table should have a row with status PENDING.
3. The Redis hash inventory:reserved:{orderId} should exist.

Build it as one request:

• HTTP: POST /orders with the body.
• Kafka tab: topic orders.events.v1, rules kafkaEvent.type = OrderCreated, kafkaEvent.correlationId = {{corr}}.
• DB tab → Post: SELECT status FROM orders WHERE id = '{{response.id}}', rule db_post.firstRow.status = PENDING.
• Redis tab → Post: EXISTS inventory:reserved:{{response.id}}, rule redis.post.0 = 1.

When the request runs you get one report row showing all four checks. Any single failure flips the request red and pinpoints which side-effect broke.

Where to next: §11 Scenario builder.

11. Scenario builder — node by node#

Scenarios are visual workflows. You drag typed nodes onto a canvas, connect them with edges, and execute the graph. Scenarios live in the Scenarios tab of the sidebar and run server-side when invoked from the desktop "Run" button, scheduled jobs, or the CLI.

Callman ships 11 node types. This chapter covers each one with purpose, configuration, outputs, and pitfalls.

11.1 Canvas mechanics#

• Drag a node from the palette onto the canvas, or click the palette entry to drop one near the viewport centre.
• Hover an edge endpoint to drag a connection.
• Click a node to open its config drawer. Double-click to rename.
• Right-click → Delete removes a node and its edges.
• The minimap (bottom right) helps with large graphs. Cmd/Ctrl scroll zooms; space-drag pans.

Every non-Condition node has two policy fields:

• On failure — stop (default; halt the run) or continue (skip to the next node).
• Retry — enabled, maxAttempts, delayMs. Retries apply only to the node itself; success after retry is success.

11.2 Edge types#

A node can have multiple outgoing default edges (a fan-out — every downstream is started after the source completes). A Condition node emits exactly one yes and one no.

11.3 Request node#

Identical surface to a request tab: URL, method, params, headers, auth, body, scripts, contract rules, DB/Kafka/Redis. When a request node runs inside a scenario it sees the merged environment plus any prior node's outputs (see §12 Scenario data flow).

Outputs accessible by downstream nodes (referenced by the node's label as shown on the canvas):

{{<node-label>.response.status}}
{{<node-label>.response.headers.<name>}}
{{<node-label>.response.body.<json-path>}}
{{<node-label>.request.url}}

Common pitfall: forgetting that scripts in a scenario node run on the server, so they cannot reference the desktop machine. No localhost-style assumptions.

11.4 DB node#

Executes a database query (Oracle, MongoDB, PostgreSQL, or MySQL) inside the workflow. Choose the connection, write the query — plus, for PostgreSQL/MySQL, optional positional bind params ($1 / ?), each template-resolved at run time — and either:

• Assert via declarative rules (same shape as the request DB tab), or
• Let downstream nodes consume the result via {{<node-label>.rows[0].id}} and {{<node-label>.rowCount}}. Note the bracket index syntax — rows.0.id does not resolve in scenario references.

Best used for state checks between HTTP calls — confirm the previous step persisted before the next step depends on it.

11.5 Redis node#

Run one or more Redis commands. Useful for:

• Asserting cache state mid-workflow.
• Setting up a fixture (priming a key) before a request.
• Cleaning up after a run (DEL test:scratch:{{runId}} in the End branch — DEL takes exactly one key, no wildcards).

Only GET, SET, DEL, EXISTS, and TTL are supported (see §9.2). If the node has a single command, the node output is that command's result — reference it as {{<node-label>}}. With multiple commands the output is an array: {{<node-label>[0]}}, {{<node-label>[1]}}, ...

11.6 Kafka node#

Subscribe to a topic with a timeout and either regex or rules matching, exactly like the request Kafka tab. Place a Kafka node after the request that triggers the event, with edges configured so the listener starts before the request fires. (In practice you draw: previous-node → Kafka, and the engine opens the listener at "node about to start" time.)

11.7 Wait node#

Pauses the workflow for the configured number of milliseconds.

Use cases:

• Give an async system time to settle before the next assertion.
• Implement a poll loop (Wait → check → Condition → loop back).
• Throttle a sequence of write requests during a soak test.

There is no scheduling fairness — a 30s Wait blocks one node slot for 30s. For long sleeps, prefer a Condition-driven retry loop with short waits.

11.8 Script node#

Runs arbitrary JavaScript with the same pm.* sandbox the request scripts use. Use when:

• You need to compute a derived value from multiple prior outputs.
• The next node's configuration depends on conditional logic.
• A test assertion spans more than one prior node and a Condition node would be too coarse.

The script's return value is stored as the node output. Reference it directly: {{<node-label>}} for a scalar return, or {{<node-label>.orderId}} when the script returns an object.

const reqOut = pm.variables.resolve("{{create-order.response.body}}");
const dbOut = pm.variables.resolve("{{verify-order.rows[0].status}}");

if (dbOut !== "PENDING") {
  pm.expect.fail(`Expected PENDING, got ${dbOut}`);
}

return { orderId: JSON.parse(reqOut).id, verifiedStatus: dbOut };

11.9 Notification node#

Sends a message to Slack. The notification provider in this build is Slack only — Teams, email, generic webhooks, and PagerDuty are not wired.

Config fields:

• Connection — saved Slack workspace connection.
• Channel override — send to a different channel than the connection's default.
• Severity — info, success, warning, error. Controls the default colour and emoji.
• Message template — {{...}} substituted. Can reference any prior node output.
• Advanced → useRichBlocks — render as Slack block-kit blocks rather than plain text.

Use case: route a scenario's failure summary to an oncall channel.

:rotating_light: Payment scenario failed at {{failed-node-label}}
Order: {{create-order.response.body.id}}
Error: {{failed-node-error}}

11.10 Condition node#

Branches the workflow into yes and no paths. Two configuration modes:

• Builder — visual rules with AND/OR groups. Each leaf is field op value against any prior node's output or any variable.
• Expert — write a JS expression that returns truthy/falsy.

Example builder rule:

group OR
  ├ create-order.response.status == 201
  └ group AND
     ├ create-order.response.status == 200
     └ create-order.response.body.status == "PAID"

Conditions have no on-failure or retry policy. They either evaluate true (emit yes edge) or false (emit no edge). An evaluation error is treated as false.

11.11 End node#

A terminal node. Two modes:

• Plain end — the branch stops here. The scenario as a whole succeeds if every End reached was non-failure.
• Loop end — when toggled on, the End jumps back to a chosen target node and increments an internal iteration counter. Config:
  • loopTargetNodeId — which node to jump back to.
  • loopIterations — max times to loop.
  • loopDelayMs — pause between loops.

Loops are the only first-class repeat construct. There is no Loop node and no Parallel node. Fan-out is achieved by drawing multiple default edges out of one node.

11.12 Aggregator node#

Merges prior node outputs into a single structured value. The merge is described by a template:

{
  "orderId": "{{create-order.response.body.id}}",
  "kafkaSeq": "{{event-check.sequence}}",
  "dbStatus": "{{verify-row.rows[0].status}}"
}

The merged object is the aggregator's output — downstream nodes (notifications, end summaries) reference its fields as {{<aggregator-label>.orderId}}.

Use case: build a structured payload to ship to Slack or attach to a report's metadata.

11.13 Sub-scenario node#

Embeds a saved scenario as a single node. The contained graph runs the way a function call does — its inputs are the merged variables at the call site, its outputs are accessible at the call site.

Two uses:

• Reuse — extract a "validate user state" subgraph once, drop it into every flow that needs the check.
• Group — collapse a noisy region of a graph into a single node for readability. Right-click → Ungroup restores the inline nodes with their original edges (the ungroup snapshot preserves incoming/outgoing edge bindings).

11.14 Failure policy decision tree#

Did the node fail?
├── retry.enabled and attempts < maxAttempts
│     └── wait delayMs, run again
├── onFailure = "continue"
│     └── mark node failed, follow outgoing edges anyway
└── onFailure = "stop" (default)
      └── halt the scenario; status = failed or completed_with_failures

completed_with_failures is the status when at least one node failed but onFailure = continue carried the run to the end.

Where to next: §12 Scenario data flow.

12. Scenario data flow#

A scenario shares state through three channels: the merged environment, prior node outputs, and the __meta workflow state. Understanding which goes where is what separates a brittle scenario from a maintainable one.

12.1 The merged environment#

When a scenario starts, the engine builds one variable scope:

workspace globals
     +
selected environment
     +
sub-scenario inputs (if this is a sub-scenario call)

Every node sees the same scope. A node writing pm.environment.set mutates the scope for every subsequent node in the same branch. The mutation does not leak back to the workspace environment after the run completes — it lives only for that run.

12.2 Node output references#

Each node's output is addressable as {{<node-label>.<path>}} everywhere a {{...}} is supported. The key is the node's label exactly as it appears on the canvas. Array elements use bracket indexing: rows[0].id, not rows.0.id.

Because references are label-based, renaming a node's label breaks every downstream {{...}} that mentions it — update them together. Labels with trailing spaces are trimmed before matching.

Besides label references, the engine also exposes the most recent result of each kind under fixed roots: {{response.*}} (last HTTP response), {{db.result[<i>].<column>}} (last DB rows), {{redis.result}} (last Redis result), {{kafkaEvent.*}} (last matched Kafka event).

12.3 __meta#

__meta is the workflow state struct available inside scripts and template expressions. Useful keys:

• __meta.scenarioId — id of the scenario being executed
• __meta.runId — unique id of this run
• __meta.triggerType — manual_server, scheduled, or cli
• __meta.startedAt — ISO timestamp
• __meta.previousNode.id / __meta.previousNode.status

Reach for __meta when you need to behave differently based on how the run was triggered — for example, only sending a Slack alert when the trigger is scheduled (during interactive runs, the operator is already watching).

12.4 Conditional branching deep dive#

The Condition node accepts either a builder rule tree or a JS expression. The rule tree is recursive: groups (AND/OR) contain either leaf comparisons or nested groups. There is no depth limit in the UI, but past two levels readability suffers — switch to the Script node if you need more.

Three real branching shapes:

• Error-handling — condition: request.status != 2xx → yes goes to a Notification + End; no continues the happy path.
• Feature flag — condition: env.featureX == "enabled" → diverging happy paths for two product variants.
• Polling — condition: check-status.rows[0].status == "READY" → yes exits the loop; no goes through Wait and loops back.

12.5 Aggregator merge patterns#

Aggregators are syntactic sugar — anything an aggregator does, a Script node could do. But aggregators are declarative, so they show up cleanly in reports and don't require sandbox time.

Pattern: collect three parallel results into one Slack message.

{
  "trace": "{{__meta.runId}}",
  "kafka_seen": "{{kafka-check.type}}",
  "db_seen":    "{{db-check.rows[0].status}}",
  "cache_seen": "{{redis-check}}",
  "duration_ms": "{{__meta.previousNode.durationMs}}"
}

12.6 Sub-scenario reuse#

When the same five-node check ("verify user, refresh token, ping health, log start, capture trace id") appears in three scenarios, extract it to its own scenario and reference it via a Sub-scenario node in each caller. Edits to the underlying scenario propagate to every caller on the next run.

Best practice: name sub-scenarios with verbs (refresh-auth, assert-user-active) and keep them under a dozen nodes each.

12.7 Retry / failure policy as a flow construct#

Retries happen inside a node — you don't see them as separate graph entries. If you need visibility into "we retried three times and finally succeeded," put the call inside a Sub-scenario and read its run report.

If you need a fail-then-compensate flow ("if POST fails, send a DELETE to undo a partial side-effect"), use onFailure: continue on the failing node and route the default edge to the compensation request. The downstream nodes can read <failed-node-id>.error.

Where to next: §13 Real-world workflows.

13. Real-world workflows#

Eight end-to-end scenarios that map to recognisable enterprise patterns. Every one is buildable today with the nodes documented above — no invented features.

13.1 Banking onboarding flow#

Goal — Verify the new-customer flow: POST creates a user, Kafka emits a CustomerOnboarded, and the user lands in the customers Oracle table.

Graph:

[Request: POST /customers]
       │
       ▼
[DB: SELECT status FROM customers WHERE id = {{create.response.body.id}}]
       │
       ▼
[Kafka: orders.events.v1 / type=CustomerOnboarded / corr matches]
       │
       ▼
[Condition: all green?]
   yes ──▶ [Notification: success to #onboarding-tests]
   no  ──▶ [Notification: error to #oncall] ──▶ [End]

Why this scenario — exercising three systems in one workflow is exactly what a request tab struggles with at scale: you'd need three separate requests with chained envs. The scenario shows the full graph in one place and produces one report row covering all checks.

13.2 E-commerce checkout validation#

Goal — A user adds an item, places an order, payment lands, and the inventory reservation appears in Redis.

Graph:

[Request: POST /cart/items]
   │
   ▼
[DB: SELECT cart_id FROM carts WHERE user = {{userId}}]
   │
   ▼
[Request: POST /checkout]
   │
   ▼
[Redis: EXISTS inventory:reserved:{{checkout.response.body.orderId}}]
   │
   ▼
[Kafka: payments.events / type=PaymentCaptured / corr matches]
   │
   ▼
[Aggregator: assemble checkout summary]
   │
   ▼
[End]

Why — checkout flows fail in interesting ways (HTTP succeeds but payment doesn't, or payment succeeds but the inventory lock didn't land). One scenario, four assertions, every failure pinpointed.

13.3 Loan approval system#

Goal — Submit a loan application, branch based on credit-score range, run a sub-scenario for fraud checks if amount > 50k.

Graph:

[Request: POST /loans]
   │
   ▼
[Script: pull score from credit bureau via pm.sendRequest, return tier]
   │
   ▼
[Condition: tier in {gold, platinum}?]
   yes ──▶ [Request: PATCH /loans/{id} status=approved] ─▶ [End]
   no  ──▶ [Condition: amount > 50000?]
                yes ──▶ [Sub-scenario: full-fraud-check]
                            │
                            ▼
                       [Condition: passed?]
                          yes ──▶ [Request: PATCH /loans/{id} status=manual_review] ─▶ [End]
                          no  ──▶ [Notification: warning #risk] ──▶ [End]
                no  ──▶ [Request: PATCH /loans/{id} status=rejected] ─▶ [End]

Why — branching plus reuse. The fraud check is its own scenario, shared between loans and high-value transfers. Future edits to the fraud subgraph propagate everywhere.

13.4 Payment orchestration with rollback#

Goal — Capture payment, write order, emit event. If the event publish fails, refund the payment.

Graph:

[Request: POST /payments/capture]   onFailure: stop
   │
   ▼
[Request: POST /orders]              onFailure: continue
   │ default
   ▼
[Kafka: orders.events / type=OrderCreated]   onFailure: continue
   │
   ▼
[Condition: orders OK and Kafka matched?]
   yes ──▶ [End]
   no  ──▶ [Request: POST /payments/{{captureId}}/refund]
              │
              ▼
           [Notification: warning to #payments-oncall]
              │
              ▼
           [End]

Why — distributed transactions don't exist in HTTP. Tests that care about side-effect rollback need to model the compensating step explicitly.

13.5 Notification pipeline validation#

Goal — When a user updates their profile, an audit row should land in the DB, an event should hit Kafka, and a Slack notification should fire downstream. We verify all three.

Graph:

[Request: PUT /users/{{userId}}]
   │
   ▼
[Aggregator (fan-out by drawing 3 edges out of above)]
   ├ default ──▶ [DB: SELECT * FROM audit WHERE actor = {{userId}} ORDER BY ts DESC FETCH 1 ROW]
   ├ default ──▶ [Kafka: users.audit.v1 / corr matches]
   └ default ──▶ [Redis: EXISTS notif:dispatched:{{userId}}]
                    │  │  │
                    └──┴──┴──▶ [Aggregator: summarise]
                                  │
                                  ▼
                               [End]

Why — fan-out lets all three checks run in the same window. The final aggregator gives you one row in the report with the full state.

13.6 Microservice integration smoke#

Goal — Hit three services in sequence (auth → catalog → cart), threading the same correlation id, and verify each step.

Graph:

[Script: generate corr, set pm.environment.set("corr", randomUuid)]
   │
   ▼
[Request: POST /auth/login   header: X-Corr: {{corr}}]
   │
   ▼
[Request: GET /catalog/items  header: X-Corr: {{corr}}]
   │
   ▼
[Request: POST /cart/items    header: X-Corr: {{corr}}]
   │
   ▼
[DB: SELECT count(*) FROM access_log WHERE correlation_id = '{{corr}}']
   │
   ▼
[Condition: count >= 3?]
   yes ──▶ [End]
   no  ──▶ [Notification: error #obs-gaps]

Why — observability gaps are usually invisible. Threading a correlation id and asserting the trail lands in your log table detects them before they hide a real incident.

13.7 Event-driven saga validation#

Goal — A saga publishes three events in order: OrderCreated, InventoryReserved, PaymentInitiated. Validate all three within a single window.

Graph:

[Request: POST /orders]
   │
   ▼
[Kafka node A: orders.events.v1 type=OrderCreated]
[Kafka node B: inventory.events.v1 type=InventoryReserved]
[Kafka node C: payments.events.v1 type=PaymentInitiated]
   │
   ▼ (drawing three default edges from POST /orders into A, B, C; then A/B/C → Aggregator)
[Aggregator: shape result for stakeholders]
   │
   ▼
[Notification: info to #saga-tests]
   │
   ▼
[End]

Why — sagas are the hardest pattern to validate end-to-end with generic tooling. Three Kafka nodes in parallel give you one clear pass/fail per event without writing a custom consumer.

13.8 CI/CD smoke suite#

Goal — Every morning at 02:00 the staging environment runs a five-request smoke test. On failure, Slack alerts. Reports are archived.

Schedule — daily 02:00 Europe/Baku, retry 2× at 5 min intervals, notify on failure to #staging-smoke.

Graph:

[Request: GET /health]
   │
   ▼
[Request: POST /auth/login]
   │
   ▼
[Request: GET /me]
   │
   ▼
[Request: GET /products]
   │
   ▼
[Request: GET /orders/recent]
   │
   ▼
[Condition: all 200?]
   yes ──▶ [Notification: info "smoke green"] ──▶ [End]
   no  ──▶ [Notification: error attach failed-node-label] ──▶ [End]

Pair with CI — same scenario kicked off from GitHub Actions on every deploy via callme scenario run, the report-id link posted to the PR.

Where to next: §14 Data-driven runs.

14. Data-driven runs#

Some tests are the same request over and over with different inputs: "register these 5,000 users from this CSV and assert each one ended up in the right state." Callman's Run with Data feature handles that.

14.1 Opening Run with Data#

In any request tab, open the 3-dot menu → Run with Data. A full-screen view replaces the editor with two columns: config on the left, live results on the right.

14.2 Data files#

Three formats supported:

• CSV — first row is the header; subsequent rows are data.
• JSON — top-level array of objects; keys become column names.
• XLSX — first sheet, first row is the header. (The XLSX parser is lazy-loaded; the bundle stays small unless you actually upload one.)

Upload via the Browse button. The preview shows row count and the first five rows so you can confirm the columns look right.

Edge cases handled:

• UTF-8 BOM in CSV headers is stripped.
• Empty rows are skipped.
• Duplicate column names are rejected with a toast.
• Whitespace in column names is trimmed.

14.3 Per-iteration variable scope#

Each row becomes a temporary variable map. If your CSV has columns userId, email, name, then on iteration 7 those three values are available as {{userId}}, {{email}}, {{name}} anywhere the request uses {{...}}.

Precedence: row column → environment → globals. So a column named host will override the env variable named host for that iteration only.

14.4 Config options#

Use a high concurrency (10-50) for ingestion-style workloads where the server scales horizontally. Use sequential with delay for systems that throttle or that share a piece of mutable state.

14.5 Token refresh during long runs#

For 5,000 requests with a 1-hour token: put a token-refresh check in the request's own pre-request script (see §6.10). The script runs once per iteration; the tokenExpiresAt env value persists across iterations within the same Run.

14.6 Live results table#

The results column streams rows as iterations finish. Each row:

• Row index
• Status pill (PASS / FAIL / pending)
• HTTP status
• Duration
• Truncated error or response snippet (~500 chars)
• "Details" link → expandable drawer with full request + response

Filter chips switch between All, Success, Failed.

14.7 Pause / stop / resume#

The Pause button freezes new iterations but lets in-flight ones finish. Stop is hard — running iterations complete, queue empties.

14.8 Exporting results#

Export JSON writes the full state (config + parsed file metadata

• every iteration result). Export CSV writes a flat table with row index, status, HTTP code, duration, and error.

14.9 When to use Run with Data vs a scenario#

Use Run with Data when:

• The unit of work is one request.
• You have a tabular file as the source of variation.
• You want concurrency without writing a worker pool.

Use a scenario when:

• The unit of work is multiple steps.
• Each iteration needs different branching.
• You need the run history server-side.

Where to next: §15 Callme CLI.

15. Callme CLI#

callme is the command-line entry point. It runs collections and scenarios in any environment that can install Node packages — CI runners, on-call laptops, bastion hosts.

15.1 Install#

npm install -g @callman/cli
callme --version    # 1.1.0

15.2 Authentication#

Two modes:

• File mode — collection and environment are local JSON files. No backend connection required. Useful for portable test suites bundled with the codebase.
• Remote mode — fetch collection and environment from the backend at runtime. Requires:
  • --token or CALLME_TOKEN — a Personal Access Token from Settings → Tokens. Prefixed cm_pat_.
  • --workspace or CALLME_WORKSPACE_ID.
  • --api-url or CALLME_API_URL — defaults to https://api.callman.io. Set this for self-hosted backends.

PATs are scoped. For scheduled scenario triggers, the PAT needs the pipeline:trigger scope. Token issuance and scope assignment happen in the desktop app under Settings → Tokens.

15.3 callme run — collections#

File mode:

callme run smoke.json --env staging.json --report json --output smoke.json

Remote mode:

callme run \
  --collection-id col_01H6A... \
  --environment-id env_01H6B... \
  --workspace ws_01H6C... \
  --token "$CALLME_TOKEN" \
  --report html --output smoke.html

Flags:

15.4 callme scenario run#

Triggers a scenario server-side. The CLI process polls until the scenario reaches a terminal status (or until the timeout) and exits accordingly.

callme scenario run \
  --scenario-id sc_01H6D... \
  --environment-id staging \
  --workspace ws_01H6C... \
  --token "$CALLME_TOKEN" \
  --report json --output scenario.json

Flags:

Every invocation finishes with a colored Exit code: N line (green on 0, red on 1, white-on-red on 2). The line is printed even in --silent mode — the exit code is the machine-readable signal CI scripts depend on.

Folder mode example:

callme scenario run \
  --folder-id flr_01H6X... \
  --workspace ws_01H6C... \
  --token "$CALLME_TOKEN"

The CLI fetches the folder's enabled scenarios, triggers them in the mode the folder was configured for, prints a per-scenario summary, and finishes with an aggregated Folder summary: N passed, M failed line before the Exit code: footer.

15.5 Exit codes#

The priority gate only applies to callme scenario run. callme run (collections) is binary: any failed request / script / contract test returns 1 unless --no-fail is set. See §23 for the priority model end to end.

15.6 CI recipes#

GitHub Actions — run the smoke suite on every push to main and post the JSON report as an artifact.

name: smoke
on: { push: { branches: [main] } }
jobs:
  smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20 }
      - run: npm install -g @callman/cli
      - run: |
          callme scenario run \
            --scenario-id ${{ vars.CALLMAN_SCENARIO_ID }} \
            --workspace ${{ vars.CALLMAN_WORKSPACE_ID }} \
            --environment-id staging \
            --token ${{ secrets.CALLMAN_TOKEN }} \
            --report json --output smoke.json
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: smoke-report, path: smoke.json }

GitLab CI — same idea, with a cached node_modules and a JUnit- adjacent artifact for the pipeline UI.

smoke:
  image: node:20
  script:
    - npm install -g @callman/cli
    - callme run collection.json --env $CI_ENV --report json --output smoke.json
  artifacts:
    when: always
    paths: [smoke.json]

Jenkins — declarative pipeline. Treat callme like any other test runner; mark the build unstable on non-zero exit.

pipeline {
  agent any
  environment {
    CALLME_TOKEN = credentials('callman-token')
  }
  stages {
    stage('Smoke') {
      steps {
        sh '''
          npm install -g @callman/cli
          callme run collection.json --env ci.json --report json --output smoke.json
        '''
      }
      post {
        always { archiveArtifacts artifacts: 'smoke.json' }
      }
    }
  }
}

The Callman backend does not ship Jenkins/GitLab/Azure DevOps plug-ins — these recipes rely solely on the CLI.

Folder smoke suite — one CLI step runs every enabled scenario in a folder and gates the build only on critical failures.

callme scenario run \
  --folder-id "$CALLMAN_FOLDER_ID" \
  --workspace "$CALLMAN_WORKSPACE_ID" \
  --token "$CALLMAN_TOKEN" \
  --fail-threshold critical

Configure which scenarios participate and whether they run sequentially or in parallel in the desktop app (sidebar → folder menu → Pipeline config). A flaky low-priority probe in the folder no longer breaks the build; only the critical scenarios do.

Where to next: §16 Scheduling.

16. Scheduling#

Scheduling turns a scenario into a recurring server-side job. The desktop app has the editor; the backend runs the cron loop.

16.1 Creating a schedule#

In the scenario editor, Schedules → New schedule. Pick a trigger type:

Every schedule has an IANA timezone (UTC, Europe/Baku, America/New_York, etc.). The cron loop uses that timezone — DST is handled automatically.

16.2 Execution policy#

The conservative default for new schedules: skip-if-running, 30 min max runtime.

16.3 Failure policy#

A retry is a fresh scenario run, not a partial resume. Use retries for transient infra issues — they don't fix a broken assertion.

16.4 Notifications#

Optional Slack notification on success and / or failure. The Slack channel and severity come from the connection configuration. Failure notifications include the report id, the failed-node label, and a direct link to the report in the desktop app.

16.5 Schedule state visible in the UI#

For each schedule the editor shows:

• lastRunAt — when it last fired
• lastRunStatus — success / failed / completed_with_failures / stopped
• nextRunAt — when it will next fire
• activeRunsCount — how many runs are currently in-flight (only non-zero if parallel runs are allowed)
• Retry state — current attempt number and nextRetryAt

16.6 Three patterns#

Hourly health probe — Cron 0 * * * *, scenario hits 3 endpoints, 0 retries, Slack on failure only. Treat as a heartbeat.

Nightly regression — Daily 02:00 Europe/Baku, 30-step scenario, 2 retries 5 min apart, Slack on both success and failure, JSON report archived via webhook.

Weekly stakeholder report — Weekly Monday 09:00, scenario that runs a series of reads, produces an aggregator-summarised JSON, notifies a public channel with the metrics.

Where to next: §17 Reports and analytics.

17. Reports and analytics#

Every scenario run produces a report. Reports are server-side and queryable by anyone in the workspace with viewer-or-better access.

17.1 Per-run metadata#

Each report has:

• workspaceId, userId (who triggered)
• scenarioId, scenarioName
• triggerType — manual_server, scheduled, cli
• startedAt, endedAt, durationMs
• status — see terminal-state legend below

17.2 Per-node detail#

For each node executed:

• stepId (deterministic per scenario)
• nodeId, label, type
• status, attempts, retryCount
• startedAt, endedAt, durationMs
• input and output snapshots
• scriptResults — every pm.test outcome
• contractResults + contractSummary — declarative checks
• error — the error message (if any)

17.3 Terminal states#

17.4 Export formats#

• JSON — the canonical machine format. Whole structure as described above. Use for diffing across runs and for CI archives.
• HTML — styled, expandable, human-readable. Use for handing a failure to a stakeholder. The HTML export is print-styled (-webkit-print-color-adjust: exact), so opening it in the desktop app or any browser and choosing Print → Save as PDF produces a faithful PDF without a separate export step.

A dedicated server-side PDF endpoint is not exposed — print-to-PDF from the HTML report is the supported path.

17.5 Replay and diff#

Inputs and outputs are stored per node. Opening a past report and clicking a node shows the inputs the node saw and the outputs it produced. Diff between two report runs is available from the report list view — pick two runs of the same scenario and click Diff.

17.6 Debugging a failed node#

The drill-down sequence that solves 90% of failures:

1. Open the report. Look at the failedNode label in the summary.
2. Open the failed node's drawer. Read the error message.
3. Inspect inputs — was the variable that fed this node correct? If not, the bug is upstream.
4. Inspect outputs — did the node receive a response the assertion didn't expect? If yes, the assertion or the service is wrong.
5. Open the previous run's report and diff. Did the input change? That points at config drift.

Where to next: §18 Tools.

18. Tools#

The Tools page (left sidebar, Tools entry) collects standalone utilities that don't need a request to execute.

18.1 Base64 encoder / decoder#

Encode text and binary to base64; decode base64 back to text or binary. Useful for inspecting Authorization: Basic headers, JWT payloads (the middle segment is base64url-encoded), or signing material.

18.2 UUID generator#

Generate v4 UUIDs one at a time, batched, or in a specific format (with / without dashes, upper / lower case). Drop into env variables for static test users.

18.3 JWT decoder#

Paste a token, see header, payload, and the expiry claim parsed as an ISO date. The signature is not verified — this is an introspection tool, not a verifier.

Common task: "the API is rejecting my token, is it expired?" → paste, read exp → if in the past, refresh.

18.4 Diff checker#

Side-by-side comparison of two text blobs (auto-detected as JSON or plain). Useful for "the response shape changed between staging and prod" investigations.

18.5 File converter#

Convert binary files to base64 (for stuffing into a JSON request body) and back. Image files render a preview so you can confirm you've encoded the right thing.

Where to next: §19 Workspace collaboration.

19. Workspace collaboration#

Workspaces are the unit of collaboration. A user can be in many; a collection belongs to exactly one.

19.1 Roles#

A read-only user can still trigger runs of scenarios and collections — viewing doesn't prevent execution, only modification. Edits to read-only collections in the desktop app are blocked at the UI layer.

19.2 Inviting members#

Settings → Workspace → Members → Invite. Enter an email; an invitation email goes out. The invitee accepts and joins as the role you picked. Owners and Editors can invite Editors and Viewers; Editors cannot invite Owners.

19.3 Personal Access Tokens#

Settings → Tokens → + New Token. The token string is shown once on creation (prefix cm_pat_) — copy it into your secrets manager immediately. Tokens are revocable from the same page and are workspace-scoped: a token issued for workspace A cannot read or trigger anything in workspace B.

Two pipeline scopes guard the CI endpoints:

pipeline:read is a base scope — the backend force-adds it to every newly minted token so a pipeline:trigger-only token cannot fetch the bundle it needs to poll the run it just triggered. Each endpoint enforces its required scope independently: a missing scope surfaces as 403 Token is missing the '<scope>' scope, which the CLI prints verbatim to make CI logs self-explanatory.

19.4 Shared environments#

Environments belong to a workspace. Every member with at least viewer access can read them. Editor and Owner can mutate them. There is no per-environment ACL — env-level access follows the workspace role.

A common pattern: an admin-only workspace holds production credentials, with three Owners. The day-to-day team workspace references those values via an env labelled prod whose contents are empty placeholders. The CI pipeline overlays the real values at run time via --env.

19.5 Audit trail#

Workspace edits (member invites, role changes, scenario edits, env mutations) record an audit entry. The audit log is visible to Owners via Settings → Workspace → Audit. Entries: actor, action, target, timestamp, details.

Where to next: §20 Best practices.

20. Best practices#

Patterns that have come up enough times across teams to be worth codifying.

20.1 Naming#

• Collections — domain-first, not project-first. Orders not Orders-Q3-2025-Sprint12. Sprint context goes in folder names.
• Folders — 01 happy path, 02 edge cases, 99 manual-only. Prefixes keep alphabetical sort meaningful.
• Requests — verb + noun + key qualifier. Create order (default), Create order (high value). Don't bury the meaning in the URL.
• Scenarios — purpose-first. Smoke / staging / nightly, Verify / saga / payment.
• Environments — exactly your deploy targets. Resist the urge to create per-user environments; use workspace globals for personal overrides.

20.2 Environment strategy#

A workable three-env baseline:

• local — points at your machine's localhost services. Contains developer-only fixture tokens.
• staging — points at the shared staging cluster. Contains a service account token. CI uses this env.
• prod — points at production. Contains a read-only service account token. Most requests are tagged so they don't run here accidentally.

Avoid one mega-env with host_staging, host_prod, host_local. Switching env should be one click, not a search-and-replace.

20.3 Reusable script patterns#

• Auth refresh at the collection level. Every request in the collection gets the same pre-request script via collection-level configuration. No copy-paste.
• Trace id injection: pre-request script that sets a header X-Trace-Id from crypto.randomUUID() and stores it in env for the Tests script to assert on.
• Backoff in scripts: await new Promise((r) => setTimeout(r, delay)) inside a poll loop. Don't try to be clever with while loops on Date.now().

20.4 Scenario sizing#

• A scenario with more than ~25 nodes becomes hard to read. Extract a chunk into a sub-scenario.
• Aggregator + Notification is the canonical exit pattern: every branch funnels into the aggregator, the aggregator feeds one notification, then End.
• Wait nodes longer than 30s in a scheduled scenario are a smell — the run sits on a worker. Use a Condition-driven retry loop with short waits instead.

20.5 Retry versus assert#

• A flaky assertion is not a retry problem. Investigate.
• A flaky integration (network, broker, replica lag) is a retry problem. Apply node-level retries with short delays.
• For "the data is eventually consistent" cases, prefer a Condition poll loop inside the scenario — it's visible in the report.

20.6 Performance#

• Concurrency 50+ in the Data-Driven Runner is fine for stateless endpoints. Past that, you're load-testing — use a load tester.
• Listening on multiple Kafka topics in parallel is cheaper than doing it sequentially. A scenario with fan-out + 3 Kafka nodes is usually under a second slower than one Kafka node.
• Run a scenario's heavy DB queries inside a Script node only when you genuinely need the result in JS. Otherwise use the DB node — it shows up cleanly in reports.

20.7 Team workflows#

• One workspace per major product domain. Cross-domain workspaces become noisy fast.
• Pin a 00 — runbook collection at the top of each workspace with the project's shared conventions, owners, and pointers to the on-call channel.
• Use a legacy folder for things that nobody owns but nobody deletes. Better than the same requests cluttering the active list.

Where to next: §21 Mocks.

21. Mocks#

A mock is a short-lived fake HTTP API that lives at https://<api-host>/mock/<mockId>/<your-path>. It is workspace-scoped, expires automatically on its TTL, and is configured from Workspace → Mocks in the desktop app. Mocks let you unblock frontend / contract work before the real service exists, inject failure modes that would be hard to reproduce in production, or stand up disposable fixtures for a demo.

21.1 What mocks are (and are not)#

A mock is a test double, not a deployment target. Each mock:

• Has a unique slug (mockId) embedded in its public URL.
• Stores its own routes, response shapes, and per-mock state.
• Counts every request and rejects callers once totalRequests or the per-second rate limit are hit.
• Vanishes from the database at its expiresAt timestamp via a Mongo TTL index — no manual cleanup needed.

Mocks are not a persistence layer (state resets when the mock expires), not load-bearing infrastructure (rate limits cap them at small dozens of RPS), and not a Postman-style monitor (no scheduling — they're inert until called).

21.2 Creating a mock#

Workspace → Mocks → Create mock. The create modal asks for:

• Name (up to 50 chars) — human label in the list view.
• Description (optional, up to 200 chars).
• TTL — one of 1 hour, 6 hours, 24 hours, 48 hours (workspace default is 48 hours; the backend forbids longer to keep test data short-lived). The TTL countdown shows the time-to-expiry in the mock header.
• Auth — public (no auth, anyone with the URL can call) or token (a bearer token you set; callers send it as Authorization: Bearer <token>).

On create you land on the mock detail page with three tabs: Routes, State, Logs, plus a Settings drawer.

21.3 Routes#

A route binds an HTTP method + URL path inside the mock to a response. Routes use Express-style path syntax, so /users/:id/posts matches /users/42/posts and exposes id as a path parameter inside the response builders.

Pick a preset when creating a route — the preset wires up sane defaults so you don't start from a blank form:

Below the route list, the route editor lets you tweak:

• The status code and response headers.
• The body shape (one of static / template / builder — see §21.4).
• The artificial delay (none, fixed ms, or random range up to 60 s) — useful for testing client timeouts and loading states.

21.4 Response types#

Each route's response is one of three discriminated shapes:

1. Static — fixed status + headers + body. The body ships verbatim, no interpolation.
2. Template — same shape as static, but the body is interpolated with {{path.id}}, {{query.page}}, {{body.email}}, request helpers, and any state key. Use it for "return what they asked for" responses without writing code.
3. Builder — the preset-driven editor. The form fields you fill out (source, filtering, pagination, lookup field, etc.) are translated into a handler at request time. Most CRUD endpoints should use builder responses; you almost never need a hand-rolled handler.

21.5 Variants#

Each route can carry an ordered list of variants — alternative responses chosen by matching the incoming request against a condition tree.

Route GET /users/:id
  Variant 1 (priority 100):  path.id equals "0"  →  404 not found
  Variant 2 (priority 50):   header.x-flaky exists  →  500 simulated outage
  Variant 3 (priority 10):   query.locale equals "ja"  →  Japanese body
  Default response:                                   →  200 + standard body

Variants are evaluated highest-priority first. A condition is a rule set combined with all (every rule must match) or any (at least one). Each rule has a field (e.g. header.x-id, query.locale, body.email, path.id), an operator (exists, not-exists, equals, not-equals, contains, starts-with, matches), and an optional value.

Up to 20 rules per condition and 64 chars per variantId. Variants share the same status / headers / body forms as the route's default response.

21.6 State#

The mock owns a state bucket — a single JSON object. State is seeded from initialState on create and mutated by create / update / delete builder routes at request time.

{
  "users": [
    { "id": "u1", "email": "ada@example.com" },
    { "id": "u2", "email": "lin@example.com" }
  ],
  "orders": []
}

The State tab shows the current snapshot and exposes a Reset to initial button. Reset is the right escape hatch when a test run left the mock in a dirty state and you want a clean fixture for the next iteration. State is capped at 1 MB by default; large fixtures should be split across multiple mocks.

21.7 Settings#

The settings drawer exposes the per-mock guardrails:

These caps exist to keep mocks "test fixture" sized; a per-workspace mock count cap (default 20) keeps the workspace from accumulating zombie fixtures.

21.8 Logs#

The Logs tab is a reverse-chronological feed of every request that hit the mock: method, path, status code, latency, and the matched route / variant. Click any entry to see:

• The full request (headers, query, body).
• The full response sent back (after delay, after rate limiting).
• A Replay button that re-fires the same request — handy when iterating on a variant condition and you want to verify the new rule resolves the same call differently.

A TTL countdown in the mock header and a request-budget progress bar in the Settings drawer keep "how much is left" visible without you opening the logs.

21.9 Calling a mock from a request#

Use the mock URL in any Callman request, scenario request node, or external client:

GET https://api.callman.io/mock/m_01H7A.../users/42

For token-auth mocks, add Authorization: Bearer <your-mock-token>. The mock surface lives at /mock/<mockId>/... — outside the /api namespace and outside workspace auth, because it has to be reachable by services that don't know about your Callman session.

21.10 Use cases#

Frontend dev parallel to backend — the API team agrees the shape of POST /orders, you stand up a mock that returns the agreed-on body, and the frontend ships against the mock until the real endpoint lands. Once it does, only the base URL changes.

Failure injection — a variant on your payment endpoint returns 500 when the request body carries simulateOutage: true. Your scenario can call the live route, then call the same route with the simulator flag, and assert the error-handling path.

Demo seed — populate the mock's initialState with the data your stakeholder wants to see, link the demo UI at the mock URL, present without touching the real database.

21.11 What mocks are NOT#

• They are not a long-running staging environment — 48 hour TTL cap. Use a real deployment for that.
• They are not a database — state is in-process JSON, lost on expiry, capped at 1 MB.
• They are not load-tolerant — rateLimitPerSecond keeps them in the "fixture" performance class. If you need load, hit the real endpoint (see §22).

Where to next: §22 Performance testing.

22. Performance testing#

Performance mode lets you turn a single saved request into a load generator: a number of virtual users (VUs), a curve that describes how many of them are active over time, and a data source that feeds per-iteration variables. The desktop app runs the load locally (against whichever target your request URL resolves to) and renders live metrics — latency percentiles, throughput, error rate — as the test progresses.

Use perf mode for capacity questions ("does our login endpoint hold 500 concurrent users?") and regression questions ("did p95 just double after the new release?"). For correctness questions, write a scenario (§11).

22.1 Opening perf mode#

Open any request, then Tabs → Perf test. The first time you open the tab for a request the app snapshots the current state (URL, headers, body, auth, env, scripts) so the test runs against a frozen copy. Editing the request afterwards does not retroactively change the perf test config — open the perf tab again to re-snapshot.

The perf tab has three screens, swapped by state:

• Config screen — pick the load profile, data source, and acceptance thresholds.
• Running screen — live charts and counters during the test.
• Report screen — final metrics, percentile distribution, and error breakdown.

22.2 Load profile#

Pick one of four curve shapes (each preview is a sparkline in the profile-picker cards):

Defaults are conservative (50 VUs / 1 min for Fixed, up to 500 VUs for the dynamic profiles). A live preview chart shows the curve before you start the test so you can sanity-check the duration.

22.3 Data source#

Pick how each VU iteration's variables are produced:

• None — every iteration uses the same snapshotted request. Right for "is this endpoint healthy under load" probes.
• Inline values — paste rows of values, optionally per-VU.
• CSV file — upload a CSV; each row is one iteration. Headers become variable names.
• Faker catalog — generate fields from a built-in faker (email, uuid, first name, etc.). Useful when you need uniqueness on every iteration without managing a CSV.

The data row is bound into the request as variables, so the same {{email}} token you wrote in the request body becomes the iteration-scoped value during perf runs.

22.4 Running screen#

While the test runs the dashboard shows, updating every second:

• Active VUs against the profile curve.
• Requests / sec.
• Latency p50 / p95 / p99 in millis.
• Error rate (4xx + 5xx + network failures over total).
• In-flight requests.
• A rolling histogram of latency.

Stop halts the test cleanly; you still get a partial report.

22.5 Report#

When the test finishes (or you stop it) the report screen presents:

• Aggregate counters (total requests, error count, error %).
• Latency table — min / mean / p50 / p90 / p95 / p99 / max.
• Throughput chart over time.
• Error breakdown by HTTP status code and by network error class.
• The load curve that actually ran (sometimes diverges from the configured curve under heavy load).

Reports stay in the desktop app's local store while the workspace is open. Export to JSON for archival or to share with a teammate who wasn't on the call.

22.6 Pre / post hooks during perf runs#

The pre-request and post-request scripts you wrote on the original request do run on every iteration during perf mode. Be aware of:

• pm.environment.set writes go to the snapshot, not the live environment — perf runs don't mutate your normal workflow.
• Heavy work in the script (CryptoJS, big JSON parse) becomes the bottleneck. If you see CPU-bound latency, strip the script.
• pm.test assertions are evaluated but not displayed per iteration — they roll up into the error rate.

22.7 When perf mode is the wrong tool#

If you need any of these, use a scenario or a dedicated load tool:

• Multi-step flows (login → place order → check status). Perf mode is single-request only; orchestrate with a scenario then load-test the slowest endpoint inside it.
• Sustained load over many hours — perf mode runs locally and is capped by your laptop's bandwidth and CPU.
• Coordinated multi-machine load. Perf mode is one process.

22.8 Best practices#

• Warm up first. Run a 30-second Fixed profile at low VU count before the real test to populate caches and DNS.
• Isolate the environment. Perf-testing a request that mutates shared production state is a recipe for outages. Point at a dedicated load environment or a §21 mock.
• Test one variable at a time. Don't change profile + script + endpoint between runs — you won't know which one moved the numbers.
• Compare to a baseline. A p95 of 240 ms is meaningless without yesterday's number to compare against. Archive every report.

Where to next: §23 Pipeline gating & scenario priority.

23. Pipeline gating & scenario priority#

Every scenario carries a priority: critical, high, medium, or low. The default for both new and legacy scenarios is medium. Priority is purely a CI concept — it drives one decision: should a failed run flip the pipeline build to red?

23.1 Why this exists#

A team running 30 scenarios in CI was used to the first flaky non-critical scenario breaking deploys. Lowering reliability of the flaky scenario was not always possible, and disabling it lost coverage. The priority gate decouples signal (this scenario failed) from action (this build should fail).

After the rollout, only critical scenario failures break CI by default. Lower-priority failures still produce a red badge in the desktop UI and a Slack notification, but the pipeline keeps moving.

23.2 The four levels#

23.3 Setting a priority#

The priority badge is everywhere the scenario shows up:

• Sidebar card — small uppercase pill next to the version chip.
• Builder header — clickable badge opens a dropdown to switch the level (autosaves on change).
• Reports list / detail — every row carries the priority the scenario had when triggered, not its current value.
• HTML export — inline badge in the hero, preserved when you print to PDF.

Color tokens — critical red, high amber, medium blue, low muted — track the theme. Light and dark mode both work without per-theme tweaks.

23.4 Snapshot at trigger time#

Priority is snapshotted onto the report at trigger time. If you lower a scenario from critical to medium after a CI run already failed, the old report still reads critical and the CI build that recorded it still says "broke on critical". This is the right default: priority changes shouldn't rewrite history.

23.5 CLI gate — --fail-threshold#

callme scenario run accepts a --fail-threshold <level> flag with the same four levels. The default is critical. A failed run flips the exit code to 1 only when the scenario's priority is at or above the threshold:

0 means "passed or gated below threshold — pipeline keeps going". 1 means "failed loud — pipeline stops".

Examples:

# Default. Only critical scenario failures break the pipeline.
callme scenario run --scenario-id sc_... --token "$CALLME_TOKEN"

# Treat anything high or above as fatal.
callme scenario run --scenario-id sc_... --fail-threshold high \
  --token "$CALLME_TOKEN"

# Legacy behaviour — every failure breaks the pipeline.
callme scenario run --scenario-id sc_... --fail-threshold low \
  --token "$CALLME_TOKEN"

When a failure is gated out, the CLI prints a clear Pipeline not failed: scenario priority "low" is below threshold "critical" line so the operator knows the result wasn't silently ignored.

23.6 --no-fail escape hatch#

--no-fail overrides everything — the CLI always exits 0. Use it when you want to collect evidence (the report, the HTML export, the Slack notification) without blocking the build under any circumstances. This is the "soak run" mode.

23.7 Folder pipeline runs#

A single CLI invocation can fan out across every enabled scenario in a folder:

callme scenario run --folder-id flr_... --token "$CALLME_TOKEN"

The folder controls:

• Which scenarios participate. Sidebar → folder menu → Pipeline config opens a modal where you uncheck scenarios that should be skipped. The list is opt-out — a freshly added scenario starts enabled.
• Execution mode. Same modal: sequential (default — one at a time, ordered logs) or parallel (every scenario triggered at once, faster wall clock, higher backend load).

After all scenarios finish (or all polls return) the CLI prints a Folder summary: N passed, M failed line and aggregates the exit code: any failed scenario crossing --fail-threshold flips the folder run to 1. A timeout on any individual scenario also flips to 1, regardless of priority — the operator should know the run didn't actually complete.

Restrictions in folder mode:

• --scenario-id, --environment-id, --report, --output, and --no-wait are rejected upfront. Folder runs always poll all scenarios and don't emit a per-folder report file.
• Each scenario uses its own environments[0] as the default env. Override the env per scenario in the desktop app, not on the command line.

23.8 Three patterns#

Cautious main branch — --fail-threshold critical on every CI build. Anything high and below shows up in Slack but doesn't gate the release.

Strict release branch — --fail-threshold high on the release candidate pipeline. Tighten the gate when the stakes go up.

Coverage probe — --no-fail on a nightly run of every scenario in the workspace. Goal isn't to block anything; goal is to find what broke overnight so the morning standup has a punch list.

Where to next: §24 Query cookbook.

24. Query cookbook — every connection type, every syntax#

This chapter is the single reference for what you type into a query box — for all four databases (Oracle, MongoDB, PostgreSQL, MySQL), Redis, and Kafka. Every example here is copy-paste runnable and works in every place a query can live: the request DB / Redis / Kafka tabs, pm.db.query in scripts, and scenario DB / Redis / Kafka nodes — whether the run happens on your desktop or in a scheduled server-side scenario.

24.1 Where queries run#

Desktop and server share the same query engine: the same SQL pass-through, the same MongoDB JSON parser, the same five Redis commands, the same Kafka publish/listen mechanics, and the same {{...}} substitution. A query you debug on your desktop behaves the same inside a 03:00 scheduled run. The one known exception is a MongoDB ObjectId nicety — see §24.6.

Two practical consequences:

• The database host must be reachable from where the query runs. A localhost:5432 connection works on your laptop but not from the backend worker — use a host the server can reach for anything that ends up in a scenario or schedule.
• Timeouts are enforced server-side per scenario run (default 5 minutes) and per node (default 60 seconds); both are deployment-configurable.

24.2 Variables inside queries#

Every query box supports {{...}} substitution before execution:

Substitution is plain text replacement. No quoting, no escaping, no type conversion. Three rules follow from that:

1. Quote SQL strings yourself. WHERE id = '{{userId}}' — the quotes are yours. For numbers, leave them off: WHERE total > {{minTotal}}.
2. A missing variable becomes an empty string, so WHERE id = '{{typoedName}}' silently becomes WHERE id = ''. If a query matches nothing unexpectedly, print the resolved query first (pm.log(pm.variables.resolve("...your query..."))).
3. Values are not sanitised. Substitution is string splicing, so treat variable values as trusted test data — never feed user-supplied input into a query template.

In JSON queries (MongoDB), put tokens inside the JSON string values: "filter": { "userId": "{{userId}}" }. To splice a number, write "limit": {{pageSize}} without quotes — just make sure the variable really holds a number, or the JSON will fail to parse.

24.3 Oracle#

Raw SQL, one statement per query box, no trailing semicolon. There are no bind parameters for Oracle — use {{...}} substitution. Rows come back as objects keyed by column name (uppercase unless you quote an alias).

SELECT id, status, total
FROM orders
WHERE user_id = '{{userId}}'
ORDER BY created_at DESC
FETCH FIRST 5 ROWS ONLY

Join + aggregate:

SELECT c.name, COUNT(o.id) AS order_count, SUM(o.total) AS revenue
FROM customers c
JOIN orders o ON o.customer_id = c.id
WHERE o.created_at > SYSTIMESTAMP - INTERVAL '1' DAY
GROUP BY c.name
HAVING COUNT(o.id) > {{minOrders}}

Existence / freshness check (the classic post-response assertion):

SELECT COUNT(*) AS cnt
FROM audit_log
WHERE entity_id = '{{response.body.id}}'
  AND created_at > SYSTIMESTAMP - INTERVAL '5' MINUTE

Write operations:

INSERT INTO orders (id, user_id, status, total)
VALUES (orders_seq.NEXTVAL, '{{userId}}', 'PENDING', {{amount}})

UPDATE orders SET status = 'PAID' WHERE id = {{orderId}}

DELETE FROM orders WHERE user_id = '{{userId}}' AND status = 'TEST'

Every query auto-commits on success — same behaviour as PostgreSQL and MySQL. An INSERT in one node is immediately visible to a SELECT in a later node (or a later run). There is no rollback and no multi-statement transaction, so clean your test data up yourself — a DELETE node before the End node is the usual pattern.

What comes back, and how to reference it. A SELECT returns each row as an object keyed by column name — and Oracle uppercases unquoted identifiers, so the keys are ID, STATUS, TOTAL:

{
  "rows": [
    { "ID": 42, "STATUS": "PAID", "TOTAL": 99.5 },
    { "ID": 41, "STATUS": "PENDING", "TOTAL": 12 }
  ],
  "rowCount": 2
}

Downstream references from a node labelled check-order — uppercase column names, or quote an alias in the SQL (SELECT id AS "id") to keep it lowercase:

{{check-order.rows[0].ID}}        → 42
{{check-order.rows[0].STATUS}}    → PAID
{{check-order.rows[1].TOTAL}}     → 12
{{check-order.rowCount}}          → 2

24.4 PostgreSQL#

Raw SQL, one statement per query box. In scenario DB nodes you also get positional bind params — $1, $2, ... — filled from the node's params list, where each param value is itself template-resolved. On the request DB tab there is no params list, so use {{...}} substitution directly in the SQL.

SELECT id, status, total
FROM orders
WHERE user_id = '{{userId}}'
  AND created_at > now() - interval '5 minutes'
ORDER BY created_at DESC
LIMIT 5

Scenario node with params (params: ["{{userId}}", "PAID"]):

SELECT id, status FROM orders WHERE user_id = $1 AND status = $2

Insert a fixture and capture the generated id in one statement:

INSERT INTO orders (user_id, status, total)
VALUES ('{{userId}}', 'PENDING', {{amount}})
RETURNING id, created_at

RETURNING rows land in rows like a SELECT — downstream nodes can read {{seed-order.rows[0].id}}.

Update, delete, join, CTE:

UPDATE orders SET status = 'CANCELLED'
WHERE id = '{{orderId}}' AND status = 'PENDING'

DELETE FROM orders WHERE user_id = '{{userId}}' AND status = 'TEST'

SELECT o.id, o.total, c.email
FROM orders o
JOIN customers c ON c.id = o.customer_id
WHERE c.email = '{{email}}'

WITH recent AS (
  SELECT * FROM orders WHERE created_at > now() - interval '1 hour'
)
SELECT status, COUNT(*) AS cnt FROM recent GROUP BY status

What comes back, and how to reference it. PostgreSQL keeps unquoted identifiers lowercase, so row keys are id, status, ...

{ "rows": [ { "id": "ord_1", "status": "PAID" } ], "rowCount": 1 }

Downstream references, per query type (node labels in parentheses):

SELECT        (verify-order):  {{verify-order.rows[0].status}}   → PAID
                               {{verify-order.rowCount}}         → 1
INSERT…RETURNING (seed-order): {{seed-order.rows[0].id}}         → the generated id
                               {{seed-order.rows[0].created_at}} → 2026-06-11T09:30:00.000Z
UPDATE        (cancel-order):  {{cancel-order.rowCount}}         → rows actually updated
DELETE        (cleanup):       {{cleanup.rowCount}}              → rows actually deleted

{{cancel-order.rows[0].anything}} after a plain UPDATE resolves to an empty string — there are no rows unless you add RETURNING.

24.5 MySQL#

Raw SQL, one statement per query box. Bind params use ? placeholders — like PostgreSQL, the params list exists only on scenario DB nodes.

SELECT id, status, total
FROM orders
WHERE user_id = '{{userId}}'
  AND created_at > NOW() - INTERVAL 5 MINUTE
ORDER BY created_at DESC
LIMIT 5

Scenario node with params (params: ["{{userId}}", "PAID"]):

SELECT id, status FROM orders WHERE user_id = ? AND status = ?

Write operations:

INSERT INTO orders (user_id, status, total)
VALUES ('{{userId}}', 'PENDING', {{amount}})

UPDATE orders SET status = 'PAID' WHERE id = '{{orderId}}'

DELETE FROM orders WHERE user_id = '{{userId}}' AND status = 'TEST'

For SELECTs, rows holds the result and rowCount the row count. For DML, rows is empty and rowCount is affectedRows.

Gotcha — generated ids. Each query runs on a fresh connection that closes immediately, so a follow-up SELECT LAST_INSERT_ID() runs on a different connection and returns 0. MySQL has no RETURNING; query the row back by a business key instead:

SELECT id FROM orders
WHERE user_id = '{{userId}}' AND status = 'PENDING'
ORDER BY id DESC LIMIT 1

What comes back, and how to reference it. Row keys match your column names as written:

SELECT (verify-order):  {{verify-order.rows[0].id}}      → 42
                        {{verify-order.rows[0].status}}  → PAID
                        {{verify-order.rowCount}}        → 1
INSERT (seed-order):    {{seed-order.rowCount}}          → 1   (no rows — fetch the id with a follow-up SELECT)
UPDATE (mark-paid):     {{mark-paid.rowCount}}           → rows actually changed
DELETE (cleanup):       {{cleanup.rowCount}}             → rows actually deleted

Note: MySQL counts an UPDATE that changed nothing (values already equal) differently from one that matched nothing — rowCount is affected rows, so asserting rowCount == 1 after re-running the same UPDATE can fail even though the row exists.

24.6 MongoDB#

A MongoDB query is a strict JSON object — not shell syntax. db.orders.find({...}) is rejected with a parse error. The envelope:

{
  "collection": "orders",
  "operation": "find",
  "filter": {},
  "projection": {},
  "sort": {},
  "limit": 10,
  "skip": 0,
  "pipeline": [],
  "document": {},
  "update": {},
  "options": {}
}

collection and operation are required; everything else is operation-specific. Eight operations are supported. Anything else (insertMany, updateMany, distinct, drop, ...) is rejected.

find with the works:

{
  "collection": "orders",
  "operation": "find",
  "filter": { "userId": "{{userId}}", "status": { "$in": ["PAID", "SHIPPED"] } },
  "projection": { "_id": 0, "orderNo": 1, "status": 1, "total": 1 },
  "sort": { "createdAt": -1 },
  "limit": 10
}

findOne:

{
  "collection": "users",
  "operation": "findOne",
  "filter": { "email": "{{email}}" },
  "projection": { "passwordHash": 0 }
}

aggregate — the Mongo equivalent of a JOIN is a $lookup stage:

{
  "collection": "orders",
  "operation": "aggregate",
  "pipeline": [
    { "$match": { "userId": "{{userId}}" } },
    { "$lookup": {
        "from": "customers",
        "localField": "customerId",
        "foreignField": "_id",
        "as": "customer"
    } },
    { "$unwind": "$customer" },
    { "$group": {
        "_id": "$customer.email",
        "orderCount": { "$sum": 1 },
        "revenue": { "$sum": "$total" }
    } },
    { "$sort": { "revenue": -1 } },
    { "$limit": 5 }
  ]
}

insertOne / updateOne / deleteMany / countDocuments:

{
  "collection": "orders",
  "operation": "insertOne",
  "document": { "userId": "{{userId}}", "status": "PENDING", "total": {{amount}} }
}

{
  "collection": "orders",
  "operation": "updateOne",
  "filter": { "orderNo": "{{orderNo}}" },
  "update": { "$set": { "status": "PAID", "paidAt": "{{$isoTimestamp}}" } },
  "options": { "upsert": false }
}

{
  "collection": "orders",
  "operation": "deleteMany",
  "filter": { "userId": "{{userId}}", "status": "TEST" }
}

{
  "collection": "orders",
  "operation": "countDocuments",
  "filter": { "userId": "{{userId}}" }
}

What each operation returns, and how to reference it. Every operation — reads and writes — lands in the same { rows, rowCount } shape. For writes, rows[0] is the driver's acknowledgement object, so counters like modifiedCount are addressable like any column. Examples below assume the node label in parentheses:

find (list-orders) — documents in rows:

{ "rows": [ { "orderNo": "A-1001", "status": "PAID", "total": 99.5 } ], "rowCount": 1 }

{{list-orders.rows[0].orderNo}}        → A-1001
{{list-orders.rows[0].status}}         → PAID
{{list-orders.rows[0].customer.email}} → nested fields chain with dots
{{list-orders.rowCount}}               → 1

findOne (get-user) — zero or one document, still in rows:

{{get-user.rows[0].email}}     → empty string when nothing matched
{{get-user.rowCount}}          → 0 or 1 (branch on this in a condition node)

aggregate (revenue-by-customer) — one row per pipeline output document; with a $group, the grouping key sits in _id:

{{revenue-by-customer.rows[0]._id}}       → customer@example.com
{{revenue-by-customer.rows[0].revenue}}   → 1240.5
{{revenue-by-customer.rows[0].orderCount}} → 7

insertOne (seed-order):

{ "rows": [ { "acknowledged": true, "insertedId": "665f1c2ab1e4d3a7c9f0e812" } ], "rowCount": 1 }

{{seed-order.rows[0].insertedId}}   → the new document's id (hex string)

updateOne (mark-paid):

{ "rows": [ { "acknowledged": true, "matchedCount": 1, "modifiedCount": 1, "upsertedId": null } ], "rowCount": 1 }

{{mark-paid.rows[0].matchedCount}}   → 1 means the filter found the doc
{{mark-paid.rows[0].modifiedCount}}  → 0 means it was already in that state
{{mark-paid.rows[0].upsertedId}}     → set only when options.upsert created a doc

deleteOne / deleteMany (cleanup):

{ "rows": [ { "acknowledged": true, "deletedCount": 3 } ], "rowCount": 1 }

{{cleanup.rows[0].deletedCount}}   → 3

countDocuments (order-count):

{ "rows": [ { "count": 12 } ], "rowCount": 1 }

{{order-count.rows[0].count}}   → 12

Mind the write-operation rowCount trap: in scenario nodes (server) it mirrors the meaningful counter — modifiedCount for updateOne, deletedCount for deletes — but on the request DB tab (desktop) a write always reports rowCount = 1 (one acknowledgement row). The counters inside rows[0] (modifiedCount, deletedCount, ...) mean the same thing on both surfaces, so assert against those.

Three caveats worth pinning to the wall:

• ObjectId. On desktop runs, a 24-character hex string (or {"$oid": "..."}) in a filter is automatically converted to an ObjectId, so "filter": {"_id": "{{mongoId}}"} works. Server-side scenario runs do not apply this conversion — a hex string stays a string and never matches an ObjectId _id. For queries that will run in scenarios or schedules, filter on business keys (orderNo, email, ...) instead of _id.
• Dates. Strict JSON has no Date type. A string like "2026-06-11T00:00:00Z" in a filter compares as a string, which never matches a BSON Date field. If the collection stores BSON dates, range-filter on a different key (a numeric timestamp, a string date column) or assert recency in the Tests script instead.
• Output normalisation. Results are made JSON-safe: ObjectId → hex string, BSON Decimal128 → string, Date → ISO string, binary → hex. What you assert against is the normalised form.

24.7 Redis#

One command per line; each line returns one result. The supported set is exactly GET, SET, DEL, EXISTS, TTL (§9.2).

SET fixture:{{userId}} "{{random.uuid}}"
GET session:{{userId}}
EXISTS cart:{{userId}}
TTL session:{{userId}}
DEL fixture:{{userId}}

Syntax rules:

• Quote values containing spaces: SET greeting "hello world". Single quotes work too; escape an embedded quote as \".
• GET / DEL / EXISTS / TTL take exactly one key. DEL a b is an error, and there are no wildcards — DEL test:* deletes nothing; it targets the literal key test:*.
• Results: GET → the string or null; SET → the written value; DEL → number of removed keys; EXISTS → true/false; TTL → seconds (-1 = no expiry, -2 = no such key).

Storing structured data — write JSON as a quoted string, parse it where you read it:

SET order:{{orderId}} "{\"status\":\"PAID\",\"total\":99.5}"

const raw = pm.redis.results.post[0];
const order = JSON.parse(raw);
pm.expect(order.status).toBe("PAID");

What comes back, and how to reference it. In a scenario, a single-command node's output is the result itself; a multi-command node's output is an array indexed in command order. A node labelled check-cache running:

GET session:{{userId}}
EXISTS cart:{{userId}}
TTL session:{{userId}}
DEL fixture:{{userId}}

produces ["a1b2c3-session-token", true, 1740, 1], referenced as:

{{check-cache[0]}}   → a1b2c3-session-token   (GET — null if missing)
{{check-cache[1]}}   → true                   (EXISTS)
{{check-cache[2]}}   → 1740                   (TTL seconds)
{{check-cache[3]}}   → 1                      (DEL — keys removed)

With a single GET, skip the index: {{check-cache}} is the value. A condition node can branch directly on these — check-cache[1] == true, or check-cache[2] > 0. The same values are also available globally as {{redis.result}} (or {{redis.result[1]}} and so on) until the next Redis node runs.

24.8 Kafka#

Kafka has no query language in Callman — you either publish a message or listen for one (§10).

Publish (request Kafka tab or scenario Kafka node in publish mode). All fields are template-resolved, including the topic:

• Topic — orders.{{env.stage}}.events
• Key (optional) — {{orderId}}
• Value — usually JSON:

  {
    "type": "ORDER_CREATED",
    "orderId": "{{create-order.response.body.id}}",
    "occurredAt": "{{$isoTimestamp}}",
    "traceId": "{{random.uuid}}"
  }
  

• Headers — key/value pairs, both template-resolved.

When the value format is JSON, the payload is validated after substitution — a token that resolved to nothing fails fast with "payload is not valid JSON" instead of publishing garbage. The node output is { topic, partition, offset, timestamp }.

Listen / capture (consume mode): subscribe to a topic for a timeout window and pick the matching message. With no rules at all, the first captured message wins. Otherwise two match modes exist — and while a message is being tested, that candidate message is exposed to your rules as kafkaEvent.

What kafkaEvent looks like. If the message value parses as a JSON object, its fields sit at the top level and the transport metadata moves under meta. For this published payload:

{ "type": "ORDER_CREATED", "orderId": "ord_42", "amount": 99.5,
  "items": [ { "sku": "ABC", "qty": 2 } ] }

the rule-visible structure is:

kafkaEvent.type                → ORDER_CREATED
kafkaEvent.orderId             → ord_42
kafkaEvent.amount              → 99.5
kafkaEvent.items[0].sku        → ABC          (bracket indices)
kafkaEvent.meta.key            → the message key
kafkaEvent.meta.partition      → 0
kafkaEvent.meta.offset         → 1547
kafkaEvent.meta.timestamp      → ISO timestamp
kafkaEvent.meta.headers.trace-id → header value, by header name

If the value is not a JSON object (plain text, a number, a JSON array), there is no spreading — the fields are kafkaEvent.value (parsed value or raw text), kafkaEvent.text (raw text), kafkaEvent.key, kafkaEvent.partition, kafkaEvent.offset, kafkaEvent.timestamp, kafkaEvent.headers.<name>.

Rules mode — what you can write. Each rule is one comparison:

<left> <operator> <right>

with exactly one operator per rule, chosen from:

A message matches only when every rule in the list passes (the rules are AND-ed; for OR, listen loosely and branch with a condition node afterwards). Each side of a rule can be:

• a path — kafkaEvent.orderId, kafkaEvent.meta.key, kafkaEvent.items[0].sku (bare, no braces needed);
• a template — {{create-order.response.body.id}}, {{env.stage}}, any prior node output or variable;
• a literal — ORDER_CREATED, "ORDER CREATED" (quotes are optional and stripped), 99.5, true, false, null.

Realistic rule sets:

kafkaEvent.type == ORDER_CREATED
kafkaEvent.orderId == {{create-order.response.body.id}}

kafkaEvent.correlationId == {{corr}}
kafkaEvent.amount > 0
kafkaEvent.meta.headers.source contains payment

There are no exists / in / regex operators in Kafka rules (those belong to the condition node) — to express "field exists", write {{kafkaEvent.field}} != "". The braces matter for that trick: a brace-wrapped missing path resolves to an empty string, while a bare token that matches nothing is treated as a literal string (so kafkaEvent.missing != "" would always pass).

Regex mode — what it matches against. The first rule line is treated as a regular expression and tested against the entire serialised message — JSON.stringify of { key, valueText, valueJson, partition, offset, timestamp, headers } — so it can hit the payload, the key, or a header value alike. Write it bare or as a /pattern/flags literal:

ORDER_CREATED
/"orderId":\s*"ord_[0-9a-f]+"/
/order_created/i

Regex mode is a substring shotgun — quick, but it can't compare a field against a variable's value numerically or distinguish "amount": 99 in the payload from the same text inside a header. When you care about which field holds the value, use rules mode.

After the match — the matched event becomes the node's output (same shape as the kafkaEvent structure above, label-addressed), and stays globally available as {{kafkaEvent.*}} until the next Kafka node runs:

{{order-event.orderId}}            ← payload field
{{order-event.items[0].sku}}
{{order-event.meta.offset}}
{{kafkaEvent.orderId}}             ← same event, fixed root

If nothing matches within the timeout, the node fails with "timed out without a matching event" and downstream nodes follow the failure policy.

24.9 Using query results downstream#

On a request (desktop) — results feed the declarative assertion rows and the script API. Assertion field expressions use dot-notation paths, including dot-indexed arrays:

db_post.rowCount
db_post.firstRow.status
db_post.rows.0.amount
redis.post.0

and in scripts:

const res = await pm.db.query({
  connectionId: "ordersPrimary",
  query: "SELECT status FROM orders WHERE id = '{{orderId}}'",
});
pm.expect(res.firstRow().status).toBe("PAID");

In a scenario (server) — downstream nodes reference a prior node's output by its label, and array indices use brackets:

{{verify-order.rows[0].status}}     ← DB node
{{verify-order.rowCount}}
{{check-cache}}                     ← single-command Redis node
{{order-event.orderId}}             ← Kafka consume node
{{seed-order.rows[0].id}}           ← Postgres INSERT ... RETURNING

The two index syntaxes are not interchangeable: request assertion fields use rows.0.amount, scenario references use rows[0].amount.

A typical chain: a DB node labelled seed-order inserts a fixture with RETURNING id, the next request node posts to /orders/{{seed-order.rows[0].id}}/pay, a condition node branches on the builder rule verify-order.rows[0].status == "PAID" (no braces in condition fields), and a notification node interpolates both into its Slack message.

24.10 Limits and gotchas#

• One statement per query. No semicolon-separated batches, no stored-procedure scripts. Need two queries? Two nodes, or two pre/post lines.
• No transactions. Every query opens a fresh connection, auto-commits on success, and closes. There is no way to span a transaction across two queries, no rollback, and no session state between queries (temp tables, LAST_INSERT_ID(), session variables). Writes are permanent the moment the node succeeds — clean up your test data at the end of the scenario.
• Result normalisation. Everything is made JSON-safe before you see it: dates → ISO-8601 strings, BigInt → string, binary → hex, Mongo ObjectId → hex string. Assert against the normalised values.
• Timeouts. A scenario run is capped (default 5 minutes) and each node is capped (default 60 seconds). A query that needs longer than a minute belongs in a migration, not a test.
• Reachability. Server-side runs connect from the backend worker. Connection definitions are fetched fresh per run, so credential rotation is picked up automatically — but a host that only resolves on your laptop will fail at 03:00.

Where to next: Appendix A.

Appendix A — pm.* API index#

Alphabetical, every public method. Section number points to where the API is explained in context. Top-level globals (CryptoJS, _, setTimeout, ...) are documented at the end of the table.

Top-level globals (no pm. prefix)#

Appendix B — Glossary#

• Aggregator — a scenario node that merges several prior outputs into one structured value.
• Assertion — a check that something is true. Two kinds in Callman: scripted (pm.expect) and declarative (Contract / DB / Kafka / Redis rules).
• Branch — one of the outgoing edges from a Condition node (yes or no) or any outgoing edge from a node with multiple fan-out edges.
• Collection — a folder of related requests.
• Connection — a saved set of credentials and parameters for a database, Kafka cluster, Redis instance, or Slack workspace.
• Contract — declarative response-shape rules (path + type + regex + enum).
• Edge — a directed connection between two scenario nodes. Types: default, yes, no.
• Environment — a named variable scope. Workspaces can have many; one is active at a time.
• Folder — a subdivision inside a collection.
• Generator — a dynamic token like random.uuid or counter.
• Globals — workspace-level variables visible from every environment.
• Iteration — one row of a Data-Driven Run.
• Node — one box on the scenario canvas.
• PAT — Personal Access Token. Workspace-scoped credential for CLI / API access. Prefix cm_pat_.
• Pre-request script — JavaScript that runs before the HTTP call is sent.
• Report — the persisted outcome of a scenario run.
• Request — the canonical unit of work. HTTP + scripts + contract + integrations.
• Run — one execution of a request, collection, or scenario.
• Scenario — a directed graph of typed nodes that orchestrates HTTP requests and integration checks.
• Schedule — a recurring trigger for a scenario.
• Scope — the merged variable map (per-iteration + env + globals).
• Sub-scenario — a scenario embedded as a single node inside another scenario.
• Tests script — JavaScript that runs after the response arrives.
• Workspace — the unit of collaboration. Holds collections, environments, scenarios, members.

End of document.