Module: crawler

Autonomous QA pipeline — thin orchestration layer for the 8-stage test generation pipeline.

Pipeline stages

# Stage Module
1 Smart crawl / Explore pipeline/crawlBrowser.js or pipeline/stateExplorer.js
↳ HAR capture pipeline/harCapture.js (attached to BrowserContext)
2 Element filtering pipeline/elementFilter.js
3 Intent classification pipeline/intentClassifier.js
4 Journey generation pipeline/journeyGenerator.js
4b API test generation pipeline/journeyGenerator.js + prompts/apiTestPrompt.js
5 Deduplication pipeline/pipelineOrchestrator.js
6 Assertion enhancement pipeline/pipelineOrchestrator.js
7 Validate tests pipeline/pipelineOrchestrator.js
8 Feedback loop pipeline/feedbackLoop.js

Explorer modes (Test Dials exploreMode)

  • crawl (default) — link-only BFS crawl via crawlBrowser.js
  • state — state-based exploration via stateExplorer.js that executes real UI actions (click, fill, submit) and tracks state transitions to discover multi-step user flows

Exports

  • generateFromUserDescription — Generate test(s) from a user description (skips crawl).
  • crawlAndGenerateTests — Full 8-stage pipeline from URL crawl or state exploration.
Source:

Methods

(static) crawlAndGenerateTests(project, run, optionsopt) → {Promise.<void>}

Full 8-stage pipeline: crawl a project URL, classify pages, generate tests, deduplicate, enhance, validate, and persist.

Parameters:
Name Type Attributes Description
project Object

The project { id, name, url, credentials? }.
run Object

The run record (mutated in place with results).
options Object <optional>
Properties
Name Type Attributes Description
dialsPrompt string <optional>

Pre-built prompt fragment from Test Dials config.
testCount string <optional>

Test count hint ("one" | "small" | "medium" | "large" | "ai_decides").
explorerMode string <optional>

"crawl" (default) or "state" — from Test Dials.
explorerTuning Object <optional>

Numeric tuning for state explorer { maxStates, maxDepth, maxActions, actionTimeout }.
signal AbortSignal <optional>

Abort signal for cancellation.
Source:
Returns:
Type
Promise.<void>

(static) generateFromUserDescription()

generateFromUserDescription — Generates test(s) from a user-provided name + description (no crawl needed).

Uses a dedicated AI prompt that produces tests matching the user's stated intent. The number of tests is controlled by the testCount dial (1–20, default "one"). Unlike the crawl pipeline which discovers pages automatically, this skips Steps 1-3 and goes straight to AI generation.

Pipeline: Step 1-3: SKIPPED (Crawl, Filter, Classify — user provides intent directly) Step 4: Generate — AI generates test(s) from name + description Step 5: Deduplicate — Check against existing project tests Step 6: Enhance — Strengthen assertions Step 7: Validate — Reject malformed / placeholder tests Step 8: Done

Source:

(async, inner) filterAndClassify(snapshots, snapshotsByUrl, project, run, signalopt) → {Promise.<{filteredSnapshots: Array.<object>, classifiedPages: Array.<object>, classifiedPagesByUrl: Record.<string, object>}>}

Shared Steps 2 & 3: Element filtering + intent classification. Extracted to avoid duplication between the "state" and "crawl" branches.

Parameters:
Name Type Attributes Description
snapshots Array.<object>

— raw page snapshots from crawl or explore
snapshotsByUrl Record.<string, object>

— URL → snapshot map (mutated in place)
project object

— project record (url used for log trimming)
run object

— mutable run record
signal AbortSignal <optional>
Source:
Returns:
Type
Promise.<{filteredSnapshots: Array.<object>, classifiedPages: Array.<object>, classifiedPagesByUrl: Record.<string, object>}>