skills/playwright-dev/trace_system_guide.md

Playwright Trace System - Comprehensive Guide

1. Overview

The Playwright trace system is a comprehensive recording and visualization framework that captures:

• Actions (API calls, user interactions)
• Network traffic (HAR format)
• Snapshots (DOM snapshots at key moments)
• Screencast frames (video of page rendering)
• Console messages and events
• Errors and logs
• Resources (images, stylesheets, scripts, etc.)

---

2. File Structure

packages/trace/src/ - Trace Type Definitions

Located in /home/pfeldman/code/playwright/packages/trace/src/

Key Files:

• trace.ts - Core trace event type definitions
• har.ts - HTTP Archive format (network traffic)
• snapshot.ts - DOM snapshot data structures
• DEPS.list - Dependencies marker

File List:

trace/src/
├── trace.ts        (183 lines) - Main trace event types
├── har.ts          (189 lines) - HAR format types
├── snapshot.ts     (62 lines)  - Snapshot data structures
└── DEPS.list       - Dependencies file

---

3. Trace Event Types (trace.ts)

3.1 Core Event Types

VERSION: 8 (Current format version)

ContextCreatedTraceEvent

type ContextCreatedTraceEvent = {
  version: number,
  type: 'context-options',
  origin: 'testRunner' | 'library',
  browserName: string,
  channel?: string,
  platform: string,
  playwrightVersion?: string,
  wallTime: number,  // Milliseconds since epoch
  monotonicTime: number,  // Internal monotonic clock
  title?: string,
  options: BrowserContextEventOptions,
  sdkLanguage?: Language,
  testIdAttributeName?: string,
  contextId?: string,
  testTimeout?: number,
};

BeforeActionTraceEvent

Emitted when an action starts:

type BeforeActionTraceEvent = {
  type: 'before',
  callId: string,           // Unique action identifier
  startTime: number,        // Monotonic time when action started
  title?: string,           // User-facing action name
  class: string,            // API class (e.g., 'Page', 'Frame')
  method: string,           // API method (e.g., 'click', 'goto')
  params: Record<string, any>,  // Method parameters
  stepId?: string,          // Test step identifier
  beforeSnapshot?: string,  // "before@<callId>"
  stack?: StackFrame[],     // Call stack
  pageId?: string,          // Associated page ID
  parentId?: string,        // Parent action (for nested actions)
  group?: string,           // Action group (e.g., 'wait', 'click')
};

InputActionTraceEvent

For input/pointer interactions:

type InputActionTraceEvent = {
  type: 'input',
  callId: string,
  inputSnapshot?: string,   // "input@<callId>"
  point?: Point,            // Mouse/pointer coordinates
};

AfterActionTraceEvent

Emitted when an action completes:

type AfterActionTraceEvent = {
  type: 'after',
  callId: string,
  endTime: number,          // Monotonic time when action ended
  afterSnapshot?: string,   // "after@<callId>"
  error?: SerializedError,  // Error if action failed
  attachments?: AfterActionTraceEventAttachment[],  // Files, screenshots
  annotations?: AfterActionTraceEventAnnotation[],  // Custom annotations
  result?: any,             // Return value
  point?: Point,            // Final pointer position
};

ActionTraceEvent (Composite)

Combines before, after, and input events:

type ActionTraceEvent = {
  type: 'action',
} & Omit<BeforeActionTraceEvent, 'type'>
  & Omit<AfterActionTraceEvent, 'type'>
  & Omit<InputActionTraceEvent, 'type'>;

Other Event Types

ScreencastFrameTraceEvent - Video frame data

type ScreencastFrameTraceEvent = {
  type: 'screencast-frame',
  pageId: string,
  sha1: string,             // Resource SHA1
  width: number,            // Frame width
  height: number,           // Frame height
  timestamp: number,        // Frame timestamp
  frameSwapWallTime?: number,
};

EventTraceEvent - Browser events (dialog, navigation, etc.)

type EventTraceEvent = {
  type: 'event',
  time: number,
  class: string,            // Event source class
  method: string,           // Event method
  params: any,              // Event parameters
  pageId?: string,
};

ConsoleMessageTraceEvent - Console output

type ConsoleMessageTraceEvent = {
  type: 'console',
  time: number,
  pageId?: string,
  messageType: string,      // 'log', 'error', 'warn', etc.
  text: string,
  args?: { preview: string, value: any }[],
  location: { url: string, lineNumber: number, columnNumber: number },
};

LogTraceEvent - Action logs

type LogTraceEvent = {
  type: 'log',
  callId: string,
  time: number,
  message: string,
};

ResourceSnapshotTraceEvent - Network request

type ResourceSnapshotTraceEvent = {
  type: 'resource-snapshot',
  snapshot: ResourceSnapshot,  // HAR Entry
};

FrameSnapshotTraceEvent - DOM snapshot

type FrameSnapshotTraceEvent = {
  type: 'frame-snapshot',
  snapshot: FrameSnapshot,
};

StdioTraceEvent - Process output (stdout/stderr)

type StdioTraceEvent = {
  type: 'stdout' | 'stderr',
  timestamp: number,
  text?: string,
  base64?: string,  // Binary output
};

ErrorTraceEvent - Unhandled errors

type ErrorTraceEvent = {
  type: 'error',
  message: string,
  stack?: StackFrame[],
};

---

4. HAR Format (har.ts)

Follows HTTP Archive 1.2 specification. Key structure for network traffic:

type HARFile = {
  log: Log,
};

type Log = {
  version: string,
  creator: Creator,
  browser?: Browser,
  pages?: Page[],
  entries: Entry[],  // Network requests
};

type Entry = {
  pageref?: string,
  startedDateTime: string,
  time: number,  // Total time (ms)
  request: Request,
  response: Response,
  cache: Cache,
  timings: Timings,
  serverIPAddress?: string,
  connection?: string,
  // Custom Playwright fields:
  _frameref?: string,
  _monotonicTime?: number,
  _serverPort?: number,
  _securityDetails?: SecurityDetails,
  _wasAborted?: boolean,
  _wasFulfilled?: boolean,
  _wasContinued?: boolean,
  _apiRequest?: boolean,  // True for fetch/axios requests
};

---

5. Snapshot Format (snapshot.ts)

FrameSnapshot

type FrameSnapshot = {
  snapshotName?: string,
  callId: string,           // Associated action
  pageId: string,
  frameId: string,
  frameUrl: string,
  timestamp: number,
  wallTime?: number,
  collectionTime: number,   // Time to capture
  doctype?: string,
  html: NodeSnapshot,       // Encoded DOM tree
  resourceOverrides: ResourceOverride[],  // Inlined resources
  viewport: { width: number, height: number },
  isMainFrame: boolean,
};

NodeSnapshot

Compact encoding of DOM tree:

type NodeSnapshot =
  TextNodeSnapshot |                    // string
  SubtreeReferenceSnapshot |            // [ [snapshotIndex, nodeIndex] ]
  NodeNameAttributesChildNodesSnapshot; // [ name, attributes?, ...children ]

ResourceOverride

Embeds resource data in snapshot:

type ResourceOverride = {
  url: string,
  sha1?: string,  // External resource SHA1
  ref?: number    // Snapshot index reference
};

---

6. Trace Storage Format

File Structure

When a trace is recorded, it creates this structure in the traces directory:

traces-dir/
├── <traceName>.trace        # Main events (JSONL format)
├── <traceName>.network      # Network events (JSONL format)
├── <traceName>-chunk1.trace # Additional chunks (if multiple)
├── <traceName>.stacks       # Stack trace metadata (optional)
└── resources/
    ├── <sha1>               # Resource files (images, etc.)
    └── <sha1>

File Formats

• .trace and .network: JSONL (JSON Lines) - one event per line
• .zip: Optional archive containing all above files
• resources/: Binary blobs indexed by SHA1 hash

Live Trace Format

For live tracing (test runner):

traces-dir/
├── <testName>.json     # Synthesized trace metadata
├── <testName>/
    ├── events.jsonl
    ├── network.jsonl
    └── resources/

---

7. Trace Recording (tracing.ts)

Located: /home/pfeldman/code/playwright/packages/playwright-core/src/server/trace/recorder/tracing.ts

Tracing Class Architecture

export class Tracing extends SdkObject implements 
  InstrumentationListener,
  SnapshotterDelegate,
  HarTracerDelegate {
  
  // Recording state
  private _state: RecordingState;
  
  // Components
  private _snapshotter?: Snapshotter;      // Captures DOM snapshots
  private _harTracer: HarTracer;           // Records network requests
  private _screencastListeners: ...        // Video recording
  
  // Methods
  start(options: TracerOptions);
  startChunk(progress, options);
  stopChunk(progress, params);
  stop(progress);
}

What Gets Recorded

1. Before Action (onBeforeCall)

• Action metadata: class, method, parameters
• Stack trace
• "before" DOM snapshot
• Associated page/frame IDs

2. Input Actions (onBeforeInputAction)

• Pointer coordinates
• Input type
• Snapshot of input

3. Action Logs (onCallLog)

• API log messages
• User-facing messages

4. After Action (onAfterCall)

• Execution time
• Return value
• Error information (if failed)
• "after" DOM snapshot
• Attachments (screenshots, files)
• Annotations (custom data)

5. Network Traffic (onEntryFinished)

• HTTP request/response details
• Headers, cookies, body
• Timing information
• Security details

6. Snapshots (onFrameSnapshot, onSnapshotterBlob)

• Full DOM tree with inlined resources
• Viewport size
• Resource references

7. Console Messages (onConsoleMessage)

• Message type (log, error, warn)
• Text content
• Arguments
• Source location

8. Events

• Dialogs
• Page errors
• Navigation events
• Downloads

9. Screencast Frames

• Video frames (if screenshots enabled)
• Frame dimensions
• Timestamps

10. Stdio/Errors

• stdout/stderr output
• Unhandled errors
• Process events

Recording State

type RecordingState = {
  options: TracerOptions,
  traceName: string,
  networkFile: string,
  traceFile: string,
  tracesDir: string,
  resourcesDir: string,
  chunkOrdinal: number,
  networkSha1s: Set<string>,
  traceSha1s: Set<string>,
  recording: boolean,
  callIds: Set<string>,
  groupStack: string[],  // For nested groups
};

---

8. Trace Loading (traceLoader.ts)

Located: /home/pfeldman/code/playwright/packages/playwright-core/src/utils/isomorphic/trace/traceLoader.ts

TraceLoaderBackend Interface

interface TraceLoaderBackend {
  entryNames(): Promise<string[]>;           // List files in trace
  hasEntry(entryName: string): Promise<boolean>;
  readText(entryName: string): Promise<string | undefined>;  // For JSONL
  readBlob(entryName: string): Promise<Blob | undefined>;    // For resources
  isLive(): boolean;  // Is this a live/developing trace?
}

Built-in Backends

ZipTraceLoaderBackend (traceParser.ts)

• Loads .trace.zip files
• Uses ZipFile utility to read entries
• Converts file paths to file:// URLs

Load Process

async load(backend: TraceLoaderBackend, unzipProgress) {
  1. Find .trace files (ordinals: "0", "1", etc.)
  2. For each ordinal:
     a. Read ordinal.trace (events)
     b. Read ordinal.network (network events)
     c. Parse with TraceModernizer
     d. Read ordinal.stacks (if exists)
     e. Sort actions by startTime
  3. Terminate incomplete actions
  4. Finalize snapshot storage
  5. Build resource content-type map
  6. Push ContextEntry to contextEntries[]
}

Output: ContextEntry[]

type ContextEntry = {
  origin: 'testRunner' | 'library',
  startTime: number,      // Min action startTime
  endTime: number,        // Max action endTime
  browserName: string,
  wallTime: number,
  sdkLanguage?: Language,
  testIdAttributeName?: string,
  title?: string,
  options: BrowserContextEventOptions,
  pages: PageEntry[],      // Screencast data
  resources: ResourceSnapshot[],  // HAR entries
  actions: ActionEntry[],   // Merged before/after events
  events: EventTraceEvent[],
  stdio: StdioTraceEvent[],
  errors: ErrorTraceEvent[],
  hasSource: boolean,
  contextId: string,
  testTimeout?: number,
};

---

9. Trace Model (traceModel.ts)

Located: /home/pfeldman/code/playwright/packages/playwright-core/src/utils/isomorphic/trace/traceModel.ts

TraceModel Class

High-level data model for trace viewer:

class TraceModel {
  // Metadata
  startTime: number;
  endTime: number;
  browserName: string;
  channel?: string;
  platform?: string;
  playwrightVersion?: string;
  wallTime?: number;
  title?: string;
  options: BrowserContextEventOptions;
  sdkLanguage: Language;
  testIdAttributeName?: string;
  traceUri: string;  // URL to trace
  testTimeout?: number;

  // Data arrays
  pages: PageEntry[];          // Page screencast data
  actions: ActionTraceEventInContext[];  // All recorded actions
  attachments: Attachment[];   // Screenshots, files
  visibleAttachments: Attachment[];  // Non-private attachments
  events: (EventTraceEvent | ConsoleMessageTraceEvent)[];
  stdio: StdioTraceEvent[];
  errors: ErrorTraceEvent[];
  resources: ResourceEntry[];  // Network resources
  sources: Map<string, SourceModel>;  // Source code
  errorDescriptors: ErrorDescription[];  // Parsed errors
  
  // Counters
  actionCounters: Map<string, number>;  // Actions per group
  hasSource: boolean;         // Has source code available
  hasStepData: boolean;       // Has test runner data
  
  // Methods
  createRelativeUrl(path: string): string;
  failedAction(): ActionTraceEventInContext;
  filteredActions(actionsFilter: ActionGroup[]): ActionTraceEventInContext[];
}

ActionTraceEventInContext

type ActionTraceEventInContext = ActionEntry & {
  context: ContextEntry,
  group?: ActionGroup,  // Added by TraceModel
  log: { time: number, message: string }[],
};

---

10. Trace Modernizer (traceModernizer.ts)

Located: /home/pfeldman/code/playwright/packages/playwright-core/src/utils/isomorphic/trace/traceModernizer.ts

Version Support

• Latest: Version 8
• Supported: Versions 3-8
• Upgrades older traces to current format

TraceModernizer Class

class TraceModernizer {
  constructor(contextEntry: ContextEntry, snapshotStorage: SnapshotStorage);
  
  appendTrace(trace: string);  // Parse JSONL trace lines
  actions(): ActionEntry[];    // Get parsed actions
  
  private _modernize(event: any);  // Upgrade event to latest version
  private _innerAppendEvent(event: TraceEvent);  // Process event
}

How It Works

1. Parses JSONL (one JSON object per line)
2. Detects trace version from first context-options event
3. Applies version-specific upgrades using _modernize_N_to_N+1() functions
4. Consolidates before/after events into unified actions
5. Builds dependency graph for nested actions
6. Stores snapshots in SnapshotStorage

---

11. Trace Viewer

Located: /home/pfeldman/code/playwright/packages/trace-viewer/src/

Structure

trace-viewer/src/
├── index.tsx              # Entry point
├── sw-main.ts             # Service worker
└── ui/
    ├── workbench.tsx      # Main UI component
    ├── actionList.tsx     # Action timeline
    ├── callTab.tsx        # Action details
    ├── snapshotTab.tsx    # DOM snapshot viewer
    ├── networkTab.tsx     # Network waterfall
    ├── consoleTab.tsx     # Console messages
    ├── timeline.tsx       # Time-based view
    ├── filmStrip.tsx      # Video frames
    ├── logTab.tsx         # Action logs
    ├── attachmentsTab.tsx # Files, screenshots
    ├── playbackControl.tsx # Video playback
    └── [other tabs...]

Data Flow

1. Service Worker (sw-main.ts) - Intercepts trace URL fetch
2. Workbench Loader - Loads trace via TraceLoader
3. TraceModel - Parses and indexes loaded data
4. UI Components - Display actions, snapshots, network, etc.
5. Playback Control - Synchronizes timeline with snapshots

Key Data Models

• TraceModel - Loaded and parsed trace data
• ActionTraceEventInContext - Single action with context
• Attachment - File or screenshot data
• SourceModel - Source code + errors

---

12. CLI Commands

Located: /home/pfeldman/code/playwright/packages/playwright-core/src/cli/program.ts

show-trace Command

playwright show-trace [trace] [options]

Options:
  -b, --browser <browserType>   Browser to use (chromium, firefox, webkit)
  -h, --host <host>            Host to serve on
  -p, --port <port>            Port to serve on (0 = any free port)
  --stdin                       Accept trace URLs over stdin
  
Examples:
  $ show-trace
  $ show-trace https://example.com/trace.zip
  $ show-trace /path/to/trace.zip
  $ show-trace /path/to/trace/dir

Implementation (program.ts: 327-355)

program
  .command('show-trace [trace]')
  .option('-b, --browser <browserType>', ..., 'chromium')
  .option('-h, --host <host>', 'Host to serve trace on')
  .option('-p, --port <port>', 'Port to serve trace on')
  .option('--stdin', 'Accept trace URLs over stdin')
  .description('show trace viewer')
  .action(function(trace, options) {
    const openOptions: TraceViewerServerOptions = {
      host: options.host,
      port: +options.port,
      isServer: !!options.stdin,
    };
    
    if (options.port !== undefined || options.host !== undefined)
      runTraceInBrowser(trace, openOptions);  // Opens in browser tab
    else
      runTraceViewerApp(trace, options.browser, openOptions);  // Opens in app window
  });

Trace Viewer Server (traceViewer.ts)

startTraceViewerServer(options?: TraceViewerServerOptions): Promise<HttpServer>
  // Routes:
  // GET /trace/file?path=<filePath>        → Serve trace file
  // GET /trace/file?path=<path>.json       → Synthesize trace metadata
  // GET /trace/file?path=<traceDir>/...    → Serve trace.dir contents
  // GET /trace/<other>                     → Serve viewer assets

runTraceViewerApp(traceUrl, browserName, options)
  // Opens trace viewer in persistent browser context

runTraceInBrowser(traceUrl, options)
  // Opens trace viewer in browser tab (tab.open)

---

13. Data Available Per Action

Per-Action Data Structure

ActionTraceEventInContext {
  // Identifiers
  callId: string;           // Unique action ID
  pageId?: string;          // Associated page
  parentId?: string;        // Parent action (nested)
  stepId?: string;          // Test step ID
  group?: ActionGroup;      // Action category
  
  // Timing
  startTime: number;        // Monotonic time (milliseconds)
  endTime: number;          // When action completed
  
  // API Information
  class: string;            // Class name (Page, Frame, etc.)
  method: string;           // Method name (click, goto, etc.)
  params: Record<string, any>;  // Input parameters
  result?: any;             // Return value
  
  // Code Location
  stack?: StackFrame[];     // Call stack with file/line/column
  title?: string;           // User-facing name
  
  // Snapshots
  beforeSnapshot?: string;  // "before@<callId>" reference
  inputSnapshot?: string;   // "input@<callId>" reference
  afterSnapshot?: string;   // "after@<callId>" reference
  
  // Errors
  error?: SerializedError;  // Error message and stack
  
  // Logging
  log: { time: number, message: string }[];  // Action logs
  
  // Attachments
  attachments?: AfterActionTraceEventAttachment[];
  // { name, contentType, path?, sha1?, base64? }
  
  // Annotations
  annotations?: AfterActionTraceEventAnnotation[];
  // { type, description? }
  
  // Interaction Details
  point?: Point;            // Pointer coordinates {x, y}
  
  // Reference
  context: ContextEntry;    // Associated browser context
}

Snapshot Data

Each snapshot can be accessed via TraceLoader.storage():

FrameSnapshot {
  callId: string,              // Associated action
  pageId: string,
  frameId: string,
  frameUrl: string,
  html: NodeSnapshot,          // Encoded DOM tree
  resourceOverrides: [         // Embedded resources
    { url, sha1?, ref? }
  ],
  viewport: { width, height },
  isMainFrame: boolean,
  collectionTime: number,      // ms to capture
  timestamp: number,           // Monotonic time
  wallTime?: number,
}

Network Data (HAR Entry)

Entry {
  request: {
    method: string,        // GET, POST, etc.
    url: string,
    httpVersion: string,
    headers: Header[],
    cookies: Cookie[],
    queryString: { name, value }[],
    postData?: {
      mimeType: string,
      params: Param[],
      text: string,
      _sha1?: string,      // Reference to resources/
    },
  },
  response: {
    status: number,        // 200, 404, etc.
    statusText: string,
    headers: Header[],
    cookies: Cookie[],
    content: {
      size: number,
      mimeType: string,
      text?: string,
      _sha1?: string,      // Reference to resources/
      compression?: number,
    },
    redirectURL: string,
  },
  timings: {               // All in milliseconds
    blocked?: number,
    dns?: number,
    connect?: number,
    send: number,
    wait: number,          // Time to first byte
    receive: number,
    ssl?: number,
  },
  time: number,            // Total time
  _monotonicTime?: number, // Monotonic timestamp
  _wasFulfilled?: boolean,
  _wasAborted?: boolean,
  _apiRequest?: boolean,   // fetch/axios
}

---

14. Quick Reference: Accessing Trace Data

In Trace Viewer

// Load trace
const traceLoader = new TraceLoader();
const backend = new ZipTraceLoaderBackend('trace.zip');
await traceLoader.load(backend, (done, total) => {});

// Access context
const contextEntries = traceLoader.contextEntries;

// Get trace model
const traceModel = new TraceModel(traceUri, contextEntries);

// Iterate actions
for (const action of traceModel.actions) {
  console.log(action.method);        // e.g., "click"
  console.log(action.params);        // parameters
  console.log(action.result);        // return value
  console.log(action.error);         // error if failed
  console.log(action.log);           // log messages
}

// Get snapshots
const snapshotStorage = traceLoader.storage();
const snapshot = snapshotStorage.snapshotByName('before@<callId>');

// Get resource
const blob = await traceLoader.resourceForSha1(sha1);

In Test Runner

// Access via trace via browser context
const trace = await context.tracing.stop({ path: 'trace.zip' });

// Use server-side Tracing class
const tracing = new Tracing(context, tracesDir);
tracing.start({ snapshots: true, screenshots: true });
// ... run test ...
await tracing.stopChunk(progress, { mode: 'archive' });
await tracing.stop(progress);

---

15. Version Information

Trace Format Versions

• Version 3: Early format (~1.35)
• Version 4: Updates (~1.36)
• Version 5: Improvements (~1.37)
• Version 6: Major changes (~10/2023, ~1.40)
• Version 7: Further updates (~05/2024, ~1.45)
• Version 8: Current format (latest)

Compatibility

• Trace viewer automatically upgrades traces
• Newer viewer can read older traces
• Older viewer cannot read newer traces (TraceVersionError)

---

16. Key Design Patterns

1. Call ID Correlation

Every action uses a unique callId to correlate:

• Before event
• Input events
• Log messages
• After event
• Snapshots (before@callId, input@callId, after@callId)
• Attachments
• Network requests (indirect via timing)

2. Lazy Loading

• Snapshots stored by SHA1
• Resources fetched on demand
• JSONL format allows streaming

3. Snapshot References

• Instead of storing full DOM repeatedly
• Later snapshots reference earlier ones: [[snapshotIndex, nodeIndex]]
• Resources inlined via resourceOverrides

4. Dual Time Bases

• wallTime: Milliseconds since epoch (for display)
• monotonicTime: Internal monotonic clock (for correlation)

5. Chunked Recording

• Tests can have multiple chunks
• Each chunk has separate .trace file
• Network resources preserved across chunks

6. Grouping

• Actions can be grouped with group() / groupEnd()
• Used for test steps, fixtures
• Group tracking in RecordingState.groupStack

---

17. File Reference Guide

File	Size	Purpose
trace/src/trace.ts	183 lines	Trace event types
trace/src/har.ts	189 lines	Network HAR types
trace/src/snapshot.ts	62 lines	Snapshot types
playwright-core/.../tracing.ts	700+ lines	Recording engine
playwright-core/.../traceParser.ts	62 lines	ZIP backend
playwright-core/.../traceViewer.ts	288 lines	Viewer server
playwright-core/.../traceLoader.ts	158 lines	Load traces
playwright-core/.../traceModel.ts	300+ lines	Data model
playwright-core/.../traceModernizer.ts	500+ lines	Version upgrades
trace-viewer/src/index.tsx	48 lines	Viewer entry
trace-viewer/src/ui/workbench.tsx	Main UI	Display