Independent implementation

Zeno Mobile Runner

A deterministic mobile test runner for AI coding agents. One Zig binary drives real iOS and Android targets, returns typed pass or fail, and writes replayable evidence with no LLM in the execution path.

Evidence boundaryThis is a public developer preview that I built independently. It demonstrates deterministic mobile execution, typed agent interfaces, and replayable evidence. It does not establish production adoption or client delivery, and it does not claim broad cloud-device-farm certification.

Problem

AI coding agents can change a mobile app quickly, but screenshot descriptions vary and a chat transcript is not a repeatable merge gate. A reviewer needs evidence of which scenario ran, what the device showed, where it failed, and whether the same check will return the same verdict in CI.

Constraints

  • The same scenario contract must drive React Native, Expo, Flutter, and native applications without moving orchestration into each framework.
  • Agent-facing tools must return typed observations and results instead of terminal prose that callers have to scrape.
  • The merge gate must remain deterministic and free of per-run model cost even when an AI agent initiated the check.
  • Device screenshots, UI trees, logs, and action inputs may contain sensitive data, so trace export needs explicit redaction and claim boundaries.
  • iOS and Android expose different native automation surfaces, but callers should not need different scenario models.

Product evidence

ZMR trace viewer showing a passed iOS run with its timeline, screenshot, UI tree, and selector payload.
The static trace viewer keeps the device state, actions, timings, and artifacts together.
ZMR command output showing a failed Android scenario, the named WaitTimeout explanation, and the same scenario passing after the fix.
The CLI returns a typed failure, names the failed step, and records the passing rerun.

Architecture

One engine, two device backends

  1. Parallel
    AI coding agentRequests mobile verification and consumes typed observations and results.
    Continuous integrationReplays committed scenarios as a deterministic merge gate.
  2. Parallel
    MCP and JSON-RPCExpose versioned agent and client methods over structured protocols.
    CLI and JSON scenariosExpose the same engine to people and repository-owned checks.
  3. Shared Zig engineRuns selectors, waits, settling, assertions, scenarios, and trace writing once.
  4. Parallel
    Android backendUses ADB, UI Automator, and an optional app-local shim.
    iOS backendUses simulator and device lifecycle tools plus an app-local XCTest shim.
  5. Evidence bundleBinds events, screenshots, UI trees, timings, assertions, and digests to the verdict.
Public architecture from the open-source implementation. App-specific integration detail is omitted.

Turn an agent session into a repeatable test

  1. Agent exploresObserves semantic mobile state and performs a goal-directed interaction.
  2. Semantic traceRecords supported actions, selectors, results, screenshots, and UI state.
  3. Discover or draftConverts supported trace evidence into a schema-validated scenario candidate.
  4. Human review and commitKeeps generated steps conservative and makes the scenario an explicit repository contract.
  5. CI replay without an LLMRuns the reviewed scenario through the same engine and device backends.
  6. Typed pass or failReturns the failed step and replayable evidence instead of a variable screenshot judgment.
Exploration stays flexible; the reviewed scenario becomes the deterministic CI contract.
  • 20 / 20passes in the published 17-step iOS determinism run
  • 45identical trace events in every published run
  • 6reference client languages over one versioned protocol

Decisions

Keep one execution engine behind every surface

ChoiceMap MCP, JSON-RPC, CLI commands, and committed scenarios onto the same Zig runner and versioned schemas.

Trade-offThe shared core reduces behavioral drift, but every public surface must preserve compatibility with the same contract.

Keep the deterministic gate free of an LLM

ChoiceLet external agents plan and explore, then use saved selectors, waits, assertions, and scenarios for the merge verdict.

Trade-offThe gate is repeatable and has no model cost, but scenario authoring remains explicit rather than relying on visual judgment at run time.

Treat traces as evidence, not debug leftovers

ChoicePackage events and artifacts with digests, typed identities, redaction controls, and validation that fails after tampering.

Trade-offEvidence is reviewable and portable, but trace format and privacy controls become product contracts that need careful evolution.

Use app-local shims only where native platforms require them

ChoiceDrive Android without mandatory instrumentation and generate an app-local XCTest shim for iOS selector actions.

Trade-offIntegration stays small and native, but iOS setup is more involved than Android and must remain visible in onboarding.

Verification

Open the implementation or run the bounded behavior directly.

Limitations

  • The v0.2.18 release documented here is a public developer preview, not a production-stable 1.0.
  • The published determinism result covers a generated Expo fixture and does not establish equivalent reliability for every mobile application.
  • Physical iOS log capture remains simulator-first, and broad cloud-device-farm certification is outside this preview.
  • Redacted exports can replace or omit visual artifacts, but they do not mask sensitive pixels inside the original raw captures.
  • This independent implementation does not establish production adoption or client delivery.