Verification and quality methodology

Proof layers

1. Source and repository evidence: current schemas, code, docs, public metadata, and exact asset hashes.
2. Offline validation: parsers, fixtures, unit tests, workflow graph checks, secret hygiene, and static-site checks.
3. Deployment evidence: exact artifact, production response, canonical, content type, and byte parity.
4. Actor-run evidence: one immutable build, bounded input, terminal status, same-run summary, complete dataset, and reconciled charge.
5. Channel evidence: the intended MCP, REST, n8n, Make, Zapier, or other client actually starts and consumes that run.
6. Named-destination evidence: the intended Sheet, channel, table, webhook, or database receives the exact expected record.
7. Natural and business outcome evidence: schedules work under ordinary load and users receive measurable value.

Evidence at one layer does not automatically establish the next.

Exact Actor verification

Every paid canary begins with an immutable or exact build, a small item cap, conservative maximum charge, and optional features disabled unless they are the subject of the test. The verifier retains the returned run ID and never switches to a latest-run lookup.

For source scrapers, it requires terminal success, the requested build, valid nomad-agent-run-summary-v4, complete default-dataset reconciliation, six closed roots, and expected source identity. For the fit scorer, it applies the distinct scorer v4 result-policy counts, fit-row, source-provenance, provider-guard, and billing checks while retaining legacy v3 compatibility in maintained clients.

Release policy

New production releases become latest. Maintained clients and saved Tasks follow that selector automatically. Every run still has its own numeric build number and immutable ID; retain those values for diagnostics and require them to stay unchanged while polling.

Schema versions remain explicit contracts. Validate each run and stop delivery on unsupported schemas or inconsistent output.

Data-quality benchmark

The open enrichment benchmark evaluates exact final-record fields, unsupported fills, deterministic-field preservation, provenance integrity, repeated-run completion, and selected-field translation for LinkedIn and EURAXESS separately.

The first human-verified dataset is still being prepared. No official accuracy percentage is published. The review plan requires qualified reviewers, blind double-labeling, adjudication, and documented exclusions before any public result. Synthetic or model-graded output is not presented as human ground truth.

Integration validation

Offline checks can prove that a workflow is importable JSON, contains no credentials, selects latest and preserves run identity, applies closed status gates, derives the correct destination key, and cannot retry more than once. They cannot prove a user’s account permissions, connection configuration, editor state, schedule, or destination.

A destination claim requires a disposable named destination, create proof, repeat/update or dedupe proof, failure-path proof, and a record of exact Actor, build, run, integration artifact, and time.

Search and website evidence

• Deployment: the intended HTML and assets are live and canonical.
• Submission: Search Console or IndexNow accepted a discovery notification.
• Crawl: a search engine fetched the page.
• Indexing: the canonical URL is eligible and present in that engine’s index.
• Demand: impressions and clicks appear for real queries.
• Conversion: a privacy-minimized CTA or activation event is attributable.

A sitemap or IndexNow success is never described as indexing. A newly deployed page is observed for at least 72 hours before submission is repeated or indexing problems are inferred.