EVIDENCE, WITH CONTEXT
A useful report beats a promise.
Browser-based agents use ordinary websites: they click buttons, search, fill forms, and move through checkout. A merchant doesn’t need an API or special integration to work with an agent.
Every agent is different
Reports and summaries are independent for each agent. Success with one agent does not imply success with another. An untested website is marked “Not yet tested,” and a few reports are marked “Insufficient evidence.” Neither means failure.
How summaries are calculated
- Only approved reports for the selected agent count. Each contributor with an account has one current report per merchant and agent. Guest reports require human verification and are limited to one per browser, merchant and agent; a browser is not a verified identity. Updating an account’s report replaces their evidence and sends it back to moderation.
- For the summary, we use conclusive reports tested within the last 365 days. If no test date was supplied, we use the original submission date. Unsure reports remain visible but don’t change the score.
- Works contributes 1, partially works 0.5, and blocked or doesn’t work 0. Each report’s weight halves every 90 days, so recent experiences matter more.
- A summary requires at least three conclusive reports in the last year, including at least two in the last 180 days. Below that, we show insufficient evidence and no score.
- A weighted result of 75% or more is “Mostly works”; 35–74% is “Mixed results.” Below 35%, we show “Often blocked” only if explicit blocked outcomes carry at least 60% of the weight; otherwise, “Often unsuccessful.” These are community summaries, not merchant policy determinations.
- Confidence is low below the evidence threshold, developing once met, and higher with at least 20 conclusive reports in the past 180 days. It describes sample size and freshness, not verified accuracy.
The public API exposes the weighted score rounded to a ten-point band, only when evidence is sufficient. It is not a probability that your next task will succeed. Community compatibility sorting requires a specific agent and excludes insufficient evidence from the top of the ranking.
Task depth matters
Contribute by selecting an agent and marking the tasks you tried in “What can an agent do here?” Written notes are optional. Leave untested tasks blank. Unless you choose an overall outcome, matching task results keep that outcome and mixed results become partially works. This summarizes only your reported tasks, not the entire website.
Finding a product and completing a purchase are different achievements. We show the furthest stage reached, counts by stage, human intervention, and links to the most recent successful and unsuccessful tests. Task depth is displayed separately and does not silently inflate an outcome score.
Reading the capability matrix
Each task uses the same recent-evidence thresholds and confidence bands as the overall summary. Supported means enough community reports support that task, not a guarantee or independent certification. Gray means untested or insufficient evidence; it never means failure. At least three conclusive observations in the last year, including two within 180 days, are needed before a task receives a status.
Optional task-by-task outcomes take priority. For older reports, a successfully reached stage supports website access and that exact task only. Browse/search, details, cart, checkout navigation and completed purchase are separate capabilities. Configuration, shipping and payment-approval stages establish access but do not prove purchase completion. We do not infer search or login from a purchase, or infer a failed step from where an agent stopped. Human-assisted stage observations are partial. Unknown task locations remain unknown.
Conflicting evidence means recent successes and failures each carry at least 20% of the task’s weighted evidence. When the sample threshold is met, conflicting tasks show partial/inconsistent results. CAPTCHA-only blocked outcomes count as partial rather than explicit blocks. Task failures apply to that task; they do not automatically classify the entire website. Details link to the original reports and identify explicit task outcomes versus observations derived from a reached stage.
A CAPTCHA is useful context
A CAPTCHA requiring human assistance is not proof that a merchant blocks every agent. Reports distinguish explicit blocks, CAPTCHAs, unsupported interfaces, authentication requirements, payment problems, and other limitations. Respect merchant restrictions and do not bypass safeguards.
Transparent, community-led evidence
We review reports before publishing. Moderation checks relevance, abuse, and private data; it does not independently reproduce every task. A single negative report never permanently classifies a website. Sponsorships and affiliate relationships do not influence summaries.