Add webcompat-triage agent by ksy36 · Pull Request #6170 · mozilla/bugbug

ksy36 · 2026-06-15T19:59:26Z

This is a first part of a webcompat triage agent. I'm planning to add a second part (chrome mask extension install) in a follow-up PR.

ksy36 · 2026-06-15T20:17:51Z

+class AutoWebcompatInputs(BaseModel):
+    bug_data: str | None = None
+    bug_id: int | None = None
+    mode: Literal["triage", "chrome-mask"] = "triage"


If we decide to have a single agent for all webcompat work (triage/diagnosis/ intervention), we could pass a mode here and have a prompt specific to the task, while keeping generic rules in system.md. I've added triage prompt as an example.

I prefer having an agent for each workflow, so each agent will be less complex and easier to improve and expand without complex side effects on other workflows. For redundant components, we could always import from other agents, or even better, moving to a common package to import from.

ksy36 · 2026-06-15T20:23:56Z

+    def _require_subject(self) -> "AutoWebcompatInputs":
+        if self.bug_data is None and self.bug_id is None:
+            raise ValueError("provide at least one of bug_data or bug_id")
+        return self


It accepts either bug_data or a bug_id. If bug_id is provided, it'll fetch the bug data using bugzilla MCP. In case if bug_data is provided, it will work with that.

It can be a report text, something like:
https://aifaceswap.io/#face-swap-playground — after generating a face-swapped image, clicking the Download button under the generated image does nothing in Firefox (works in Chrome).

It is currently a str, but we can agree on format that would come from bugbot.

I suggest having a separate agent for each workflow. One agent could handle bugs, while another could handle text reports. Each agent would need different tools, and the prompts might require different tweaks.

ksy36 · 2026-06-15T20:25:47Z

+## Task: Chrome Mask test
+
+Run the **Chrome Mask test** to determine whether the broken behavior stems from
+User-Agent sniffing.


This is just a placeholder for now, I'll add the actual prompt and a creation of a profile with an xpi in a follow up PR.

ksy36 · 2026-06-16T01:17:34Z

+        result=result_msg.result,
+        num_turns=result_msg.num_turns,
+        total_cost_usd=result_msg.total_cost_usd,
+    )


Looks like result of the run is stored as an artifact in a json file. We should decide what the result should look like. So for a sample triage run it produced this in summary.json:

{ "status": "ok", "error": null, "findings": { "num_turns": 78, "total_cost_usd": 3.2635555, "mode": "triage", "result": "I've confirmed the reproduction. Here's my report:\n\n## Reproduction Report\n\n**URL:** https://aifaceswap.io/#face-swap-playground\n**Browser:** Firefox (headless via Firefox DevTools MCP)\n**Date tested:** 2026-06-15\n\n### Steps taken\n1. Navigated to the Face Swap Playground.\n2. Selected the Mona Lisa template via the \"use it\" button, then replaced the Original image by uploading `face.jpg` (a woman portrait fetched from the site's own template set).\n3. Uploaded `face2.jpg` (a man portrait, also from the site's templates) into the **Face image** slot.\n4. Clicked **Start face swapping**. The queue progressed from ~25% \u2192 95% \u2192 complete (~35 seconds).\n5. A swapped result image appeared in the right pane, with three action buttons below it: a small circular **Download** button (download icon), **Upscale**, and **Enhance**.\n6. Clicked the Download button (`<button class=\"btn btn-icon btn-sm btn-primary me-5\"><i class=\"bi bi-download\"></i></button>`).\n\n### Result \u2014 bug reproduced\nClicking Download **does nothing** in Firefox:\n- No file save dialog appears.\n- No new tab opens.\n- No network request is issued after the click.\n- No console messages are logged.\n- The page state is unchanged.\n\nI tried both a UID-targeted click via DevTools and a programmatic `.click()` from JavaScript \u2014 both produced no visible effect and no observable side effects.\n\nThe reported web-compatibility issue is reproducible as described." }, "actions": []

The sample triage prompt didn't ask to leave a comment or attach testcases/screenshots, but if we add that and there is a bug id passed as well, these could be in the actions list:

ENABLED_ACTION_TYPES = [ "bugzilla.update_bug", "bugzilla.add_comment", "bugzilla.add_attachment", "bugzilla.create_bug", ]

For example,

"actions": [ { "type": "bugzilla.add_comment", "params": { "bug_id": 2046445, "text": "Reproduced on Firefox Nightly using firefox-devtools MCP tooling on Linux.\n\n**Steps performed:**\n1. Navigated to https://aifaceswap.io/#face-swap-playground\n2. Uploaded two images via the \"Original image\" and \"Face image\" upload widgets (programmatically populated the two `input[type=\"file\"]` elements with image/webp files and dispatched a `change` event)\n3. Clicked \"Start face swapping\" \u2014 the face-swap pipeline ran, the progress bar advanced to completion, and the swapped image appeared in the result panel along with the action toolbar (download icon button, Upscale, Enhance)\n4. Clicked the download icon button (`button.btn.btn-icon.btn-sm.btn-primary.me-5` containing `<i class=\"bi bi-download\">`)\n\n**Result:** Nothing happens.\n- No file download is initiated (no \"Save As\" dialog, no entry added to downloads).\n- No new tab/page is opened.\n- No navigation occurs.\n- No visible UI change on the page.\n- No anchor element with a `download` attribute is programmatically clicked (verified by instrumenting `HTMLAnchorElement.prototype.click`).\n- No console messages are emitted.\n\nI also tried dispatching a full `mousedown` \u2192 `mouseup` \u2192 `click` MouseEvent sequence at the button's center coordinates in addition to a plain `.click()` \u2014 the behavior was identical (no download, no UI change).\n\nThis matches the actual behavior described in comment 0. Issue reproduces on the current site build.\n\n*This is an automated analysis result. If this result is incorrect please add a needinfo and feel free to correct the error.* ", "is_private": false }, "reasoning": "Confirming reproduction of the broken download button for the webcompat triage. Reporting only what was observed, no root-cause analysis." }

My preference, which I don't think is universal, is that we try to get it to produce those artifacts without modelling it as an update to the bug (even if it doesn't actually perform the updates). The bug is intrinsically shared global state, and the more we depend on the model interacting with that directly, the harder it is likely to be to run experiments or use in non-bugzilla contexts (e.g. with the dashboard).

Sounds good, I've changed it to this format for now (defined in result.py):

{ "status": "ok", "error": null, "findings": { "num_turns": 19, "total_cost_usd": 0.47287900000000005, "result": { "reproduced": true, "summary": "The reported issue reproduces in Firefox Nightly. After loading https://www.mbusa.com/en/owners/manuals, the page renders the site header at the top and jumps straight to the \"Stay connected to all things Mercedes-Benz\" newsletter section and footer. The main content region \u2014 where the vehicle model catalog (the model picker grid users select to view their manuals) should appear \u2014 is empty. Inspection of the DOM shows the `<main>` element contains a `<css-css-oom-generic>` custom element host whose laid-out height is 0px and which contributes no visible text, so the catalog never becomes visible. A cookie/consent dialog (Usercentrics) overlays the page on first load but dismissing it does not bring the catalog back; the area between header and footer remains empty. This matches the report's expected vs. actual behavior.", "steps": "1. Launch Firefox with a clean profile (Firefox 154 Nightly or Release 151 \u2014 both reproduce per the report).\n2. In a new tab, navigate to https://www.mbusa.com/en/owners/manuals .\n3. Wait for the page to finish loading. Observe that the Mercedes-Benz site header (logo and Vehicles / Electric / Shopping / Owners / My Account / Find a Dealer / Search) appears at the top.\n4. If a Usercentrics \"Analytics and Advertising\" consent dialog appears at the bottom of the viewport, click \"Confirm\" to dismiss it (this step is not required to see the bug, but removes the overlay so the empty area is clearly visible).\n5. Scroll down from the top of the page and observe that immediately below the site header the page shows the \"Stay connected to all things Mercedes-Benz\" newsletter band followed by the footer (Vehicles / Shopping Tools / Electric / Owners Info / Discover Mercedes-Benz columns), with no vehicle model catalog rendered in between.\n6. (Optional verification) Open DevTools Console and run `document.querySelector('main').getBoundingClientRect()` \u2014 note the `<main>` element's height is ~80px and `document.querySelector('main').innerText.length` returns 0, confirming the catalog content area is empty.\n7. (Optional comparison) Load the same URL in Chrome with a clean profile and observe that the vehicle model catalog renders correctly in the main area." } }, "actions": [] }

jgraham · 2026-06-16T11:14:14Z

+
+class AgentInputs(BaseSettings):
+    bugzilla_mcp_url: str
+    bug_data: str | None = None


I think we should consider making this explicitly a JSON object rather than just a string (although maybe in the end it doesn't make much difference?).

jgraham · 2026-06-16T11:21:08Z

+    options = ClaudeAgentOptions(
+        system_prompt=system_prompt,
+        mcp_servers={
+            "bugzilla": bugzilla_mcp_server,


I wonder if we should configure things so that if we don't pass in a bug id we don't give it any access to bugzilla?

As mentioned in #6170 (comment), having a separate agent for each workflow would be simpler. Therefore, a workflow that doesn’t use Bugzilla wouldn’t need the Bugzilla MCP in the first place.

But in this case the actual workflows would be more or less identical, with very similar prompts and the same outputs, just the source of input data would be slightly different. It feels very heavyweight to say that we need to have a different workflow for every combination of configuration options (even if it makes some sense to separate things out when the agent is fundamentally trying to do different things).

I totally agree with you. I see it as the agent more or less should adapt to your workflow where you want to inject its results, not serving multiple workflows that are fundamentally different in their nature, we do not need all the team's use cases in one agent. We defiantly do not want to create a new agent for every combination of configuration options, it is more about what the type of work that the agent will be doing, not the inputs and configurations.

Contexts: While we are building this, my goal is to reduce the blueprint when creating new agents as much as possible (we still have a considerable amount of blueprints), but also I do not want to have things to be magically, because that would make things harder, not easier. For that, I wanted to onboard more agents with identical use cases before starting to optimize on that.

After discussion with James, we decided we want to keep both bug_id and bug_data. I made it so bugzilla MCP is not available if bug_id is not passed. I've also removed mode and made this a "triage" agent only

jgraham · 2026-06-16T11:24:51Z

+        result=result_msg.result,
+        num_turns=result_msg.num_turns,
+        total_cost_usd=result_msg.total_cost_usd,
+    )


My preference, which I don't think is universal, is that we try to get it to produce those artifacts without modelling it as an update to the bug (even if it doesn't actually perform the updates). The bug is intrinsically shared global state, and the more we depend on the model interacting with that directly, the harder it is likely to be to run experiments or use in non-bugzilla contexts (e.g. with the dashboard).

jgraham · 2026-06-16T11:36:50Z

@@ -0,0 +1,33 @@
+#!/usr/bin/env bash


I don't love this being a shell script. It's not really a blocker, but I think pretty soon we're going to want something like mozdownload / mozinstall that knows how to work with multiple platforms.

I've dded a firefox_install.py script with mozdownload / mozinstall that runs before the agent starts

jgraham · 2026-06-16T12:31:16Z

+      dockerfile: agents/autowebcompat/Dockerfile
+      target: agent
+    environment:
+      - RUN_ID


Should we prefix these either like AUTOWEBCOMPAT_ or BUGBUG_? We can change the settings class to strip the prefix.

It will not be necessary since this will not be shared with other agents or so. Keeping it shorter would make it cleaner when running it locally.

jgraham · 2026-06-16T12:42:23Z

+import logging
+from contextlib import asynccontextmanager
+
+import bugsy


I appreciate this is not your fault, but I'm rather sad that bugbot is using bugsy, which is not maintained.

@jgraham This was temporary :)

We have a draft PR to migrate to libmozdata: #6180

That said, even though libmozdata is maintained and widely used across our tools, I think we need a more modern library that offers:

Type safety with proper type hints/annotations (perhaps Pydantic)

Async support

A clean query builder API for constructing advanced queries

Sensible retry defaults for when Bugzilla is having bad time

It might be worth starting a conversation with you, me, and @dklawren to figure out the best path forward.

ksy36 force-pushed the autowebcompat-agent branch 2 times, most recently from 7e023dc to 32fabae Compare June 15, 2026 20:13

ksy36 commented Jun 15, 2026

View reviewed changes

ksy36 commented Jun 16, 2026

View reviewed changes

Add autowebebcompat agent

bf902a2

ksy36 force-pushed the autowebcompat-agent branch from 32fabae to bf902a2 Compare June 16, 2026 01:26

jgraham reviewed Jun 16, 2026

View reviewed changes

Code review changes

46f7727

ksy36 changed the title ~~Add autowebebcompat agent~~ Add webcompat-triage agent Jun 18, 2026

ksy36 added 2 commits June 18, 2026 10:54

Remove recording of actions

7d164ac

Only make bugzilla mcp available if bug_id is passed

5a027a2

ksy36 marked this pull request as ready for review June 18, 2026 15:45

ksy36 requested review from jgraham and suhaibmujahid June 18, 2026 15:46

Conversation

ksy36 commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ksy36 Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ksy36 Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ksy36 Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ksy36 Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ksy36 commented Jun 15, 2026 •

edited

Loading

ksy36 Jun 15, 2026 •

edited

Loading

ksy36 Jun 15, 2026 •

edited

Loading

ksy36 Jun 16, 2026 •

edited

Loading

ksy36 Jun 18, 2026 •

edited

Loading