How to build WebMCP tools: lessons from a fediverse hashtag index

At the end of August, OpenAI ran a ten-day challenge for sites built with WebMCP, a draft standard that lets a web page register tools an AI agent in the browser can call. I had been meaning to try it, and a deadline is a good way to find out whether an idea holds up. So I built something and entered it.

The result is the Fediverse Hashtag Activity Index. It watches hashtag activity on a handful of Mastodon-compatible servers and publishes what it sees. It also registers six WebMCP tools, so if you open it in ChatGPT’s browser you can ask questions in plain English and the agent answers using the site’s own data. The code is on GitHub under the MIT licence, and there is a three-minute demo video if you would rather watch than read.

Play demo, 2:46. Loads from YouTube when you press play.

This article is about the design decisions, mostly the ones I got wrong first. It is aimed at people who are thinking about adding WebMCP to a site of their own and want to know what the work actually consists of.

What WebMCP is, in a paragraph

A page calls document.modelContext.registerTool() with a name, a description, a JSON schema for the input, and an execute function. That is all. An agent running in the browser can then see the tools and call them directly, instead of reading your HTML or clicking through your forms. Because the page and the agent share a tab, a tool can also change what the person is looking at. That last property is the one that makes it different from a JSON API, and it took me a few days to understand what to do with it.

At the time of writing, two browsers support it: ChatGPT’s desktop app has it on by default in its in-app browser, and Chrome has it behind a flag. Both were used for judging, so I tested in both.

Why a hashtag index

The fediverse has no centre. There is no server that sees every post, so there is no such thing as a complete count of anything. Any number about hashtag activity is a view from somewhere, and an honest site has to say where.

The index does that at every turn. It polls public hashtag timelines on eight servers, deduplicates what it sees, and publishes each figure with its provenance: which servers contributed, how many were healthy, and when the last successful update was. When two time windows were not covered by comparable slices of the network, it refuses to call a trend at all. I built that part on the first day, before any of the WebMCP work, and I built it that way because I have watched too many dashboards present a partial number as if it were the whole picture.

Then I thought about what happens when an agent reads that page. It finds the number. It repeats the number. The caveats are prose, and prose gets summarised away. Everything I had done to make the figures honest would evaporate the moment someone asked a chatbot about them.

That was the angle. Most entries in a challenge like this will be sites where the agent does something on your behalf: books, filters, buys. There is nothing to book here. What I had instead was a data source built around the fact that it cannot see everything, and a way to hand that fact to the agent in a shape it cannot drop.

What the tools do

I did not start from the API. I started by writing down seven things a person might actually say to an agent while on the site, and for each one, the reply I would want to hear. Then I worked out which tools those replies needed. Seven utterances resolved to six tools.

• “I’m writing this post, which hashtags fit it best?” The agent reads the draft and proposes candidates, because language is what it is good at. Then evaluate_hashtags reports how many distinct accounts used each tag, across how many servers, and whether that looks like a conversation or a few accounts posting a lot.
• “Compare #pottery, #art and #photography.” compare_hashtags takes the person to a real page, /compare?tags=a,b,c, with sparklines and a warning when the tags’ evidence is not like for like.
• “What’s trending?” trending_hashtags ranks the tracked set by distinct authors rather than post volume.
• “Tell me about #photography.” lookup_hashtag gives the full picture and moves the page to that tag.
• “How much of the fediverse can you actually see?” describe_coverage lists the monitored servers and tells the agent how to phrase any claim.
• “Can you say #news is rising across the fediverse?” check_claim referees what the agent is about to say. It is my favourite, and it gets its own section below.

Every result has the same shape. There is a status, the figures, and a provenance object as a required sibling, never optional. If the index cannot answer, status says so and a reason says why. The refusal branch of each tool was designed before its success branch.

The decision I reversed: drawing on the page

The first version of compare_hashtags did what the WebMCP pitch suggests. The agent called the tool, and the tool injected a comparison table into whatever page was open, kept in sessionStorage so it survived navigation. It worked, and in the ChatGPT browser it looked impressive for about a minute.

Then I saw it sitting on top of an unrelated tag page and it read as a hack. The context underneath was wrong. The state was invisible to anyone else and impossible to share. It was a view no person could reach without an agent, on a site whose whole point is that a person can check everything.

So I replaced it with a page. /compare?tags=a,b,c is server-rendered, has a form, and has a nav link. The tool sends the browser there and returns the same figures the page shows. The URL is the shared state, the same as everywhere else on the site. When the agent says “I’ve opened a comparison”, the person is looking at something they could have got to themselves, and could paste into a chat.

I think this generalises. If a tool changes what the person sees, make it change it to something with an address.

The tool that says no

The other five tools hand the agent evidence and trust it to phrase things well. That trust is mostly rewarded, but “mostly” is not a design. So the sixth tool closes the loop. The agent submits the claim it intends to make, a tag, and a scope, and the index referees it against the same rules the site applies to itself.

export type Claim = 'rising' | 'falling' | 'busy' | 'quiet' | 'in_use' | 'unused';
export type Scope = 'index' | 'fediverse';
export type Verdict = 'allowed' | 'qualified' | 'blocked';

The rules are short. Directions such as rising and falling are checkable, so they come back allowed or blocked: blocked when the two hours are not comparable, when there are too few authors to call it, or when the data says the opposite. Adjectives such as busy and quiet are judgements, so they come back qualified with the exact sentence the index can stand behind. “Unused” is always blocked, because absence of evidence is not evidence of absence. And any claim scoped to the whole fediverse is blocked outright, because there is nothing to check it against.

Every verdict carries a field called may_say, a sentence that is true on the evidence given. A blocked claim is never a dead end. The agent always has something honest to fall back on, and the tool description tells it to prefer that sentence over its own wording.

The exchange that made me think this was worth entering happened on the last day of August. I asked ChatGPT, on the site, whether it could say #news was rising across the fediverse. It called the tool and replied:

No. The index blocks a fediverse-wide claim, and its comparable observed data says the opposite: #news is falling on the servers it monitors, 182 authors in the prior hour vs 109 in the latest hour. Coverage is thin (8 monitored servers).

It refereed both scopes, quoted the counter-evidence and carried the coverage grade through. That is the sentence I wanted a person to hear, and it is the exchange that closes the video.

Protecting the site from its helpers

There was one thing I had not thought about until the second day. On the site, looking up a tag is a signal. Enough lookups promote a tag towards being collected, and there is a hard ceiling on how many tags the index watches at once. That is deliberate. A person searching for a tag is telling the index that the tag matters.

An agent is not a person. An agent asked “what are people talking about?” might look up thirty tags in a second, and each one would count. So anything that evaluates many tags at once goes through a separate read-only endpoint that registers nothing. The one tool that does have a side effect, lookup_hashtag, says so in its description and reports in its result whether the lookup caused anything to change. Every result carries a side_effects object, even when it is all zeros, so the agent can tell the person what the call did.

How to build WebMCP tools: the practical bits

These are the things that cost me time and that I would want to know before starting. Each one answers a question I searched for and did not find a good answer to.

How to register WebMCP tools with document.modelContext

To register WebMCP tools, call registerTool() on document.modelContext once for each tool, from one inline script in the shared layout. Feature-detect it, so a browser without a model context runs nothing and the HTML form stays the human path.

function host() {
  try { if (document.modelContext) return document.modelContext; } catch (e) {}
  try { if (navigator.modelContext) return navigator.modelContext; } catch (e) {}
  return null;
}

function register() {
  var h = host();
  if (!h || typeof h.registerTool !== 'function') return;
  tools.forEach(function (t) { h.registerTool(t); });
}

register();
window.addEventListener('DOMContentLoaded', register);
window.addEventListener('load', register);

Register on document.modelContext. The W3C draft and most of the early write-ups use navigator.modelContext, and I did too, and ChatGPT’s browser reported “no WebMCP tools” for a whole morning. It turns out ChatGPT exposes only the document form, and Chrome 151 now logs a deprecation warning for the navigator one. Keep navigator as a fallback, never as the first choice. The challenge’s own sample code had this right and I did not read it closely enough.

Some hosts attach the model context after the page has loaded, so run the check again on the load events. Guard against registering twice.

How to return a result from a WebMCP tool

A WebMCP tool returns its result from the execute function as an object with a content array. The function receives the input, sometimes as an object and sometimes as a JSON string depending on the host, so parse defensively. Return the result as text content:

async execute(args) {
  if (typeof args === 'string') { try { args = JSON.parse(args); } catch (e) { args = {}; } }
  var res = await fetch('/api/v1/check?tag=' + encodeURIComponent(args.tag) +
    '&claim=' + encodeURIComponent(args.claim));
  return { content: [{ type: 'text', text: JSON.stringify(await res.json()) }] };
}

Each tool’s execute is a fetch to the site’s own JSON API, with a header naming the tool so the server logs can tell agent traffic from people. No judgement lives in the browser that is not already in the tested server code.

How to write a WebMCP tool description so the agent picks the right tool

The description is the routing layer. It is the only thing the agent reads when deciding which tool to call, and I underestimated it.

My first “compare these two tags” request went to evaluate_hashtags, because that description read as the general-purpose tool and nothing rendered on the page. The fix was to make each description name the other tool for the other job, in both places. evaluate_hashtags now says “if the person asks to COMPARE tags, use compare_hashtags instead”, and compare_hashtags leads with the verbs a person uses: compare, versus, side by side. After that, the routing was right every time.

Put the instruction for how to handle the result in the description too. “Read standing_note before quoting a figure, and treat unseen as no evidence, not as unused” did more for the quality of the agent’s replies than anything I did to the data.

How to test WebMCP tools in ChatGPT and Chrome

Test WebMCP tools against a deployed HTTPS URL, not a local dev server. The model context is only available in secure contexts, and the interesting failures only happen with real data: thin coverage, a cold tag, the tracked-set ceiling.

I added a /webmcp page to the site that reports what the visiting browser exposes and lists the registered tools. That page is how I found the document versus navigator problem, and it is also how a judge without an agent can see the whole surface. It shows the same question answered twice, once with the tools and once from the page text alone, which makes the point of the project visible in about ten seconds.

For Chrome, document.modelContext.executeTool(tool, input) in the console runs a tool exactly as the agent would, which is a quicker loop than typing into a chat panel.

What I would change next

The agent-originated queries are currently treated the same as a person’s. I can tell them apart, because the tools send a header, and I would like to de-weight them so a curious agent cannot promote a tag on its own. I would also like a tool that explains whatever page is open, using the ontoolchange event to keep it in step with navigation. Both are recorded as open questions in the repository and neither made the deadline.

The site itself will stay up through judging, which runs until 21 September, and after that for as long as it costs nothing to run.

Takeaways

• WebMCP is small. Registration is one call, and the work is in designing the tool surface, not the plumbing.
• Start from what a person would say, not from your API. Seven sentences gave me six tools and a script for the video.
• Use document.modelContext. Fall back to navigator, but never lead with it.
• The description is the routing layer. If two tools overlap, say so in both.
• If a tool changes what the person sees, make it a page with a URL, not an overlay.
• Structured provenance beats prose caveats, because an agent can summarise prose away and cannot drop a required field.
• Design the refusal before the success. The moment an agent said “No, and here is why” was the moment the project made sense.

If you try it, open the live site in ChatGPT’s browser and ask it to compare a couple of hashtags. Then ask it to tell you one is rising across the fediverse, and watch it decline politely.