We Thought Our Test Coverage Was Good. Then We Asked Our Own MCP.

There is a thread in the Atlassian Community about Atlassian's Claude and Jira MCP beta that became one of the most viewed conversations we have seen there. People wanted to know what talking to Jira is actually good for, beyond the demo.

We answered with a description of how we work day to day: plan in Jira by talking to it, then connect testing the same way. We build BesTest, a test and requirements management app for Jira, and we had just published an MCP server for it. The answer resonated, so this post is the long version, with our real project data behind every claim.

Fair warning: the data turned out less flattering than we expected. That is exactly why we think it is worth publishing.

BesTest's own QA runs inside BesTest. Every requirement, every e2e test case, every regression cycle for the product lives in the product. Dogfooding was the deal from day one, because if a number in our product lies, we want to be the first team it hurts.

So when "just talk to your project" became a normal way to work with Jira, connecting the testing side the same way was the obvious next step. We pointed our own AI agents at the BesTest MCP and made it part of the daily loop.

Two things matter about this setup:

It is production data. The numbers in this post come from the QA project we actually ship from, not a demo tenant seeded to look good.

The agent uses what customers use. Same MCP server, same permission model, same governed queries. Every call goes through the same role-scoped access control as the app itself.

This is the loop as we actually run it today, combining the Jira MCP and the BesTest MCP:

1. Pull the work items from Jira. Planning starts as a conversation: what is open, what is blocked, what changed since last week.

2. Analyze them in context. The Jira MCP is good at the wider questions across many work items: duplicates, dependencies, scope that quietly grew.

3. Set up requirements in BesTest. Once a feature is understood, the agent creates the requirements through the BesTest MCP, with complexity and impact set. A typical feature for us lands with 10 to 20 requirements.

4. Develop, and keep Jira current. An agent implements the feature and updates the work items as it goes.

5. Write test cases against the requirements. When the build settles, e2e test cases go into BesTest, linked to the requirements they cover. The bulk-create flow creates the cases and the coverage links in one step, so coverage is never a separate chore to forget.

6. Build the traceability matrix and hunt for gaps. Open the Requirements Traceability Matrix report, or ask the agent to compute it from the raw links. This step is what the rest of this post is about.

7. Repeat until covered, then release. When the matrix is clean for the feature, the work item moves toward Done, and the Jira side helps decide when and how to release.

No single step here is exotic. The compound effect is what changed how we work: QA data stopped being something you open a tool to look at, and became something you can question in the middle of any conversation.

As of July 17, 2026, BesTest's own QA project contains:

• •462 requirements
• •658 test cases
• •1,077 traceability links between requirements and test cases
• •1,731 test case executions across five regression cycles, 446 of them in the latest full run

Every feature ships with tests. That is a hard rule in our pipeline, enforced before anything merges. So when we first asked the agent to build the full traceability matrix, we expected a formality.

Here is what it reported instead: 366 of our 462 requirements are covered by at least one linked test case. That is 79.2%, and 96 requirements with nothing behind them.

79.2% may read as respectable. The per-feature breakdown is where it got uncomfortable:

• •37 feature areas hold requirements. Eleven of them sit at 100% coverage.
• •One feature, shipped early in our beta, has 24 requirements marked ready and not a single linked test case. Zero. It sat invisible under a healthy-sounding average.
• •Two mature, released features are at 58%: seven covered requirements out of twelve each.
• •Of the 96 uncovered requirements, 5 are drafts, where coverage is not expected yet. The other 91 are real debt.

The reverse direction had news too: 44 of our 658 test cases, 6.7%, do not trace back to any requirement. Some are exploratory by design. Some, on inspection, verify behavior nobody asked for anymore.

The point is not that our numbers are bad. The point is that we believed something different until the matrix disagreed, and we had every advantage: a small codebase, a strict tests-with-every-feature rule, and a team that builds coverage tooling for a living.

The gap list became a first-class QA backlog. For each uncovered requirement, the agent proposes test cases, a human reviews them, and the bulk-create flow files them with the coverage links already attached.

The matrix went from a quarterly cleanup ritual to a question we ask casually, because asking now costs one sentence. And because the numbers live one question away, they get asked more often, by more people, in more moods. That, more than any single report, is what keeps the debt from re-forming.

A note on trust, because it comes up every time we show this workflow: the agent does not get a magic side door. Reads and writes go through the same governed API as the product UI, with role-scoped tokens, so "the AI updated our test data" means exactly as much, and exactly as little, as "a user with that role updated our test data".

Every statistic above was pulled live by an AI agent, on the day of writing, through the same MCP this post describes. Nothing was quoted from memory. The agent queried the requirements, test cases, links, and executions, computed the per-feature coverage itself, and the uncomfortable parts stayed in.

We think this should be table stakes for writing about your own product: if your app claims to give teams visibility, its numbers about itself should survive being fetched by a machine and published unedited.

The BesTest MCP is in closed beta today, GA soon. If you are on the beta, access is requested from Project settings, under API & MCP, and the setup is documented in the MCP beta docs.

If you are not on the beta yet and want your testing data to answer questions like the ones in this post, BesTest itself is on the Atlassian Marketplace, and the beta is the kind you can simply ask to join.

And if your traceability matrix has ever surprised you the way ours surprised us, we would genuinely like to hear about it, here or in the community thread that started this.