Measuring knowledge base coverage of recurring tickets

Question and scope

This study asks what share of recurring ticket themes has an internal article that a support person can find and follow successfully. The unit is one ticket theme, not one article, and the population is the themes that produced a minimum number of tickets during a declared quarter. The result is a coverage estimate for that queue, not a grade for the documentation team. Freeze the theme rule and the quarter before counting.

Include tickets closed by the service desk, the infrastructure team, and the vendor. Exclude one-off project work and requests that were never intended to be repeatable. Record the threshold for a recurring theme, such as five tickets in the quarter, and apply it consistently. A theme that falls below the threshold can still be important, but it belongs in a separate watch list rather than the coverage denominator.

Define coverage

Coverage has three parts: an article exists, it matches the current system, and a support person can complete the work from it. An article that exists but describes an old menu is not coverage. An article that matches but cannot be found is also not coverage. Write the three-part definition in the protocol before reviewing any page so the standard does not drift once the gaps become uncomfortable.

Decide whether the article must resolve the ticket or only help diagnose it. Those are different levels of coverage and should be scored separately. A diagnostic page can reduce the time to route a case even when the fix belongs to another team. Record the level for each theme. The report should show both the full-resolution rate and the diagnostic-only rate so the reader can use the right number for the decision.

Build the ticket theme list

Group the tickets by symptom and system rather than by the words in the subject line. Use the ticket category, the affected service, and a short description to form the theme. Review a sample of grouped tickets to confirm that the theme is coherent. A theme called email is too broad, while a theme called Outlook search returns nothing is useful.

Keep a crosswalk from each theme to the ticket identifiers that formed it. The crosswalk is what lets a reviewer check the grouping and the counts. If the ticket system has no usable category, use the first response text and the resolution note as the source. Do not rely on the ticket title alone, because titles are often written in a hurry and reused for unrelated problems.

Sample and match

If the theme list is long, sample themes by volume and by risk. Include every theme that caused a customer outage or a repeat escalation, even if it is not in the random sample. For each selected theme, search the knowledge base with the words a support person would actually use. Record the search terms and the top results. A page that appears only under an internal code is not discoverable.

Use a second searcher with the same terms but without the first reviewer's notes. Compare whether they find the same article. If the two searches diverge, the article's title or tags are the problem. Record the path to the page, the last review date, and the system version it describes. The match is stronger when a fresh reader can reproduce it.

Grade the match

Grade each theme as covered, partial, or missing. Covered means a current article matched the theme and a support person completed the steps in a test. Partial means an article exists but needs a correction, a missing screenshot, or a clearer trigger. Missing means no usable article was found. Report the grades beside the theme count, not instead of it.

Test the steps for at least the covered themes, not only the obvious failures. A page that looks complete can contain a command that no longer exists. Ask the tester to follow it without help and record where they stopped. If the test requires a production change, use a safe read-only equivalent and mark the result accordingly. The grade should reflect the evidence, not optimism.

Measure use, not only existence

Add the page views, the ticket links, and the search terms that led to the article. A page with no links from the ticket templates is unlikely to be used during an incident. Compare the usage with the theme volume. A high-volume theme with a well-written page that nobody opens is a discoverability problem rather than a writing problem.

Ask a small group of support staff which page they would open for the top five themes. Their answer is a practical test of the navigation and the titles. Record the answers before showing them the coverage results. If they name a page that was not in the search results, the search configuration needs attention. Usage data and human recall should agree before a page is called effective.

Report the gaps

Lead with the themes that have the highest volume and no coverage, followed by the themes that are covered but frequently reopened. Show the grade counts and the pages that need correction. Keep the full theme list in an appendix so the owner can see the scope. The report should be short enough to act on in one meeting.

Recommend a small number of articles to write or repair, ordered by the risk of the underlying system. Do not propose a complete rewrite of the knowledge base. A focused repair of five high-volume themes is usually more valuable than a new taxonomy. Assign an owner and a review date for each recommendation.

Build a repair queue, not a documentation project

Turn the coverage gaps into a short repair queue ordered by ticket volume and system risk. Write or fix one article at a time and test it with a support person who did not write it. Do not open a six-month documentation programme that will be overtaken by system changes. A visible improvement in the top five themes is more persuasive than a new style guide.

Assign each repair to the team that owns the system, with the service desk as a reviewer. The owner confirms the technical steps while the desk confirms that the language and the trigger match real tickets. This split prevents an engineer from writing a page that only another engineer can follow. It also gives the desk a reason to keep the page current.

Retire duplicate and obsolete pages as part of the same work. A search result that offers three versions of the same fix is a coverage problem even when one version is correct. Redirect old links where the platform allows it and note the replacement in the ticket template. A smaller, current knowledge base is easier to trust than a large, uneven one.

Limits and next use

The study measures the themes that were captured in the ticket system and the articles that the reviewers found. It does not prove that every future ticket will match the same theme. A change in the product or the customer mix can move the queue quickly. State those limits and name the systems that were not included in the search.

Repeat the coverage check after the repairs and compare the same themes. If the new articles are not used, investigate the ticket template and the search terms before writing more. The next study should add a short usability test with a new support person. Coverage is a property of the whole path from ticket to page to successful fix, not a count of documents.

Keep the coverage report with the knowledge base owner and the service desk lead. When a new system is introduced, add its themes to the next review rather than waiting for the annual clean-up. The article list should follow the queue, not the other way around. A short quarterly check is enough to keep the gap visible.