Testing evidence completeness in asset inventory without a convenience sample

Question and scope

Write the primary question and one supporting question, then stop; a long list of questions usually hides an undefined unit. State how duplicate requests are identified and whether they are merged, linked, or counted separately. A case is complete only when the asset inventory shows the approved destination value and the access request supports it. Write the decision the study is meant to inform before choosing the measure. Keep the measurement note short enough to read with the case and specific enough to support the reported patch compliance.

The protocol does not estimate an industry benchmark; it produces a local description that IT managers and MSP owners can review. State how duplicate requests are identified and whether they are merged, linked, or counted separately. Distinguish an administrative pause from a failure: a pause waits for a named decision, while a failure leaves the next action unowned. Name the population, the observation window, and the systems included in the extraction. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.

Population and unit

Record the opening count by owner and channel, because a single total can hide a queue concentrated in one person. Define whether a request that generates several MFA reset log updates is one case or several before counting begins. If the definition of orphaned account count changes, keep the old and new values side by side for the transition period. Separate cases opened before the window from cases opened and closed inside it, and report both counts. Link the observation to the exact field in the identity provider and to the MFA reset log that supports it.

Record the opening count by owner and channel, because a single total can hide a queue concentrated in one person. The population should include unresolved cases, not only closures, or the study will overstate completion. Use plain language in the definition and add one example and one non-example for the reviewer. Report a case that moved between owners as one case with two owner periods unless the process creates a new requirement. Do not let a summary written later replace the evidence that was available at the first decision.

Definitions and measures

Define the controlled event as the moment a request for asset inventory becomes work, not the moment someone starts typing a note. Do not store sensitive content in the analysis file when a reference to the approved repository is enough. Use the same status vocabulary in the queue, the evidence extract, and the report so a case does not change category. If two reviewers disagree, preserve both classifications until the definition is clarified. If the case moved between owners, include both owner periods in the review note and name the handoff time.

Test each definition on a routine case, a missing-evidence case, and a case that crossed owners. Record the version of the backup test and the backup console snapshot used for each case. A completeness measure should name the required fields and the evidence state; an overall percentage is not reproducible. Give one concrete example and one boundary case to each reviewer before collection begins. Keep the review note with the case so the next sampler does not have to reconstruct the decision from memory.

Data dictionary and evidence

Keep free-text notes separate from coded fields so the analysis does not depend on inconsistent wording. Name the required fields for each article type and mark conditional fields in the dictionary. Keep the source reference and the coded value in the same row, and do not overwrite the original with a later correction. Pilot the dictionary with two reviewers and compare their classifications before the main sample. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract.

Record the version or date of the ticket intake and the vendor portal value so a later reviewer can see what was known at the first action. If a field is derived, record the formula and the source fields used to calculate it. Record the source identifier, the field name, the observation time, and the reviewer for every extracted value. Do not use a zero or an empty string to represent missing evidence. At IT Virtual Assistant, the asset inventory review should end with a dated note and a named owner for the next check.

The data dictionary should name the allowed values for status, exception reason, owner, and close evidence. Use a controlled list for exception reasons; do not let free text become the analysis category. Record the source identifier, the field name, the observation time, and the reviewer for every extracted value. Define who may change a coded value and how the change is logged. If the evidence is sensitive, record the approved reference and access path rather than copying the content.

Sampling and collection

A convenience sample of recently closed cases will overrepresent easy work and underrepresent the tail that needs attention. Collect the case from the vendor portal, the ticket intake, and the approval record, and stop at the verified close. Include reopened, unresolved, and exception cases in the sample frame. Do not let the reviewer choose the sample after seeing the results. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.

Use a random or systematic start rule for the first case, then take consecutive cases within each stratum. Keep the collection window and the number of collectors in the report; both affect how the result should be interpreted. If a stratum has fewer cases than planned, report the smaller count instead of replacing it with a convenience case. Keep a table of planned, eligible, selected, and reviewed counts for each stratum. Record this decision in the study log with the case key, the review date, and the person who approved the classification.

Include adverse cases: a missing access request, a conflicting source, an unavailable security lead, and a tight deadline. A second reviewer should re-extract a small subsample and record disagreements before the full analysis. Keep the sample selection reproducible by recording the export date, filter, and random seed or start rule. Record the sampling frame as an export with a date, filter, and row count. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.

Analysis and reporting

Report the eligible count, the sample, the missing-field rate, the exception rate, and the unresolved count at the close of the window. Use case-level review to explain an outlier before treating it as a process signal. Show the unresolved cases by owner, age, and dependency instead of reporting a single completion percentage. Report the number of cases that drive a small-sample result and the direction of the effect. If the source changes after the observation, preserve the earlier value and add a dated note instead of replacing it.

A difference between two periods is a signal to inspect, not proof that a process change caused the change. If two measures disagree, report both and explain what each one counts. If the result is unstable, say which cases create the variation and what a larger sample would test. Report the numerator, denominator, observation window, and exclusion count for each measure. At IT Virtual Assistant, the reviewer should be able to explain this step to IT manager from the case record alone.

Limitations

The record may be incomplete, delayed, or maintained outside the team, so the method cannot infer missing work from a clean summary. The record may be incomplete or delayed, so the method cannot infer work that never entered the queue. If the queue changed during the window, explain how the change affects the denominator and the interpretation. If the reviewer performed the work, use a second reviewer for a subsample and report the disagreement. Use the same note in the case file and the study extract; do not create a second version in a personal document.

The method also cannot observe work that never entered the queue, which may be the most important missing population. The record may be incomplete or delayed, so the method cannot infer work that never entered the queue. If the reviewer also performed the work, note the independence limitation and use a second reviewer for a subsample. If the queue changed during the window, explain how the change affects the denominator. When a case needs an exception, record the reason, the owner, and the date before continuing the review.

Review and next action

Repeat the protocol after the change with the same definitions, window, and exclusions. Keep the review note with the asset inventory evidence so the next owner can see why the process changed. Link the action to the specific asset inventory cases that motivated it. Keep the prior procedure available so the next reviewer can compare the old and new states. Keep the measurement note short enough to read with the case and specific enough to support the reported orphaned account count.

A quality review should end with an owner, a date, and one testable change, not a list of intentions. Report what moved, what stayed the same, and what the study still cannot explain. Link the action to the specific asset inventory cases that motivated it. If the action needs approval, record the approver and the date before the change is applied. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.