Skip to main content

Audit methodology

Audit relative to the metric's stated decision contract and the fields actually returned. A finding is scored only when the evidence establishes a concrete configuration risk. Otherwise record an observation or not assessed.

Scored checks​

Rule IDConditionDefault severity
identity.generic_nameexplicit name is empty, placeholder-like, or cannot distinguish the metricMEDIUM
identity.no_descriptiondescription property is present and empty, leaving measure/scope ambiguousMEDIUM
data.missing_measurefull definition explicitly lacks a measureCRITICAL
data.missing_time_dimensionfull definition explicitly lacks required time configurationCRITICAL
data.invalid_field_referencedefinition references a field absent from a complete, matched datasource schemaHIGH
aggregation.incompatibleaggregation contradicts the documented metric definition or data grainHIGH
aggregation.running_total_misuserunning total is applied to a rate, average, or point-in-time measure without justificationHIGH
format.currency_unspecifiedexplicit currency format lacks required currency contextMEDIUM
configuration.no_granularityfull configuration exposes an empty supported-granularity setHIGH
configuration.invalid_enumstored value is rejected by the active tool/product schemaHIGH
insights.none_for_required_useall applicable insights are explicitly disabled despite a stated alert/explanation requirementMEDIUM
privacy.unsafe_row_labelrow-level display exposes sensitive/raw identifiers to the intended audienceHIGH

Adjust severity only with explicit impact evidence. Deduplicate identical rule/evidence pairs.

Contextual observations​

These are not automatically defects:

  • count of allowed dimensions or filtered metrics;
  • neutral favorable direction;
  • one/no comparison;
  • absent goals;
  • absent row-level configuration;
  • disabled individual insight types;
  • datasource update timestamp as a freshness proxy;
  • lack of a year-over-year comparison;
  • metric definition with zero accessible metrics.

Promote an observation to a scored finding only when the metric's decision contract shows a concrete failure—for example, no comparison on a metric explicitly intended to report period-over-period change. Document that rationale.

Scoring​

Start at 100 per fully assessed definition. Subtract 10/5/2/0.5 for CRITICAL/HIGH/MEDIUM/LOW and bound to 0–100. A partial definition may receive a provisional score but is excluded from the composite. Failed definitions receive no score. The composite is the mean of fully assessed definition scores.

Grade bands: A+ 97–100; A 93–96.5; A− 90–92.5; B+ 87–89.5; B 83–86.5; B− 80–82.5; C+ 77–79.5; C 73–76.5; C− 70–72.5; D+ 67–69.5; D 63–66.5; D− 60–62.5; F below 60.

Readiness is separate from the hygiene score​

Output includes per-definition readiness and overall_readiness. Any scored critical finding makes readiness not_ready, even if the numeric grade is high. Incomplete/failed coverage produces not_assessed unless a known critical finding already makes the result not ready. Fully assessed results with no critical findings return no_critical_findings, not a guarantee of readiness. Present this status before scores and prioritize critical repairs.