Skip to content

Check whether combining subgroups reverses a comparison

Inspect stable task categories before interpreting an overall difference that may be driven by a changed mix of reviewed cases. Preserve subgroup limits.

Guides

Could a combined result hide different patterns within tasks?

Two periods of public support discussions can show a larger overall share of blocked tasks even when each comparable task category appears less problematic. A changed mix of easy and difficult tasks can create that apparent reversal. Before describing a worsening experience, examine whether the aggregate result combines materially different kinds of cases across the periods.

Example research prompt
Inspect supplied pairing and firmware recovery cases by task and period. Compare aggregate and within-task patterns using exact eligible counts, leaving unsupported proportions unresolved.
01 / Guides

Define meaningful strata

Use explicit task categories such as initial pairing and firmware recovery when those categories matter to the decision. Include only records that can be assigned under a shared rule. Exclude guessed user expertise as a stratification shortcut, and retain uncertain task categories separately instead of placing them wherever the comparison looks cleaner.

02 / Guides

Reconstruct exact reviewed counts

Build a table of supported outcomes within each task category and period using the actual eligible records. Do not derive missing denominators from rounded prose or partial result pages. If the available evidence cannot support exact proportions, show descriptive counts and mark the comparison unresolved rather than fabricating a precise reversal analysis.

03 / Guides

Compare mix and within-category patterns

Inspect whether one period contains more difficult task categories and whether the same outcome definition applies throughout. Report aggregate and category-level views together. A reversal can be a property of the reviewed set without proving any underlying causal mechanism, and small or selected categories may remain too unstable for a strong directional conclusion.

04 / Guides

Deliver a composition warning

Explain whether the overall difference could reflect task mix and which conclusions remain valid within comparable categories. Avoid choosing whichever view supports a preferred narrative. If categories or outcomes cannot be reconciled, recommend a better-structured follow-up rather than publishing one combined metric that hides the evidence needed to interpret its apparent direction responsibly.

Keep the boundaries clear.

  • An aggregate comparison can change when the mix of reviewed tasks changes.
  • Available records and platform badges do not establish complete source coverage.

Start with a read-only check: ask your agent to report actual source coverage before collecting records or exporting an audience.

Primary references

Documentation reviewed 2026-11-13. Product capabilities, client availability and plans can change. Third-party marks identify their products, not an affiliation.

Know what social media shows about you.
Start with a question.

Get started
montage