When should an evolving codebook stop changing during a comparison?
An exploratory review of public home security discussions may develop new categories while reading. If the team then compares apartments with houses, later-reviewed material could receive more detailed labels simply because the taxonomy matured. Freeze the relevant rules before comparison so differences do not merely reflect when the reviewer encountered each source set in the workflow.
Freeze the supplied security discussion taxonomy before comparing explicitly stated installation contexts. Recode earlier material consistently and keep new out-of-framework mechanisms separate.Identify the comparison boundary
State which groups or conditions will be compared and which codebook version applies. Include only conditions explicitly described in the evidence. Exclude inferred living arrangements or household characteristics. The comparison should concern documented contexts, not private attributes assigned by a reviewer attempting to complete a convenient segmentation table for the final presentation.
Review earlier material consistently
Apply the frozen definitions to all included groups, including records read during exploratory development. Keep a record of changed labels. Do not assume that early coding remains comparable merely because category names stayed the same; a definition may have narrowed or expanded while retaining its original short title in the spreadsheet.
Park novel mechanisms visibly
When new evidence does not fit, mark it outside the frozen framework and preserve it for a possible next version. Avoid forcing it into an existing category or quietly revising only the later group. A meaningful new mechanism can be reported qualitatively without corrupting the consistency of the current structured comparison between source sets.
Deliver versioned comparisons
Present the results under the frozen framework and a separate note on novel evidence. If a revision is essential, recode all affected groups before replacing the comparison. State whether observed differences are robust to the change instead of implying that a more detailed later taxonomy simply discovered more problems in the group reviewed last.
Keep the boundaries clear.
- Stable category names do not guarantee stable category meanings.
- Badges identify a research context; they do not promise access or complete evidence.
Start with a read-only check: ask your agent to report actual source coverage before collecting records or exporting an audience.
Primary references
Documentation reviewed 2026-11-13. Product capabilities, client availability and plans can change. Third-party marks identify their products, not an affiliation.