Are you counting records, messages or distinct people?
Duplicate handling changes the meaning of a research count. Two retrieved copies of one post are different from two comments by the same account, and neither establishes that matching handles on different platforms are one person. Choose the unit before deduplicating, then preserve enough provenance to explain every merge or exclusion.
Deduplicate repeated source records using platform and original identifiers where available. Report records and distinct platform accounts separately, preserve original references and ranks, and do not merge cross-platform handles without evidence or fetch replacement accounts.Define the counting unit
State whether the output counts source records, content items, platform accounts or independently established people. An audience usually needs distinct account identities, while a content brief may legitimately keep several posts by one author. Do not remove repeated authors merely because you intended to remove duplicate rows; those are separate operations with different effects.
Use identifiers before text similarity
Where available, compare the platform and original record identifier or canonical source URL to identify repeated retrieval of the same item. Retain the observation context when it matters. Similar wording can be a quote, a common question or a separate event, so do not merge independent records solely because an agent finds their text alike.
Keep cross-posts and identity links explicit
Label cross-posted content as related evidence without inventing shared ownership between accounts. Require inspectable evidence before collapsing cross-platform identities. Keep the original source references even if the presentation groups repeated wording into one theme. This allows reviewers to distinguish a theme's message frequency from the number of independently observed authors.
Publish an auditable count
Report the starting record count, the counting unit and why exclusions were made. Keep original ranks or references when the task requires a frozen ordering. Do not fetch replacements just to recover the old count after removing a duplicate. If deduplication creates a shortfall against a requested audience size, state the new actual count and the remaining gap.
Keep the boundaries clear.
- Repeated authors and duplicate records are not the same thing.
- A deduplicated shortfall must remain visible.
Start with a read-only check: ask your agent to report actual source coverage before collecting records or exporting an audience.
Primary references
Documentation reviewed 2026-11-13. Product capabilities, client availability and plans can change. Third-party marks identify their products, not an affiliation.