How researchers pick data skews what they find about politics
A study of Swedish parliamentary speeches reveals that different data collection methods produce wildly different pictures of who speaks about policy—even when the actual content stays similar. For policymakers and analysts relying on text analysis to track political debate, the finding is a warning: your conclusions are only as reliable as your sampling strategy.
Originaltitel: How You Sample Determines What You Find: Investigating Bias in Parliamentary Data Sampling Methods
<p>This study addresses the issue of sampling error within research on subsets of parliamentary text corpora. Two samples of parliamentary speeches relating to the marketisation of the Swedish education system, drawn thr ough different sampling techniques, are analysed and compared. The analyses find that diverging sampling methodologies can be complementary as each method adds substantial quantities of unique documents to the dataset. Further, the diverging sampling methodologies employed produce documents with similar semantic content. However, analyses of the distributi on of speeches between party affiliations and speakers indicate va st differences between the two samples. These results indicate that sampling frames can substantially influence the findings of parliamentary text analyses. We conclude that combining different sampling techniques can be a way to reduce the ris k of sampling error, which in turn can have a strong influence on the conclusions drawn from analyses of parliamentary texts. </p>