The effect of nonresponse and measurement errors on turnout estimates. Experiences from using administrative linked data in the Norwegian General Election Survey 1969–2021

Abstract

Estimates on electoral turnout from election surveys often shows a considerably higher share of voters compared to the official turnout. A well-known phenomenon from survey methodology is ‘social desirability bias’: people will try to represent themselves in a way that reflects well on them. But this is not the only source of error that should be taken into consideration. Estimates are not equal to the true values because of variability (due to random effects) and bias (systematic effects). A survey will be exposed to both sampling errors and non-sampling errors. Sampling errors occur because only a subset of the population is selected. Non-sampling errors, like nonresponse and measurements errors, apply to all statistical processes. Nonresponse error is the difference between the statistics computed from the collected data and those that would be computed if there were no missing values. Measurement errors occur during data collection and cause the recorded values of variables to be different from the true ones. We use data from the Norwegian General Election Survey 1969–2021, with linked administrative records for the respondents on electoral turnout, population demographic characteristics and questions from the surveys. Data from administrative records enable us to study the direct effect of measurements errors and nonresponse errors. Our results show that when we produce estimates on electoral turnout from surveys, the surveys will almost certainly show a higher percentage of voters when compared with official results. Both nonresponse bias and measurement bias (over-reporting) contribute to this. The two error sources pull in the same direction. Nonresponse bias is of much greater concern than measurement bias (over-reporting). Our analysis shows an agreement rate (survey*register) between 92 and 99 per cent from 1969 until 2021. On average, the agreement rate is 96 percent, which is high compared to other survey variables. Still, over-reporting always occurs; voters are more liable to over-report than under-report. We speculate that this has something to do with ‘social desirability bias’. Although the overreporting is relatively low for the whole group of respondents, it is considerable higher for subgroups like young respondents with low education. The same subgroup that has a lower response rate and the largest nonresponse bias. We also demonstrate that respondents who claim they have voted in the election but are not confirmed by the administrative register seem to be spread across parties. Respondents who answer that they have voted in the election in the survey, but will not reveal their party choice, are by and large confirmed to be voters by the register.