On the trail of educational bias

The low participation of people with a low level of education in social science surveys is a difficult problem to solve for both the survey institutes and the researchers. The GESIS - Leibniz Institute for the Social Sciences has organized a workshop in which the causes of this educational bias were investigated across institutions and solution strategies were considered and discussed. The results are to be saved in the form of an article and thus also made available to non-participants of the event.

Around 30 social scientists came together in Mannheim on November 9-10, 2023 for the workshop entitled "Educational bias in (self-administered) cross-sectional and longitudinal surveys in Germany" and contributed their experiences from numerous large longitudinal and cross-sectional studies in Germany. These included, among others the General Population Survey of the Social Sciences (ALLBUS), the Bundesbank Online Panel - Households (BOP-HH), the German Ageing Survey (DEAS), the German Center for Higher Education Research and Science Studies (DZHW), the European Social Survey (ESS), FReDA, the German Emigration and Remigration Panel Study (GERPS), the German Internet Panel (GIP), the German Longitudinal Election Study (GLES), the National Educational Panel Study (NEPS), the Panel Labor Market and Social Security (PASS), the Programme for the International Assessment of Adult Competencies (PIAAC), the Robert Koch Institute's Health in Germany Today (GEDA) study, the Survey of Health, Ageing and Retirement in Europe (SHARE) and the German Socio-Economic Panel (SOEP).

In a first step, an attempt was made to develop a common understanding of the education bias. This showed that the proportion of low-educated people in the population varies depending on the definition and criteria used to measure the level of education. For example, the classification can be based on ISCED or the highest school-leaving qualification. As the Federal Statistical Office's microcensus is often used as a reference value, there are correspondingly different deviations. However, regardless of how high the bias is based on the criteria used, the workshop participants are certain that people with a low level of education are underrepresented in samples and that the data is therefore distorted. And this bias has actually increased in the last decade, according to the observation.

There are many reasons for this. Here is a list of possible explanations, which can certainly be expanded.

Language and cultural barriers:

Even the cover letter of a recruitment wave only reaches some of the selected addressees. Sometimes letters cannot be delivered, sometimes people throw the letter away. This makes it all the more important that the recipients who actually read the letter are addressed correctly in the text and encouraged to participate. Language and comprehension difficulties could prevent low-educated people in particular from participating in the survey at the recruitment stage. Overall, reading skills in this population group tend to decline.
In addition, the wording of the questions in random samples can be complex. Sometimes technical terms are used or specific knowledge is requested. Some topics touch on the cultural understanding of the respondents. In addition, working through a questionnaire in full can require a certain amount of discipline and patience - a long survey is likely to deter people with a low level of education in particular.

Incentives:

Incentives do help with recruitment, but they have very little effect in terms of the willingness of people with a low level of education to participate. Even a very high incentive often only leads to around 8 to 9 percent more people with a higher level of education taking part. The participation of people with a low level of education only increases by around 1 percent.

Survey mode:

People with a low level of education are most likely to be persuaded to take part in a survey through personal contact with an interviewer or in a face-to-face interview. If they have the choice between a paper questionnaire and an online survey, many low-educated people prefer the paper version. It is possible that people with a low level of education have less access to computers and the internet, perhaps they feel insecure when using online tools. However, many studies rely primarily on online surveys.

Distrust of institutions:

People with a low level of education may have reservations about authorities and institutions, perhaps because they have concerns about the use of their data and data protection. They may lack sufficient information about reputable survey institutes and research institutions.

Lack of time:

Low-educated people may be more likely to work in jobs that require less time flexibility or longer working hours. As a result, they may have less time to participate in surveys.

Selective willingness to participate:

People with higher levels of education may be more willing to participate in surveys because they have a greater understanding of the importance of surveys and the value of research. They may also be more interested in social issues. Low-educated people, on the other hand, may be less interested in social science studies because they do not recognize the usefulness of these surveys.
Attempts to increase the entertainment value of a questionnaire, i.e. to increase the fun of answering the questions, have so far been unsuccessful. Gamefication, especially in online surveys, has not led to higher participation among people with a low level of education.

Strategies to reduce the education bias

Targeted inclusion of people with a low level of education in samples could be achieved through several strategies. The main aim is to reduce potential barriers and facilitate participation.

Cover letter:

To avoid the letter with the cover letter being thrown away unread, the envelope itself could be designed in such a way that it catches the eye and arouses curiosity and interest. The cover letter itself should be written in simple language. This means not using any technical terms, avoiding complicated sentence structures and keeping the text short and clear. In terms of content, the importance of individual participation should be emphasized as concretely and comprehensibly as possible, and the purpose of the data collected should be clear to the addressees.

Incentive:

A small prepaid incentive is useful to increase the overall willingness to participate. However, the incentive alone will not reduce the educational bias.

Survey modes:

Face-to-face interviews would probably be ideal for increasing participation among the low-educated. As a cost-effective alternative, participation via paper questionnaires should be possible. To reduce the education bias, it would be helpful if the survey were kept short and only required a short amount of time. The questions and explanations should be written in simple language. A further incentive for participation could be to address relevant topics for this group of people in order to arouse their interest.

Discussions with focus groups are probably useful to understand why people with a low level of education often fail to participate.

The exact reasons for the non-participation of this population group are still largely unknown. What is certain is that there is no single measure to counteract the education bias, but that work needs to be done on several levers at the same time. Although ideas already exist, the individual strategies still need to be evaluated and there is currently a lack of evidence and publications. The workshop participants agreed that more basic research is needed here. An initial contribution in this context will be a scientific paper on this workshop.