NEWS
October 2nd 2023

Mixed Mode Research

The best of two worlds

The telephone as a means of market research? Has already been established for decades as a reliable and fast instrument for high-quality surveys. And the Internet? Just a few years after its mass compatibility at the beginning of the 2000s, it supplemented the survey standards of the time - for target groups with an affinity for technology who were either averse to a telephone survey or difficult to reach by telephone and who also wanted to determine the time and pace of the survey themselves.

Surveys by telephone - Computer Assisted Telephone Interview (CATI) and by web - Computer Assisted Web Interview (CAWI) have therefore been recognized for a long time and both have their advantages when it comes to researching specific target groups, products or opinions. Both methods are also excellent for topics that require larger samples, especially across national borders.

But both telephone and web have their methodological blind spots.

The answer to declining willingness to participate and heterogenous target groups: Mixed Mode

The fact that the basic willingness to participate has been declining for years across all target groups is no secret, but a problem that we market research institutes face every day. This is especially true for small niche samples, for which representativeness can only be achieved with considerable effort. In addition, clients are increasingly reducing their budgets, but still expect consistent quality, preferably with increased speed.

At this point, we, moweb research and AMR Advanced Market Research now present an innovative, because cross-method approach, which addresses the aforementioned challenges: Mixed Mode Research.

Mixed mode describes the simultaneous use of telephone and online surveys as part of a single market research project. The aim is to combine the advantages of each method and to unite the best of the different worlds in order to generate greater reach, better sample coverage and even higher data quality.

This applies not only to niche target groups, but also to particularly heterogeneous subjects who are to be interviewed for a survey. In this context, heterogeneous means that a sufficiently representative sample cannot be achieved with either method, or only with a great deal of effort.

With Mixed Mode, subjects have a choice - and therefore participate more often

Mixed Mode works with the parallel or sequential use of online and telephone surveys. We give our participants the choice: either the personal survey, conducted by a telephone interviewer, or the independent completion of a questionnaire on a digital device - in some circumstances even both! We create not only the possibility of choosing the method, but also the opportunity to change the medium during an ongoing survey or to continue or complete the process later with the preferred survey alternative. Even this flexible freedom of choice is appreciated by test persons as a service, which can have an additional positive influence on their willingness to participate.

Your gain in flexibility means an increase in influence on our side - as an institute. With so-called "CATI follow-ups" we can, for example, remind participants by telephone to (re)start a survey after their online recruitment. With "CAWI follow-ups" we mean the reminder by e-mail if the telephone contact was unsuccessful - with the request or offer of a telephone callback.

Mixed Mode reaches even heterogenous target groups - the example of a school

A school is probably not one of the first projects to come to mind when thinking about challenging and heterogeneous target groups. However, our client presented us with the task of surveying three target groups from a given type of school in a given federal state in very specific grades.

The subject of the survey: the teaching material used in class from a specific subject canon, i.e. the opinion on their selection as well as the evaluation of the material. A representative sample of the subgroups parents, students and teachers was to be surveyed - each with its own questionnaire.

Structurally, students, parents and teachers are anything but homogeneous. Students, i.e., young people, have a strong affinity for the Internet. For their teachers - which also applies to some extent to the generation of parents - this is only true to a very limited extent. Parents and teachers are therefore much more difficult to reach via digital media than students. In terms of numbers, teachers are also considerably smaller than the other two groups.

Mixed Mode increases exploitation rate & representativeness - for more quality

All three target groups are thus clearly defined and distinguishable, but highly heterogeneous in their sample size and media affinity and accessibility. Such differing survey preferences thus stood in the way of the "online-only" (CAWI) approach. However, a pure telephone survey was also out of the question, which was made impossible by the customer's insufficient budget.

At this point, we therefore used mixed mode from CATI and CAWI, in a three-stage process:

1. addressing all three groups via digital media (CAWI).
2. approaching teachers who could not be reached online via telephone (CATI).
3. completing the field phase with a minimum sample of parents and students via CATI, as a cross-check and verification of the previously collected responses.

In this way, we achieved the necessary representativeness of all three subgroups within the budget and time constraints and provided our client with an even higher response and thus data quality through the third field phase.

Mixed Mode surveys require sensitive design

As described, mixed mode enables effective market and opinion research, especially in difficult target group situations. However, this method is anything but trivial and requires specific technical, personnel and organizational prerequisites.

For example, the data sets from two different survey sources (CATI & CAWI) can only be coherently integrated, analyzed and processed if they are compatible with each other. This requires sophisticated data management through appropriate software solutions.

Furthermore, the identification and elimination of duplicates and multiple participants is another essential criterion for obtaining valid survey results. However, fraudulent respondents also try to express their opinion "multi-modally" across different survey forms in ONE survey. We effectively counteract this challenge with elaborate participant tracking or de-duplication of the data.

Biase as a challenge & haronization of questionnaires

Another sensitive problem across all survey forms: biases in the response behavior of respondents. Trained interviewers are aware of the famous social desirability bias, i.e. the danger that respondents tend to give answers that are socially desirable and thus may deviate from their actual opinion. Such a bias (there are many more - here is a short overview) can be avoided or at least reduced - already by the design of the questionnaire.

Compared to a telephone survey, an online survey has less of a problem with the social desirability bias, because the anonymity of the Internet and the fact of being unobserved usually "frees" the participant noticeably from influencing social pressure.

What is clear, however, is that biases play a role in both the face-to-face interviewer and the impersonal online survey, but they differ in their respective characteristics. The use of mixed mode gives us the chance to offset the disadvantages of one method with the advantages of the other. But be careful: the whole thing must succeed without ending up with questionnaires that are too different. However, with a precisely coordinated and overlapping design, we can ensure that the answers can be merged afterwards.

Mixed Mode for a better »respondent experience« (rx)

Cross-method surveys require a high degree of experience and competence in planning, implementation and evaluation. moweb research and AMR Advanced Market Research are well-established institutes and each has a leading position with its products for telephone-based (AMR) and Internet-based digital surveys (moweb).

Through the joint management of both companies, we not only bundle the relevant competencies of professional market researchers, interviewers and programmers, but have also brought interface management for the mixed-mode process described above to market maturity.

Surveys with highly heterogeneous target groups, each of which must be addressed with different media forms, require new approaches in order to nevertheless achieve valid, fast and cost-effective results. Mixed mode delivers exactly that: high exhaustion and response rates when individual survey methods reach their quantitative and qualitative limits. The weaknesses of individual modes are thus not only compensated for, in the end this approach even leads to a better »respondent experience«.

Conclusion: Mixed Mode for increased data quality

In times of decreasing budgets and nevertheless increasing expectations, we must not shake our claim - the best possible result for our business customers: And we do this by constantly remeasuring, deploying and optimizing our resources and competencies as market and opinion researchers.

This is necessary not least because society, markets and technical conditions (keyword: AI) are constantly increasing in complexity and we must also meet these challenges with new solutions.

In this context, Mixed Mode is just another product innovation to respond to this development and to be able to meet the wishes of our customers in the future as well.