BLOG
December 10th 2024

Herberts World #16

Issue No. 16 of the Mafo.de column ‘Herbert's World’! The topic this time: What do respondents actually think about their OWN role in the market research process these days? With all the digitalisation do they perhaps see themselves as nothing more than a stopgap for AI personas? Almost like a fig leaf for more and more synthetic data?

As always, Herbert has his very own opinion on this and also a warning to the industry. Have fun reading!

Humans first. Technology second. Does that still apply?

I am a market researcher with heart and soul, but for the first few lines of this column I am slipping into the role of an average citizen and consumer - a change of perspective that I can only recommend to all of us. But why at this point? So that I can better empathise with a potential survey participant.

How would I currently experience this situation if a research institute were to approach me today with a request to provide them with answers to a catalogue of questions online, by telephone or in person? How much have I been courted by colleagues BEFOREhand and subsequently informed, motivated, respected and fundamentally protected in my role?

I recently stumbled across this post by Karine Pepin on LinkedIn, which I spontaneously cite here as an example of a lack of empathy, mindfulness and appreciation:

Source: Karine Pepin auf LinkedIn

Back to me as a test subject: MY vote counts after all. Or does it not?

Am I really being made to understand in a positive way that without me as a human being, there is no way forward in this ‘cognitive gear called market research’? Is my voice given so much trust and weight that it can ultimately even make a difference? That it is worthwhile for me to invest time and make an effort?

My rather sober answer (now again as a market researcher): I'm not sure.

In times like these, why should a recruited consumer believe that their subjective views are valued in any depth by the institute and its clients? Sure, the respondent is kindly asked to participate and often receives a certain incentive in return, but how is our market research bubble currently perceived by such outsiders?

This bubble has changed dramatically in recent times: The number of surveys has been growing at an inflationary rate for years, including a variety of problems such as participation fraud or poor response rates. The use of new types of synthetic respondents or even complete AI personas, which basically devalue real people as panel members, has recently been added. In addition, new developments around popular LLM programmes such as chatGPT, which - at least apparently - have answers to EVERYTHING, are emerging almost daily.

The respondent needs to be told: You are needed. More than ever!

It is precisely these AI applications that are not only used for questionnaire design or subsequent data analysis (for which they actually seem well suited), but increasingly also for the collection - NO - the ‘production’ of data. The latter, however, is increasingly nibbling away at the value of real test subjects, who already suspect that they will no longer be needed at the end of this automatization and digitalisation. And if they are, then perhaps only as a corrective for the oh-so-clever findings of the ‘silicone samples’.

None of this is likely to have a motivating effect on the commitment of every research participant. Yet we need them! And today more than ever. For authentic opinions, genuine emotions, inspiring impulses and creative ideas.

Don't get the wrong idea: We can no longer work without the many new applications under any circumstances, especially if we don't want to leave the ‘field’ to the fraudsters in the battle with more and more bots and yes, the frauds. The same applies to the now incredibly good solutions for optimising processes before and after data collection (pre- and post-production in the end-to-end cycle).

They - the AI helpers - not only make us better in terms of quality, they also save us a lot of time with efficiency gains. This frees up resources that we can use for the really important issues in our industry.

One possible approach: the ‘Participant Bill of Rights’. One step in the right direction, for example, is the ‘Participant Bill of Rights’, launched by the Insights Association (IA) and the Council for Data Integrity. Market researchers and opinion pollsters should commit to binding and proactive compliance with 16 clauses in the interests of their respondents. For example, it is about their ‘right to withdraw’: so it should be simple and easy to withdraw from a study. And what about the ‘right to notification in the event of data breaches’ or the ‘right to feedback’, which the test subject can communicate (confidentially) to the institute at any time.

The charter postulates a fundamentally fair relationship at eye level between companies and participants at all touchpoints of a survey. That's good! But the following questions remain unanswered:

• Which body should monitor the implementation of the charter?
• Who can and should sanction offences?
• And who mediates in the event of problems and misunderstandings?

Market research is about give and take!

By the way: Functioning relationships between two parties are always based on reciprocity. So where there are rights, there are also obligations - also to promote mutual respect. The test subjects themselves ‘could’ therefore commit to pledging their own part in the success of the project. For example, with a commitment to honesty, conscientiousness and diligence. However, tact is required here, as we must not give the impression that all participants are under general suspicion of sloppiness or dishonesty.

If an incentive is paid, the question of its amount naturally always arises. Especially in the online panel area, where a rather symbolic amount can rarely be exceeded. Otherwise, any survey or study would quickly become unaffordable and, moreover, who wants to estimate what the raw data of a participant is ultimately really worth?

The fun stops with money. Or a ‘minimum wage’ for participants?

What we should strive for is a kind of ‘minimum wage’ on a monetary level, but primarily also on an informational and ethical level. So firstly, real incentives and not handouts, secondly, consistently respectful communication at eye level and thirdly, compliance with all ethical principles in the interests of the participants.

It applies: Human first. Technology second!

Back to the beginning: It was about valuing the role of (real) people within market research. Meaning ‘Human first. Technology second’ and not the other way round. However, I am almost certain that not all those in positions of responsibility see it this way and that they therefore invest even more in supplementing or even substituting participants made of flesh, blood and brains.

That is their right, but not my position. The clients of my institute have always asked me to ‘validate the data collected with reality’. However, reality is only found in real people and not in AI systems.

At this point I refer to the work of Mohammad Atari et al. (see chart) and quote from memory: ‘The further you are culturally from the US, the lower the correlation between GPT and population’. One reason for this is that much of the learning data in common LLMs appears to be Caucasian, male, affluent, over 60, and from the United States - the implied biases actually make me dizzy.

Secondly, these algorithms are almost all programmed in white-dominated Western countries in the Global North. You'll excuse my cynicism at this point, but here the apples are supposed to simulate the pears?!

Source: Mohammad Atari et al „Which Humans?” https://osf.io/preprints/psyarxiv/5b26t

Conclusion: Without the people, we are sawing at the branch we are sitting on.

We need to pay more attention to PEOPLE - the foundation of our work - precisely BECAUSE of all the progress. We need to pick them up more communicatively, motivate them, reduce uncertainty and value & protect their opinions! Not just from data theft, but by ensuring that their voices are not diluted in the clutter of artificial actors or fraudulent noise. It's up to us to keep reassuring them of just that!

They must never fear being relegated to being a fig leaf for mainly synthetic data. They still have the leading role in our business, as only they make true and new insights possible. Despite all our enthusiasm for technology, we should not forget this.

---