Read on for our ESOMAR 28

1. What experience does your company have in providing online samples for market research?

RONIN has performed multi-country data collection studies for over 25 years.

In the early years, this research was conducted solely by phone. Web-based data collection was introduced as early as 1999 and now accounts for well over half of the business activity.

RONIN has an internal, GDPR-compliant database of medical specialists, tech, and business decision-makers, including senior opinion formers such as policymakers, journalists, academics, and business leaders. We continue to maintain a 110-seat CATI Centre to support our multichannel delivery capabilities and recruitment efforts. RONIN also offers targeted digital recruitment for participants seldom available via panels and can target by job title, company, location and key skills, to verify eligibility ahead of an online or phone interview.

In 2023, RONIN launched RONINEdge—our brand new platform for senior decision-makers and thought leaders who have opted in to future participation in market research activities.

2. Please describe and explain the type(s) of online sample sources from which you get respondents.  Are these databases?  Actively managed research panels?  Direct marketing lists?  Social networks?  Web intercept (also known as river) samples?

Our internal participant database has been built from phone recruitment by RONIN’s experienced telephone interviewers who contact respondents at their place of work or, via digital recruitment where eligibility and qualification is known ahead of contact and invitation.

At the point of interview, respondents are invited to take part in future market research studies if they so wish. If CATI recruitment to online, then various subscribed company databases are used to identify appropriate organization, for example we subscribe to the D&B Worldwide Database, Crunchbase, WINMO, and CISION amongst others.

For digital recruitment we have subscription services which utilise LinkedIn, Twitter, Reddit, Stack Overflow, and similar sites for both advertising campaigns to targeted individuals or for organic, one-on-one recruitment by our in-house digital research team. Desk research of company websites and listings are also used to build robust company sample frames.

RONIN does not use river sampling or web intercepts.

3. If you provide samples from more than one source: How are the different sample sources blended to ensure validity?  How can this be replicated over time to provide reliability?  How do you deal with the possibility of duplication of respondents across sources?

Sample from different sources is cleaned, recoded to fit our internal database fields and de-duplicated against existing database records. Once the sample is used either in online or phone studies, its original data is checked and overwritten, when applicable, using actual survey data. Our internal database is designed to retain as little personal data as possible.

Sample for CATI calling is deduplicated by company name, phone number and address ahead of calling. The same deduplication process is followed for client supplied or internal sample.

For online studies hosted by RONIN we rely on the Imperium product line to carry out IP address deduplication, including additional fraud-related checks on participants.

4. Are your sample source(s) used solely for market research?  If not, what other purposes are they used for?

Yes, RONIN only conducts market research projects in accordance with the Market Research Society (MRS) Code of Conduct and the guidelines of ESOMAR.

5. How do you source groups that may be hard to reach on the internet?

RONIN uses multiple approaches to recruit hard-to-reach audiences, one of our specialisms.

Our desk research team will support our project managers and clients in building an initial sample frame of suitable participants, using secondary research to sense check the universe size, best channels to approach and recruit, and build initial company lists.

We then have the option of using our London, Lancaster and Hamburg CATI Centre teams of highly-trained interviewers working in over 40 languages at native level to interview by phone or recruit via phone to online studies, after an initial telephone screener—including reminders.

Alternatively or in addition to our CATI teams, RONIN also has a digital recruitment team used to identify niche groups via targeted recruitment to phone or online interviews, using the most appropriate platform or network for your target audience, at speed and at scale.

6. If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners?  Is it your policy to notify a client in advance when using a third party provider?

If RONIN needs to work with a local supplier for sample, for example in markets where applicable data protection law prevents us from collecting personal data from a third country, or in in China, where local firewalls can restrict non-domestic internet access, then we rely on a vetted network of carefully selected partners. RONIN remains fully responsible for the quality of all deliverables commissioned with us and carries out quality control on all partner work.

In such circumstances then clients are informed at costing stage.

Additional sample, if ever required during fieldwork, will come only from vetted and approved partners. For example, RONIN has ongoing relationships with suppliers for consumer online sample and will disclose their use to a client in advance. Partners are selected through recommendations, meetings, adherence to relevant industry standards and experience. All projects are run in accordance with IQCS, ISO 20252:2019, ISO 27001, MRS, BHBIA, Insights Association and ESOMAR standards. Suppliers must comply with RONIN standards and have signed Master Services Agreements covering all aspects of the partnership including confidentiality and quality, and flowing down any terms agreed with clients to them.

7. What steps do you take to achieve a representative sample of the target population? 

Sample selection varies depending on client requirements and project specifications.

Where possible, profiled sample is used. As a rule, RONIN aims at interviewing a widely spread sample in geographical, industry and job function terms. Sample is randomized before being put into field and quotas are implemented as required by our project specification.

For sample pulled from company databases such as D&B then a random sample will be selected based on available companies within sector, size, or location.

For RDD sample within the national population then RONIN uses dual frame sample of cell and landline numbers matching current recommended splits.

RONIN uses Forsta (previously known as Confirmit) as its data collection software for online and CATI sample management and survey administration and can set robust quota control to ensure the required targets by quota (size, location, sector, etc.) are met and not exceeded.

8. Do you employ a survey router?

No, we do not employ a survey router.

9. If you use a router: Please describe the allocation process within your router.  How do you decide which surveys might be considered for a respondent?  On what priority basis are respondents allocated to surveys?

RONIN does not use a survey router.

10. If you use a router: What measures do you take to guard against, or mitigate, any bias arising from employing a router?  How do you measure and report any bias?

RONIN does not use a survey router.

11. If you use a router: Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers?

RONIN does not use a survey router.

12. What profiling data is held on respondents?  How is it done?  How does this differ across sample sources?  How is it kept up-to-date? If no relevant profiling data is held, how are low incidence projects dealt with?

Any information held by RONIN without explicit participant consent is at a company level only.

If a participant agrees to future contact, then information is retained including phone number, email address, job title and information held at the screening stage of the completed interview.

For low incidence projects, we can either carry out initial desk research to support our targeting efforts based on our understanding of the market or sector.

13. Please describe your survey invitation process.  What is the proposition that people are offered to take part in individual surveys?  What information about the project itself is given in the process?  Apart from direct invitations to specific surveys (or to a router), what other means of invitation to surveys are respondents exposed to?  You should note that not all invitations to participate take the form of emails.

Invitations are kept simple and straightforward.

No matter the recruitment methodology, participants are told the following at recruitment, from the moment initial contact is made up until the start of the survey.

All invitations are designed to encourage participation without prompting participants to self-select, or in such a way that their participation might be skewed or biased in any way.

Our internal project number/reference

A general description of the purpose of the project

The estimated length of their participation time (LOI)

The closing date for participation (if applicable, as an estimate)

The full incentive offered and any terms and conditions applicable (including who will administer it, what it will be, and when they will receive it)

The name of the client on whose behalf the invitation is being sent

Purpose for which the data will be used

Retention period applicable to the personal data we collect

Reassurances that their individual answers will remain confidential

Reassurances that their answers will remain anonymous and not linked to them, OR where this is not the case, information that their responses will be identifiable in the reporting

Reassurances that their participation is voluntary.

The name of our company and our contact details

A link to our privacy notice or a way for participants to access it easily

A method for participants to unsubscribe or opt out from the market research project

If cookies will be used, and for what purpose

 

14. Please describe the incentives that respondents are offered for taking part in your surveys.  How does this differ by sample source, by interview length, by respondent characteristics?

Incentives are of a specified monetary amount and vary according to interview length, seniority of participants, countries involved, and the level of effort required of participants. Incentives for the healthcare studies are subject to relevant Fair Market Values (FMV) if these were communicated to us at costing stage.

Incentive amounts will be specified in any cost proposal separately from all other costs, including the administration cost of the incentives.

We can also offer to make charity donation in lieu of an incentive, with certain types of participants who cannot accept monetary incentives directly.

15. What information about a project do you need in order to give an accurate estimate of feasibility using your own resources?

To establish an accurate estimation of feasibility we require the following, at minimum:

Countries

Information about the audience size

The incidence rate to be expected or screening criteria

Sample size required

Length of interview

Information about any quotas and sub-quotas applicable

RONIN will also mark the assumed qualification incidence for any provided cost.

Example potential participant profiles (anonymized) are also regularly provided at costing stage to ensure that the correct target audience has been assumed.

16. Do you measure respondent satisfaction?  Is this information made available to clients?

We defer to our clients if they wish participant satisfaction with a questionnaire or survey to be recorded, as we only use client-provided or client-approved materials in our surveys.

Our Quality Controllers regularly communicate with participants in their respective language.

Participants are invited to provide feedback on the quality of the survey during the interview or at the end of the interview. Participation, completion, and drop-out rates are measured to assist in assessing participant satisfaction with a particular survey or with all RONIN studies.

We also conduct separate surveys with our members to understand how satisfied they are with their relationship with RONIN and surveys in general.

17. What information do you provide to debrief your client after the project has finished?

Clients are constantly updated on survey progress and completion figures, including giving them access to bespoke live reports through online portals that track key metrics. After the project has finished, a debrief can be organised to cover all relevant points, such as feedback from fieldwork teams, participant experience, screener design and quota design, sample structure, or indeed any element likely to affect research quality and outcomes.

A Project Manager is allocated to every project by RONIN and all client communication will be from this PM during project set up, fieldwork and up until completion.

18. Who is responsible for data quality checks? If it is you, do you have in place procedures to reduce or eliminate undesired within survey behaviours, such as (a) random responding, (b) Illogical or inconsistent responding, (c) overuse of item non-response (e.g., “Don’t Know”) or (d) speeding (too rapid survey completion)? Please describe these procedures.

Quality is at the heart of RONIN’s business.

Ahead of fieldwork, surveys are assessed carefully by the Survey Compliance team to identify anything that may result in ambiguous or erroneous data (poor wording, lack of routing validations, insufficient instructions, etc.) against rigorous survey quality checklists.

Data on all projects hosted by RONIN is quality checked and analysed on an ongoing basis by Project Managers and our DP team. Duplicates are removed and undesired behaviour is identified through median Interview length, trick questions, pattern responses, quality of open-ended questions and comparing cumulative and individuals’ data collected. This includes looking at ‘unrealistic’ close-ended data, for example by cross-referencing potentially conflicting data points in the survey or validating brand awareness trends in each market.

If necessary, respondents may be re-contacted by phone to verify data.

We also place emphasis on a carefully checked and implemented screening questionnaire to ensure that the correct participant is verified ahead of survey completion.

19. How often can the same individual be contacted to take part in a survey within a specified period whether they respond to the contact or not?  How does this vary across your sample sources?

This generally varies depending on the audience involved.

General public: call attempts are usually restricted to 5 calls.

Business-to-business: call attempts are usually restricted to 7 call attempts.

For email invitation reminder emails are usually restricted to 2 reminders post-invitation.

Once a definitive outcome status is recorded – the recorded is automatically disabled.

20. How often can the same individual take part in a survey within a specified period?  How does this vary across your sample sources?  How do you manage this within categories and/or time periods?

Unless the study is a designed as a multistage study or interview, or as a pre/post-test, then an individual will only be permitted to take part once in any study.

Participants are only invited to another study once any incentives have been paid.

Our database management software can restrict the regularity of contact from one study to another, typically we will only contact on a 3-monthly basis, unless the client project requirements demand otherwise.

21. Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your survey respondents?  Are you able to supply your client with a project analysis of such individual level data?

RONIN maintain individual participant engagement history and logs any relevant information on participant behaviour. This is tracked through individual ID numbers assigned to completed interviews (and indeed all sample) when contacted on a project. On client request, participant history can be provided to clients in an anonymised form.

22. Do you have a confirmation of respondent identity procedure?  Do you have procedures to detect fraudulent respondents?  Please describe these procedures as they are implemented at sample source registration and/or at the point of entry to a survey or router.  If you offer B2B samples what are the procedures there, if any?

For CATI projects all participants are verified over the phone and regular monitoring and back checking takes place to Interviewer Quality Control Standard (IQCS) requirements.

For online interviews then phone numbers can be recorded for verification and back checking via our CATI Centre, as required, and through separate desk research if necessary.

For digital recruitment then all invited participants are verified at the point of contact and screened ahead of interview.

23. Please describe the ‘opt-in for market research’ processes for all your online sample sources.

RONIN applies a double opt-in methodology.

Participants are recruited at their place of work by our CATI Centre or by our Digital Recruitment team with preapproved GDPR-compliant scripts and reviewed consent statements. Participants who wish to take part in future research are asked profiling questions over the phone and sent an invitation email with the link to the registration site thereafter.

Our platform allows participants to confirm their identity, order incentives and to update their details. Every survey invitation will provide additional information to allow them to consent to taking part in an informed and compliant manner.

24. Please provide a link to your Privacy Policy. How is your Privacy Policy provided to your respondents?

RONIN has a published privacy policy (https://www.ronin.com/privacy/). The company is also ISO 20252:2019-certified and all work is compliant with relevant standards, including local and international laws and regulations. This includes MRS, ICC/ESOMAR/BHBIA guidelines and standards. The panels are audited on a yearly basis as part of the ISO 20252 registration. The policy is provided to participants whenever they take part in a study hosted by RONIN (with a link at the bottom of the screen) and in survey invitation (with a similar link at the bottom of the e-mail), or provided over the phone at the start of our CATI studies.

25. Please describe the measures you take to ensure data protection and data security.

RONIN is an ISO 27001-certified and Cyber Essentials Plus-certified organisation.

We have a significant range of measures in place to ensure data protection and data security across our entire operations, both at rest and in motion. These include, but are not limited to, a physical security policy, information security policy, background screening practices, access control, technical vulnerability management, organisational security and documentation, secure configuration and asset hardening, encryption, and data leakage management.

Robust policies are in place to ensure that access to all company, confidential, or personal data is on a need-only basis and users with elevated access rights are logged by Group IT.

All access rights are reviewed on a regular basis to ensure the business justification for a particular set of rights remains relevant. Any remote access to servers is done via SSL VPN and activity is monitoring constantly by software-based measures and Group IT team.

26. What practises do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents?

The final decision whether it is appropriate to show sensitive material to online participants is generally the final decision of the client, but our Compliance Director can provide advice and guidance. RONIN will advise the client accordingly once aware of such material.

27. Are you certified to any specific quality system? If so, which one(s)?

RONIN is certified to ISO 20252:2019 and to ISO 27001. We also hold Cyber Essentials Plus certification, the UK Government’s cybersecurity programme.

We maintain a corresponding Quality Management System (QMS) and Information Security Management System (ISMS) to deliver on these requirements. They cover our end-to-end market research data collection operations and are subject to an annual external audit.

To view our ISO 20252 certificate, please click here.

To view our ISO 27001 certificate, please click here.

28. Do you conduct online surveys with children and young people? If so, do you adhere to the standards that ESOMAR provides? What other rules or standards, for example COPPA in the United States, do you comply with?

We do not conduct research with children/young people (as defined in local markets).