All campaigns, whether commercial or social, always aim to change the opinions, feelings and behavior of those targeted. In order to design campaign strategies and materials, and to find out whether the desired changes actually occur or not, we need to measure the opinions, feelings and behavior of the target group before the campaign begins. The most recommended way is to conduct a quantitative survey by drawing a random sample, or commonly referred to as a baseline survey.

However, based on our experience since 2016, it is rare for a non-profit or civil society organization to conduct a baseline survey before starting a campaign. The reasons are various. Some do not know that a baseline survey is necessary. Some of them already know but are hampered by cost or time factors.

Baseline surveys with a large number of random samples taken are usually quite expensive and take a long time. Fortunately, there is a partial solution that can help us conduct baseline surveys easily, cheaply, and quickly. Although keep in mind, this method also has some limitations which we will also discuss here.

Baseline survey using Google Surveys

Google Surveys is a survey platform that can distribute online questionnaires to internet users who read content or use mobile applications on sites that are included in the Google Opinion Rewards for Publishers network. To be able to enter this network, site managers can register their sites, and if accepted, they will receive income from advertisements based on the number of questionnaires filled out.

Unfortunately, Google does not reveal how many sites in Indonesia are members of this network. But for comparison, there are around 52 thousand sites in Indonesia that “rent” space for ads from Google through AdSense, according to builtwidth.com. Usually sites looking for additional income starting from AdSense, and then some will also enter Google Opinion Rewards for Publishers.

Google Surveys distributes questionnaires to these sites, and randomly selected “selected” visitors will be asked to fill out a questionnaire. The method of “intercepting” site visitors is called web intercept, and is essentially the same as the method of selecting respondents by intercepting people in public places, such as mall intercepts. In 2018, the response rate from Google Surveys on online sites was 25% (meaning, 1 in 4 site visitors who were presented with a questionnaire eventually wanted to complete it).

In other words, essentially the sampling method from Google Surveys is convenience sampling, not random sampling. If the sampling method is convenience-based, how can we know the data collected can represent the population? First, let’s recall that the population of Google Surveys in Indonesia and in any country is internet users, not the national population. According to the Digital 2022 Global Overview Report compiled by We Are Social and Hootsuite, the number of internet users in Indonesia is 74% of the population or ± 202 million people. Second, to reduce bias, Google Surveys will automatically weight the survey results. For example, from the survey sample, the percentages of women and men are 52% and 48%, while in the population of internet users in Indonesia, the proportions are 50% and 50%, respectively. Google Surveys will assign a weight of 52 / 50 = 1.04 to all female respondents, and 48 / 50 = 0.96 to all male respondents. The same weighting will also be done for age.

If the sampling method of Google Surveys is convenience sampling, then how is it different from distributing survey questionnaires to an organization’s followers on social media? Yes, distributing questionnaires to followers is technically a convenience sampling method. However, this method contains a greater selection bias, because for someone who can become a social media follower of an organization, he or she may already be much more interested and understand the issues discussed by the organization. This selection bias will be significantly reduced in Google Surveys.

Three easy steps to create Google Surveys

Creating a survey using Google Surveys is not difficult, it only takes three steps. First, write questions (we assume the reader already knows how to formulate the questions in the questionnaire according to social research methodology); secondly selecting the target respondents; then the third to confirm and payment.

Step 1: Write the questions

If you have ever created a questionnaire in Google Forms, then you can certainly fill out questions in Google Surveys. However, it should be noted that the maximum number of questions in Google Survey is 10 items. If your survey requires screening questions, then you can only enter a maximum of 4 questions, and you can only add 6 more questions.

What does it mean? Some surveys require screening questions to select respondents to fill out the questionnaire, usually based on demographic profiles (gender, age, region), behaviour, interests, or attitudes. In screening questions, respondents who choose the appropriate answer can continue to fill out the questionnaire (screen in), while respondents who choose an inappropriate answer will stop filling out the questionnaire (screen out). For example, the survey was conducted targeting respondents with an age range of 19-39 years. People who chose answers in that age range could continue to fill out the questionnaire, while others could not. If further granular age data is required, demographic (not screening) questions about age can still be made.

Figure 1: Writing Questions

Step 2: Selecting target respondents

It should be noted that Google Surveys has a policy of not targeting respondents under the age of 18. In addition, unlike other countries, Google Surveys in Indonesia does not yet have a feature that can target respondents based on criteria based on gender, age, and geography (targeted audience). Therefore, if you want to target certain respondents, you must use screening questions with a maximum number of 4 items.

Figure 2: Selecting Target Respondents (United States)

Figure 3: Selecting Target Respondents (Indonesia)

Step 3: Confirmation and payment

Overview of Google Surveys

Prior to payment, the survey is sent to the GS team for review. The review process from the GS team can take some time. Here are some things to keep in mind so that the review process can run more smoothly:

1. The use of spelling, grammar, and capital letters must be correct.
2. Questions about demographic data need to include “opt-out” answers such as “do not want to answer”.
3. When a question cannot be applied to everyone, it is necessary to include an “opt-out” answer such as “none” or “other”.

Other policies can be found on the following Google Survey Help page.

The policies of the Google Surveys team need to be well understood because their feedback sometimes doesn’t explicitly say what went wrong and needs to be fixed in a question. If the survey creator is familiar with Google Surveys policies, it’s easier to understand what went wrong with rejected questions.

Iteration with the Google Surveys team lasts a maximum of 3 times, if more than that Google Surveys informs that the survey cannot be run. If you are just creating a survey with Google Surveys for the first time and it has been more than 3 iterations and your survey was rejected, this can be circumvented by creating a new survey with the same content. The trick is to click copy on the old survey and resubmit it for review by the Google Surveys team.

Payment

The thing to note about payments is, until now payments can only be made with Google Surveys accounts that are set to come from the United States with payments using USD. If you make a payment with an account originating from Indonesia, an error will appear stating that the country and currency values used do not match.

Google Surveys prices are cheaper when compared to conventional surveys

Google Surveys pricing is calculated per completed response. The completed response is when a respondent fills out the survey completely, from the first question to the last question. The price for each response is IDR 15,000. Even if you use a screening question, the price will still range from IDR 45,000 – IDR 75,000 because it is distributed online. This is cheaper when compared to conventional research which can reach IDR 170,000 – IDR 500,000 per respondent.

Google Surveys results can be seen in minutes when they start sharing

Google Survey results can be viewed once the survey is approved and distributed by the Google Surveys team. Results are updated within minutes, when the first respondent finishes filling out the questionnaire. Surveys for 100 respondents can be completed in 5 days, while for 500 respondents it can be in 10-20 days. This is quite fast when compared to conventional surveys, where for 3000 respondents it takes 6 months, which means 500 new respondents can be obtained in 1 month (30 days).

However, Google Surveys is only limited to simple research

Although easy, cheap, and fast, Google Surveys is only suitable for simple research where the respondent population is Internet users. You should only generalize the findings from Google Surveys to internet users in Indonesia, and not the population or even Indonesians.

If you plan to campaign only using internet media (eg social media, websites and blogs, advertisements in media and social media, search engines), using Google Surveys for baseline surveys can be tried. On the other hand, if you are doing research whose findings must be generalized to the population level, don’t use Google Surveys.

The Google Surveys questionnaire can only accommodate a maximum of ten questions including screening questions. In addition, GS cannot accommodate skip logic questions that can make respondents have a certain question line based on what they answered in the previous question. It is this limitation that makes Google Surveys suitable only for simple research.