How can public opinion be measured

The difference between probability samples where the inclusion probabilities for all units of the target population is known in advance and non-probability samples which often require less time and effort but generally do not support statistical inference is crucial.

Probability samples are highly affected by problems of non-coverage not all members of the general population have Internet access and frame problems online survey invitations are most conveniently distributed using e-mail, but there are no e-mail directories of the general population that might be used as a sampling frame.

Because coverage and frame problems can significantly impact data quality, they should be adequately reported when disseminating the research results. Due to the lack of sampling frames, many online survey invitations are published in the form of an URL link on web sites or in other media, which leads to sample selection bias that is out of research control and to non-probability samples. Traditional solicitation modes, such as telephone or mail invitations to web surveys, can help overcoming probability sampling issues in online surveys.

However, such approaches are faced with problems of dramatically higher costs and questionable effectiveness. Online survey response rates are generally low and also vary extremely. In addition to refusing participation, terminating surveying during the process, or not answering certain questions, several other non-response patterns can be observed in online surveys, such as lurking respondents and a combination of partial and item non-response.

Response rates can be increased by offering monetary or some other type of incentive to the respondents, by contacting respondents several times, and by keeping the questionnaire difficulty as low as possible.

The use of design features should be limited to the extent necessary for respondents to understand questions or to stimulate the response. The features should not affect their response as that would mean lower validity and reliability of data.

It is important that uncontrolled variations in how a questionnaire appears are minimized. Web-based survey methods make the construction and delivery of questionnaire instruments relatively easy, but what is difficult to ensure is that everyone sees the questionnaire as its designer intended it to be. This problem can arise due to the variability of software and hardware used by respondents. An important aspect of telephone polling is the use of interviewers. Telephone polling is also fairly cost efficient, depending on local call charge structure, which makes it good for large national or international sampling frames.

However, there are some disadvantages to telephone polling. For instance, there is some potential for interviewer bias e. Number of fixed telephone lines globally : This chart shows the numbers of fixed telephone lines from to There are three main types of telephone polling: traditional telephone interviews, computer assisted telephone dialing, and computer assisted telephone interviewing CATI. Privacy Policy. Skip to main content. Public Opinion. Search for:. Measuring Public Opinion.

Constructing Public Opinion Surveys An opinion poll is a survey of public opinion from a particular sample, and is designed to represent the opinions of a population. Learning Objectives Discuss how public opinions surveys are designed and executed.

Key Takeaways Key Points There are several ways of administering a survey, such as telephone, mail, online surveys, personal in-home surveys, personal mail or street intercept survey, and hybrids of the above.

A survey typically consists of a number of questions the respondent answers in a set format. There are open-ended and closed-ended questions.

An open-ended question asks the respondent to form their own answer, while closed-ended questions have the respondent choose an answer from a given option. The most important aspects of a survey include identifying and selecting potential sample members, contacting individuals and collecting data from those who are hard to reach, evaluating and testing questions, and selecting the mode for posing questions and collecting responses.

Self-Selection Bias : Although individuals chosen to participate in surveys are often randomly sampled, errors due to non-response may exist. Some prospective respondents may simply be less likely to respond to polls generally, they may not be interested in the subject, or they may be unreachable for a variety of reasons. Learning Objectives Identify the historical origins of public opinion research in the United States. Key Takeaways Key Points As The Harrisburg Pennsylvanian correctly predicted a Jackson win, such straw votes gradually became more popular, but they remained local, usually city-wide phenomena, until the early s.

The method The Literary Digest used to correctly predict the victories of Warren Harding in , Calvin Coolidge in , Herbert Hoover in , and Franklin Roosevelt in was mailing out millions of postcards and simply counting the returns. Key Terms The Harrisburg Pennsylvanian : The Harrisburg Pennsylvanian was the publication that conducted the first example of an opinion poll during the Jackson-Adams presidential race. George Gallup : George Gallup conducted a small but scientific survey that correctly predicted a landslide victory for Roosevelt in the Roosevelt-Landon race, thus establishing the Gallup Poll.

When the Digest conducted their election using an inaccurate sample causing them to predict the wrong winner, they lost all credibility and the Digest itself soon went out of business. The Gallup Organization Gallup Inc. Learning Objectives Locate the historical origins and significance of the Gallup Organization for public opinion research in the United States. Key Terms Gallup Poll : The Gallup Poll is the division of Gallup that regularly conducts public opinion polls in more than countries around the world.

The Gallup Organization : Gallup Inc. Today, Gallup, Inc. Learning Objectives Identify the purpose of national election studies. Key Takeaways Key Points The consistency of the studies, as in asking the same questions repeatedly over time, makes it very useful for academic research.

As a result, it is frequently cited in works of political science. Key Terms National Election Studies : The American National Election Studies is the leading academically-run national survey of voters in the United States, conducted before and after every presidential election.

Types of Polls The main types of polls are: opinion, benchmark, bushfire, entrance, exit, deliberative opinion, tracking, and the straw poll. Learning Objectives Compare and contrast the different types of polls utilized to measure public opinion. Tracking polls may also be used by news organizations to inform their own reports of the campaign. A push poll is an interactive marketing technique, most commonly employed during political campaigning, in which an individual or organization attempts to influence or alter the view of respondents under the guise of conducting a legitimate poll.

A straw poll, or straw vote, is a poll with nonbinding results. In meetings subject to rules of order, impromptu straw polls often are taken to see if there is enough support for an idea to devote more meeting time to it, and for the attendees to see who is on which side of a question.

Tracking Poll : A tracking poll is a poll repeated at intervals generally averaged over a trailing window. Exit Poll : An exit poll is taken immediately after the voters have exited the polling stations. Pollsters conduct exit polls to gain an early indication as to how an election has turned out. Conducting Polls Steps to conduct a poll effectively including identifying a sample, evaluating poll questions, and selecting a question and response mode.

Learning Objectives Describe the various methods taken by pollsters to conduct surveys. Key Takeaways Key Points A questionnaire is a series of questions asked to individuals to obtain statistically useful information about a given topic.

According to the three stage theory, or the sandwich theory, initial questions should be screening and rapport questions.

Key Terms Stratified Sampling : Stratified sampling is a method of probability sampling such that sub-populations within an overall population are identified and included in the sample selected in a balanced way.

Closed-Ended Question : A closed-ended question asks the respondent to pick an answer from a given number of options. Questionnaire : This is an example of a questionnaire. Analyzing Data A very important tool in data analysis is the margin of error because it indicates how closely the results of the survey reflect reality. Learning Objectives Discuss how data is broken down and subject to analysis after conducting surveys.

The confidence level, the sample design for a survey, and in particular its sample size, determine the magnitude of the margin of error. Sampling Techniques Sampling is concerned with choosing a subset of individuals from a statistical population to estimate characteristics of a whole population. Learning Objectives Compare and contrast the different sampling techniques used for opinion polls.

Key Takeaways Key Points Probability-proportional-to-size PPS is sampling in which the selection probability for each element is set to be proportional to its size measure, up to 1.

This approach can improve accuracy by concentrating a sample on large elements that have the greatest impact on population estimates. Maintaining the randomness in a sample is very important to each sampling technique to ensure that the findings are representative of the population in general. Panel sampling is the method of selecting a group of participants through a random sampling method and then asking that group for the same information again several times over a period of time.

This longitudinal sampling-method allows for estimates of changes in the population. Key Terms Systematic Sampling : Systematic sampling relies on arranging the target population according to some ordering scheme, a random start, and then selecting elements at regular intervals through that ordered list. Simple Random Sampling : A simple random sampling SRS is a sample of a given size in which all such subsets of the frame are given an equal probability to be chosen.

Stratified Sampling : Stratified sampling is a method of probability sampling such that sub-populations within an overall population are identified and included in the sample selected in a balanced way. The Importance of Accuracy Polling organization will lose credibility if they publish inaccurate results.

Learning Objectives Discuss the importance of maintaining accuracy when conducting measuring public opinion. Key Takeaways Key Points Relevance of the survey information, quality of the data, and overcoming personal bias are integral to polling accuracy. The quality of the accurate and timely results must be assessed prior to release. If errors in the results occur they should be directly corrected and the public should be informed as soon as possible.

When social scientists speak of good research the focus is on how the research is done rather than on whether the results of the research are consistent with personal biases or preconceptions. The Problems with Polls Problems with polls typically stem either from issues with the methodology that bias the sample or the responses that cause the bias. Learning Objectives Identify some of the common problems with conducting opinion polls.

Key Takeaways Key Points It is well established that the wording of the questions, the order in which they are asked, and the number and form of alternative answers offered can influence results of polls. Coverage bias is another source of error involving the use of samples that are not representative of the population due to the polling methodology. Self-selection bias arises in any situation in which individuals select themselves into a group, causing a biased sample with non-probability sampling.

Telephone and Internet Polling Internet and telephone polls are very useful as they are much cheaper than most other polls and are able to reach a wide population. Learning Objectives Identify the advantages and disadvantages of telephone and internet polling.

Key Takeaways Key Points Probability samples of internet polls are highly affected by problems of non- coverage. Not all members of the general population have Internet. In addition, online survey invitations are distributed using e-mail, but there are no e-mail directories of the general population.

Online survey may be affected by non-response as response rates are generally low and vary extremely. Some may refuse participation, terminate surveys during the process, or not answer certain questions.

There are some disadvantages to telephone polling such as interviewer bias, the fact it cannot be used for non-audio information, and it is unreliable for consumer surveys in rural areas where telephone density is low. Key Terms Internet Polls : Internet polls are becoming an essential research tool for a variety of research fields, including marketing and official statistics research.

Telephone Polling : Telephone polling is also fairly cost efficient, depending on local call charge structure, which makes it good for large national or international sampling frames. Licenses and Attributions. CC licensed content, Shared previously. Then it must survey a representative number of neighborhoods from within a city.

At a polling location, interviewers may have directions on how to randomly select voters of varied demographics. If the interviewer is looking to interview a person in a home, multiple attempts are made to reach a respondent if he or she does not answer.

Gallup conducts face-to-face interviews in areas where less than 80 percent of the households in an area have phones, because it gives a more representative sample. Most polling now occurs over the phone or through the Internet.

Some companies, like Harris Interactive, maintain directories that include registered voters, consumers, or previously interviewed respondents. If pollsters need to interview a particular population, such as political party members or retirees of a specific pension fund, the company may purchase or access a list of phone numbers for that group.

Other organizations, like Gallup, use random-digit-dialing RDD , in which a computer randomly generates phone numbers with desired area codes.

Using RDD allows the pollsters to include respondents who may have unlisted and cellular numbers. Questions about ZIP code or demographics may be asked early in the poll to allow the pollsters to determine which interviews to continue and which to end early.

The interviewing process is also partly computerized. Many polls are now administered through computer-assisted telephone interviewing CATI or through robo-polls. A CATI system calls random telephone numbers until it reaches a live person and then connects the potential respondent with a trained interviewer.

As the respondent provides answers, the interviewer enters them directly into the computer program. These polls may have some errors if the interviewer enters an incorrect answer. Robo-polls are entirely computerized. A computer dials random or pre-programmed numbers and a prerecorded electronic voice administers the survey. The respondent listens to the question and possible answers and then presses numbers on the phone to enter responses.

Proponents argue that respondents are more honest without an interviewer. However, these polls can suffer from error if the respondent does not use the correct keypad number to answer a question or misunderstands the question. Robo-polls may also have lower response rates, because there is no live person to persuade the respondent to answer.

There is also no way to prevent children from answering the survey. Lastly, the Telephone Consumer Protection Act made automated calls to cell phones illegal, which leaves a large population of potential respondents inaccessible to robo-polls. The latest challenges in telephone polling come from the shift in phone usage. A growing number of citizens, especially younger citizens, use only cell phones, and their phone numbers are no longer based on geographic areas.

The millennial generation currently aged 18—33 is also more likely to text than to answer an unknown call, so it is harder to interview this demographic group. Polling companies now must reach out to potential respondents using email and social media to ensure they have a representative group of respondents.

Yet, the technology required to move to the Internet and handheld devices presents further problems. Web surveys must be designed to run on a varied number of browsers and handheld devices. Online polls cannot detect whether a person with multiple email accounts or social media profiles answers the same poll multiple times, nor can they tell when a respondent misrepresents demographics in the poll or on a social media profile used in a poll.

These factors also make it more difficult to calculate response rates or achieve a representative sample. Yet, many companies are working with these difficulties, because it is necessary to reach younger demographics in order to provide accurate data.

For a number of reasons, polls may not produce accurate results. Two important factors a polling company faces are timing and human nature. Unless you conduct an exit poll during an election and interviewers stand at the polling places on Election Day to ask voters how they voted, there is always the possibility the poll results will be wrong. The simplest reason is that if there is time between the poll and Election Day, a citizen might change his or her mind, lie, or choose not to vote at all.

Timing is very important during elections, because surprise events can shift enough opinions to change an election result. Of course, there are many other reasons why polls, even those not time-bound by elections or events, may be inaccurate.

Created in to survey the American public on all topics, Rasmussen Reports is a new entry in the polling business. Rasmussen also conducts exit polls for each national election. Polls begin with a list of carefully written questions. The questions need to be free of framing, meaning they should not be worded to lead respondents to a particular answer. For example, take two questions about presidential approval.

Similarly, the way we refer to an issue or concept can affect the way listeners perceive it. Many polling companies try to avoid leading question s , which lead respondents to select a predetermined answer, because they want to know what people really think. Some polls, however, have a different goal. Their questions are written to guarantee a specific outcome, perhaps to help a candidate get press coverage or gain momentum.

These are called push polls. Sometimes lack of knowledge affects the results of a poll. A poll to discover whether citizens support changes to the Affordable Care Act or Medicaid might first ask who these programs serve and how they are funded. Respondents who cannot answer correctly may be excluded from the poll, or their answers may be separated from the others.

People may also feel social pressure to answer questions in accordance with the norms of their area or peers. If they are embarrassed to admit how they would vote, they may lie to the interviewer. This result was nicknamed the Bradley effect , on the theory that voters who answered the poll were afraid to admit they would not vote for a black man because it would appear politically incorrect and racist.

In , Proposition 19 , which would have legalized and taxed marijuana in California, met with a new version of the Bradley effect. Nate Silver, a political blogger, noticed that polls on the marijuana proposition were inconsistent, sometimes showing the proposition would pass and other times showing it would fail.

Silver compared the polls and the way they were administered, because some polling companies used an interviewer and some used robo-calling. He then proposed that voters speaking with a live interviewer gave the socially acceptable answer that they would vote against Proposition 19, while voters interviewed by a computer felt free to be honest. African Americans, for example, may give different responses to interviewers who are white than to interviewers who are black. The measure was defeated on Election Day.

One of the newer byproducts of polling is the creation of push poll s , which consist of political campaign information presented as polls. A respondent is called and asked a series of questions about his or her position or candidate selections.

In , a fracking ban was placed on the ballot in a town in Texas. Fracking, which includes injecting pressurized water into drilled wells, helps energy companies collect additional gas from the earth.

It is controversial, with opponents arguing it causes water pollution, sound pollution, and earthquakes. During the campaign, a number of local voters received a call that polled them on how they planned to vote on the proposed fracking ban. For example, Gallup had to change how interviewees were selected after predicting that Thomas Dewey would defeat Harry Truman in Fortunately, polling organizations learn from experience. Television stations often ask viewers to call so that they may express an opinion for or against a particular policy.

Newspapers and Internet sites also occasionally indulge in this form of entertainment. These gimmicks may be called "polls," but they are completely unscientific because respondents choose whether to participate, and the group that is motivated enough to do so will not represent everyone else in a community.

A key element of scientific polling, by contrast, is the representative sample, which requires that every possible respondent has the same probability of participating. This is accomplished today by using computers to dial telephone numbers randomly and then picking which person in a household to interview using another random method.

Obviously, no poll is perfect. But if the pollster succeeds at generating a random sample, then between 1, and 1, people will give an accurate picture of national opinion. The level of accuracy is often called the margin of error and indicates how much answers will bounce around the truth from poll to poll. Some people wrongly assume that the margin must include the truth, so if a poll estimates that 54 percent of Americans oppose the licensing of gun owners, with a margin of 3 percentage points, they assume the truth must lie between 51 percent and 57 percent.