How has the PULS been done?

The importance of probability sampling

The PULS uses probability sampling. This means that all adults in Victoria living at residential addresses have a chance to be included in our sample – and that we know what that chance is. This sets it apart from non-probability approaches where some in the population have no chance of selection, which means you have only partial information about the relationship between your sample and the population. Probability sampling is typically more difficult and expensive but is important in ensuring the data allow us to generalise our findings across the Victorian adult population.  

Sometimes non-probability approaches (like opt-in online panels or those using quota, convenience and purposive sampling) claim to be ‘representative’. Looking like the population of interest (e.g. on the basis of similar demographics) is not the same as being representative. While people using non-probability approaches often generalise their findings to their population of interest, it is rarely appropriate to do so.  

To be truly ‘representative’ and able to generalise findings to Victoria as a whole, you need to use probability sampling.

Choosing the right number of respondents

We chose 6,000 respondents to maintain a balance between the ability to look at individual problems or social and demographic groups in detail and value-for-money. Particularly when we look at everyday legal problems, numbers of respondents need to be high enough to be able to talk about experience with confidence. This is particularly the case for rarer problem types, and 6,000 respondents gives us scope to explore the data in a wide variety of ways.

Gold standard survey

Surveys can be conducted in a range of ways and they all have their pros and cons. Face-to-face, telephone, online, postal surveys, and combinations of these, vary in cost, speed, response rate, burden on respondents, the questions you can ask, and how complex the questionnaire can be.  

Face-to-face surveys are generally considered the ‘gold standard’ for a survey like the PULS, yielding higher quality data than other modes of delivery. They typically have higher response rates and allow longer and more complex questionnaires, while the presence of an interviewer can help to engage respondents. Face-to-face surveys also have excellent coverage, with only a small percentage without a chance to participate. Interviewing in person can introduce some bias (e.g. more socially desirable responses when interviewed in person), though interviewers can also clarify and further explain to the respondent and encourage more information.  

Face-to-face surveys are however the most expensive, and in large geographic areas costs can be prohibitive. Victoria’s manageable distances mean we can take advantage of the benefits of face-to-face administration without prohibitive fieldwork costs.