An Introduction to Sample Surveys

Sample surveys are widely used as a means of data collection for gathering information required for policy formulation on public issues, business decision-making and social studies. However, conducting a scientific sample survey is not simple. To ensure the reliability of the survey results, it is essential that adequate professional knowledge be applied in the planning and execution stages as well as in data processing and analysis.

This leaflet briefly introduces the following major steps in conducting a sample survey :

  1. Overall planning
  2. Design and selection of samples
  3. Design of questionnaires
  4. Collection of raw data
  5. Compilation and analysis of statistics and dissemination of survey results

(A) Overall Planning

  1. The survey objectives should be clearly specified.
  2.  In conducting a survey, professional input is often required. It is thus advisable to seek the opinions of professional statisticians right from the start, since rectification is normally very difficult to make when the survey is completed and its results are found invalid.
  3. The population of the survey, which refers to the totality of objects under study, should be clearly defined.
  4. It should be ensured that respondents of the survey are able to provide the information required.
  5. Adequate resources such as manpower, time and finance should be secured before deciding to proceed with the survey.

(B) Design and Selection of Samples

  1. Probability sampling methods should be used as far as practicable so that scientific inference can be drawn from the sample results about the entire population. Non-probability sampling methods, such as haphazard sampling, should be avoided as such methods are not scientific and bias normally exists in the survey results.
  2. icon2 There are different types of probability sampling methods. It is not absolutely necessary to adopt a sample design where the probability of selection of every individual respondent is equal. More importantly, the probability of selection of different respondents must be known. In case of unequal probabilities of selection, it is necessary to ensure that proper weighting methods are applied to give unbiased population estimates.
  3. A good sampling frame, from which sampling units are to be selected, should be acquired.
  4. In case households are taken as sampling units with household members as the enquiry units, a proper method by which the target household members are selected for enumeration should be adopted.
  5. The sample size depends on the characteristics of the population, the choice of sampling method and the desired level of precision of estimates. It can be worked out using established statistical methodology.
  6. Once the sampling units are selected, alterations are not allowed. This is one of the requirements of probability sampling.

(C) Design of Questionnaires

  1. Questions should be relevant to the survey objectives. They should be arranged in a proper order and the wordings should be appropriate, specific and precise. If the questions involve expression of opinions, care should be taken to avoid questions leading respondents towards the direction of a certain answer or questions being loaded in favour of a particular response.
  2.  Some respondents may reluctantly give answers to questions which they are unable to answer or do not hold any genuine opinions. Alternative answers such as "don't know" and "no opinion" should be included as options.
  3. Long questionnaires are undesirable.
  4. Questionnaires should be tested on some prospective respondents before they are finalised for use in surveys so as to ensure their effectiveness.

(D) Collection of Raw Data

  1. The modes of data collection include personal interviewing, telephone interviewing, online questionnaire and self-administered questionnaires by mail. An appropriate mode should be selected by carefully considering respondents' willingness to co-operate, the degree of complexity of the subject of enquiry and other relevant factors. Very often, mixed modes of data collection can be used.
  2.  Various arrangements related to fieldwork such as the proper allocation of workload among interviewers and the provision of adequate transport facilities, should be carefully planned to ensure smooth and efficient operation. Fieldwork procedures should be thoroughly tested before implementation.
  3. Interviewers should be properly trained on the concepts and definitions of terms used in the survey and carefully briefed on the work procedures before they start working.
  4.  Interviewers should ensure that respondents understand the questions. If in doubt, probing or giving further explanations should be done; but such should not be overdone so as not to exercise undue influence on respondents.
  5. Every effort should be made to achieve a high response rate, i.e. to minimise non-contact and refusal cases, because a low response rate may induce a serious bias in the survey results and is liable to mislead users of the results. Adequate publicity of the survey, proper introduction of the surveying organisation, clear identification of interviewers and giving advance notice to sampled respondents should help. Arrangements should also be made to follow up non-contact respondents and persuade uncooperative respondents to participate.
  6.  If the non-response rate is not negligible, methods should be devised to assist in the interpretation of survey results (such as to obtain some observable characteristics of the non-respondents).
  7. The survey progress should be properly controlled. Interviewers should be closely supervised and if possible, the contents of some of the completed questionnaires should be checked with the respondents.
  8. The identity and information supplied by individual respondents must be kept confidential. The survey results are to be presented in the form of aggregate statistics. With such assurance, respondents would feel at ease and be more willing to provide the required data. It should be emphasised that sample survey is a kind of statistical work and not an investigation on individuals.

(E) Compilation and Analysis of Statistics and Dissemination of Survey Results

  1. Raw data should be carefully and thoroughly checked before compilation of statistics.
  2.  Due deliberation should be made to decide on an appropriate subgrouping scheme in analysing survey results at some disaggregate levels, taking into account the precision of estimates derived from the sample.
  3. Appropriate statistical methodology should be adopted in compiling and analysing data.
  4. A complete survey report should contain details about different aspects of the survey, in particular details on population coverage, sample design, sample size, sampling error, response rate and likely sources of non-sampling error. Where appropriate, specimen of the questionnaire should be attached.
  5. When the survey results are released, sufficient details on the survey methodology should be provided in addition to survey findings, otherwise the readers will have no basis to assess the reliability of those findings.