Survey Evidence Practice Note (GPN-SURV)
General Practice Note
1. Introduction
1.1 This practice note applies to any proceeding in which a party may seek to adduce evidence based upon out-of-court statements or responses of respondents to a survey ("survey evidence"). This practice note should be read together with:
(a) the Central Practice Note (CPN-1), which sets out the fundamental principles concerning the National Court Framework ("NCF") of the Federal Court and key principles of case management procedure;
(b) the Federal Court of Australia Act 1976 (Cth), including ss 37M and 37N (overarching purpose), and Part 23 of the Federal Court Rules 2011 (Cth);
(c) the Evidence Act 1995 (Cth) ("Evidence Act"), including Parts 3.1, 3.3 and 3.11; and
1.2 This practice note takes effect from the date it is issued and, to the extent practicable, applies to proceedings whether filed before, or after, the date of issuing.
1.3 Survey evidence may be adduced in a variety of practice areas. However, given that it is most often sought to be relied upon in the Commercial and Corporations and Intellectual Property National Practice Areas ("NPAs"), parties should also be familiar with the contents of the Commercial and Corporations NPA Practice Note (C&C-1) and Intellectual Property NPA Practice Note (IP-1).
2. Survey Evidence Generally
2.1 Survey evidence can give rise to a number of problems which may result in it not being admitted into evidence or being given little or no weight. The party adducing the survey evidence is seldom in a position where it can remedy problems arising out of poor design or execution of a survey during the course of the trial. Even if a party is in a position to remedy such problems, allowing a party to do so at a late stage of the proceeding may be unfairly prejudicial to the other party and/or unduly disrupt the proceeding. It is therefore important that, at an early stage in the proceeding (and before any survey is conducted), a party proposing to adduce such evidence give close attention to the survey design and the steps that will be taken to ensure that the survey evidence is both relevant and reliable.
2.2 Experience shows that a great deal of time can be spent at the trial on evidence and submissions concerning the admissibility and/or reliability of survey evidence which, if the evidence is excluded, or given no weight, can result in a significant waste of time and costs. Any party seeking to adduce such evidence has a responsibility to take appropriate steps to ensure that the survey that is proposed to be conducted will be properly designed and executed and that the data obtained will be properly recorded and analysed. A party that fails to satisfy the Court that it has done these things may find not only that the survey evidence is excluded or given little or no weight but that it is also required to pay the other party's costs of responding to the survey evidence, even if it is otherwise successful in the proceeding.
2.3 One purpose of this practice note is to establish a procedure for the giving of notice of a party's intention to adduce survey evidence in order to ensure that the other party is given a meaningful opportunity to respond by raising any issues it may have in relation to the proposed survey design or methodology and/or the relevance of any results that may be obtained to matters in issue in the proceeding before the survey is conducted.
2.4 Another purpose of this practice note is to provide a party wishing to rely upon survey evidence with some guidance in relation to matters that may bear on the admissibility of such evidence or the weight that it might be given if admitted into evidence.
2.5 Needless to say, it will always be a matter for the judge hearing a matter either during or prior to the trial to rule upon the admissibility of survey evidence. Among other things, it may be open to a judge in a particular case to exclude survey evidence pursuant to s 135 of the Evidence Act. Further, a judge may either dispense with or modify the application of this practice note in any particular case as the judge considers fit including, without limitation, the requirements of this practice note with respect to the giving of written notices.
3. Case Management
3.1 A party that proposes to conduct a survey which is to be the subject of survey evidence ("initiating party") should raise the matter with the other party ("responding party") prior to undertaking the survey. It may be that the matter to which the proposed survey is directed is not in issue or may be the subject of an admission that is acceptable to both parties thereby avoiding any need for the initiating party to adduce survey evidence.
3.2 Before the survey is undertaken, the initiating party must file and serve written notice of its intention to conduct the survey ("Survey Notice") which should include the following information:
(a) a precise statement of the issue or issues to which the survey will be directed including, in particular, any fact or facts in issue that the survey evidence will be relied upon to prove;
(b) details of the survey design including:
(i) the name and qualifications of the person or persons responsible for the survey design;
(ii) the nature of the survey (eg. telephone, on-line, face-to-face);
(iii) a precise description of the population to be investigated ("target population");
(iv) details of the sample frame from which the sample will be drawn, the sampling procedure and the sample size that will be used;
(v) a copy of any proposed survey questionnaire or instrument ("survey instrument") and any information or instructions to be given to survey participants or to any person who will administer the survey instrument;
(vi) the name and qualifications of any person responsible for coding, processing and analysing the data;
(c) details of any pilot survey that the initiating party or any researcher acting on its behalf has conducted or proposes to conduct for the purpose of designing or testing the survey instrument.
3.3 Within 21 days of receiving the Survey Notice the responding party should file and serve a written notice ("Responding Notice") which indicates with specificity what issues (if any) it has with the initiating party's proposed survey design and what design changes it proposes in order to address any such issues. In an appropriate case, a judge may determine that a responding party may not be permitted to rely upon any submission or evidence that raises an issue relating to the survey design not adequately identified or explained in the Responding Notice.
3.4 Parties are encouraged to engage in constructive discussions about any disputed features of the proposed survey design. Each party and its expert should reflect on the points raised by the other side and, where appropriate, modify their position with a view to reducing the scope of any dispute in relation to the design of the proposed survey.
4. Guidance
4.1 The admissibility of a survey, or the weight given to it, will be determined by a judge in light of all the relevant evidence. The purpose of this section of the practice note is to highlight some of the problems that have been found to diminish the quality and reliability of survey evidence so that the initiating party may avoid making some of the mistakes that have been made by parties adducing such evidence in previous cases.
4.2 Some potential problems that should be considered and, if possible, avoided by the initiating party include:
(a) failure to adequately identify what it is that the survey is designed to measure;
(b) use of biased or leading questions, preambles and/or excessive use of probing;
(c) use of questions open to different interpretations and/or poorly defined or ambiguous terms or concepts;
(d) use of a survey instrument that is excessive in length and/or complexity;
(e) inadequate testing of the survey instrument;
(f) use of a sample frame not sufficiently representative of the target population;
(g) use of an inadequate sample size;
(h) inappropriate or poor quality recording and/or coding of responses;
(i) excessive non-response and/or non-completion rates;
(j) inappropriate or poor quality data analysis;
(k) use of researchers for interviewing and/or coding of responses with insufficient training or experience;
(l) lack of adequate quality controls (eg. failure to review data for evidence of problems with participants' understanding of questions and/or the reliability of recording / coding of responses).
4.3 As to the design of the survey instrument, it is important to note that what may be acceptable in one case may not be acceptable in another. For example, in some circumstances it may be preferable to use open-ended questions rather than closed questions. Similarly, probing for additional and/or more specific responses may be acceptable in one case but unacceptable in another.
4.4 Whether or not a survey instrument is excessive in length or complexity is a matter that will usually be assessed having regard to the complexity of the subject matter of the survey, the sophistication of the target population, the survey methods used (eg. telephone, on-line or face-to-face) and the qualifications and experience of the persons administering the survey instrument. Except in cases involving a very straightforward design, it is expected that the proposed survey instrument will have been properly tested before it is deployed in the field.
4.5 As to the involvement of lawyers in the survey process, lawyers should only be involved with survey design to the extent needed to ensure that relevant questions are directed to the relevant population. Similarly, lawyers should, wherever practicable, not participate in the survey administration or the data analysis. Those conducting the interviews should be independent and objective and not affiliated with the initiating party.
5. Survey Reports
5.1 When presented, survey reports should provide an appropriate level of detail about the survey, including:
(a) the purpose of the survey;
(b) a definition of the target population;
(c) a description of the sampled population;
(d) a description of the sample design;
(e) a copy of the survey instrument;
(f) calculations and estimates of sampling error;
(g) clearly labelled statistical tables;
(h) a copy of the interviewer instructions;
(i) coding and related instructions;
(j) quality control measures; and
(k) details of any unforeseen problems encountered in the course of the survey work that might reasonably be expected to have an impact upon the quality or reliability of the data or results.
5.2 Any expert preparing such a report should also have regard to the requirements set out in the Expert Evidence Practice Note (GPN-EXPT).
6. Further Practice Information and Resources
6.1 Practitioners should familiarise themselves with the all of the Court's practice notes which are available on the court's website.
6.2 Further information to assist litigants, including a range of helpful guides, is also available on the Court’s website. This information may be particularly helpful for litigants who are representing themselves.
J L B ALLSOP
Chief Justice
25 October 2016
