Pilot Testing Add/remove

Purpose:

Pilot testing is an important tool to use when creating a interview, survey, usability test, or any other kind interaction with externals. The aim of pilot testing is to find out how questions or instructions are interpreted, and whether or not your questions provide neutral, unbiased answers. If you do not pilot test, you risk that the time and resources spent on the real test does not provide the answers you were seeking.

Step by step Facilitating tips Materials needed Goes well with Download template (pdf)

Tips to include participants who are not able to:

Hold

If a participant is unable to hold make sure to have a partner that can do the writing or use a software for writing that can be controlled with other means than hand gestures.

Touch

If a participant is unable to touch make sure to have a partner that can do the writing or use a software for writing that can be controlled with other means than hand gestures.

Move

If participants can not move or control movements, make sure to have a partner that can do the writing. This could be a helper, documenter, facilitator or other group members.

Focus

If participants have a hard time concentrating for the length of the test arrange little breaks or bring along a co-interviewer so that you can take turns asking questions and noting down.

Think

If participants are overwhelmed by guiding the test, make sure gave few, clear defined questions/instructions. It might be necessary to bring a helper or team member that co-assists while conducting the test.

Speak

If participants can not speak, tests could be conducted by bringing along a translator or a co-interviewer. Another possibility is to do the entire test in written form.

Hear

If participants can not speak, tests could be conducted by bringing along a translator or a co-interviewer. Another possibility is to do the entire test in written form.

Overview

Input

An idea for a test

Output

A thoroughly examined test

Complexity

Moderate

Time

1-2 hours

Participants

2-3

Activity

Core abilities:

  • Writing
  • Listening and focusing
  • Speaking

Step by step:

  1. Write down the questions and/or instructions for your test participants in the first column of the template.

  2. Find your pilot test participants! The upside of a pilot test is that it can be done with ‘convenience sampling’ – meaning it is okay to use colleagues, friends or like, if the objective of the pilot is to see if your questions are easily and correctly understood. Pilot testing does not require many participants – often, you will quickly see the flaws in your test set.

  3. Conduct the pilot test on the first participant. Optimally, one person talks to the participant, while another observes and notes down the answer in the relevant columns of the template.

  4. By the end of the test, ask the participant if they have any comments on the clarity of the questions/instructions.

  5. Proceed with the rest of the pilot test participants.

  6. When all of the pilot test participants have gone through, come together as a team and discuss your findings. Use the filled out template to look at patterns of what was clear, what was not, and what have you still not gotten answers to?

  7. Modify the test according to your findings!

When doing this method you should consider:

  • Remember that the pilot test participants do not count in your real following test.
  • Always let pilot participants know that you are making a pilot test. This way, it is easier for them to open up about uncertainties and critique of your test.
  • Take your time with pilot test participants and value their input. If they do not understand your questions, it is a sign that you have to change something.

Materials needed:

  • paper & pen
  • Template