In our gamification design work we often recommend setting up a pilot test, to test that are design assumptions are indeed correct and appropriate for the audience. In our design process a pilot test comes after we have established objectives for the gamification design given the company culture and the environment it operates in. It also typically comes after our user research phase. The reason for this is that we want to make sure our design is taking into account the crucial factor of setting up for results and understanding your player.
When it comes to pilot tests, they are not the final roll-out. These often are mocking-ups of what the design will look like or tests of the functionality with really crude looking wire frames. We have also carried out pilot tests on paper, whiteboards and even powerpoint. The most important is that we choose a representative group of people to experience it and have a look at it. Representative of the majority of the workforce, a mix of gender typically speaking, a mix of seniority and a mix of how uses may differ among the target group. For example if we are working on a gamified learning scenario, we want a few learners, their managers and then a combination of HR and learning team members who may be the administrators for the tools.
Presenting your prototype to the pilot group can be via weblink to tools such as Invisio, sometimes we use learning authoring viewers, other times we are present in person to elicit responses especially if we have gone for paper or whiteboard solutions. We tend to have a survey set-up to capture the likes and dislikes and the obvious stumbling blocks or niceties as perceived by the pilot testers. We aim for an 80% or above approval rate and when feedback came back several times from users then we will also tweak the design to ensure the go-live version has an optimal structure based on the feedback.
The size of the pilot group can vary, if we are working on a multinational scale, we want people in all location to be represented. If it is a secret roll-out, where the majority is kept out of the information loop, then we work with the project team and a small selection of fresh eyes, who need to sign confidentiality agreements. In any case always ask at user research, whether people want to be part of the pilot test group who get’s to see everything. That way you set yourself up for a good sample size for your pilot group. It is rare if the ones giving input into user research totally opt out in further parts of the roll-out other than time constraints, it’s the lovely curiosity motivation that wants us to find out what our input is influencing.
When you complete the pilot test, look at the data collected objectively and then decide on your strategy of what you will change, what you will keep and how to go about your roll-out in practical terms for the whole organisation.
If you want us to run your gamification design scenarios, by all means get in touch, we would love to assist!
