We do a lot of user testing here at Evantage. A typical project can have a few rounds of testing. It’s that love of creative solutions for complex problems that leads us to find out if users totally get it or if they are struggling to complete a task we had all high-fived over when the prototype was initially designed. Lots of user testing ensures that we are certain that users will spend time on the site/application/device and find everything they need.
Performing all this testing requires users. Depending on the number of personas or profiles I create for a client this could be anywhere from 7-20 people for each round of testing. This doesn’t seem like much and clients often think that they can save the recruiting costs if they do it internally, but I find that hiring a recruiting agency to find users is more efficient and even cost effective in the long run.
Internal recruiting seems like a great option – as a client you may have a staff person with a few hours of free time on their hands and there is a big fat user list of people who already use your service. Why not have the consultant train that staff person to place a few calls and get a few users for a test? It won’t take that long, right?
Wrong! It takes loads of time to recruit users. Just to schedule one person you will probably call about 100 people. Potentially more, if you can only call during the day and have difficulty reaching people at home. Add to that a staff person who did not think their job would include making phone calls trying to convince people they are not selling anything. Job satisfaction scores could go down and the likelihood of getting that same staff person to recruit again is small.
Don’t get me wrong, there are times when internal recruiting works just fine. If all the users you need to recruit work for your company, they have more of a vested interest in participating; especially if the new design will make their jobs easier. Internal recruiting can also go well if the staff person does customer service calls and enjoys talking to someone who is not complaining about the company.
If you want to do recruiting internally, I have a few guidelines I’ve put in place:
- Review the screener with the person who will be recruiting so both of you know exactly who you are recruiting for. I learned this recently when I started testing and realized one segment of users was not correct. The staff person then had to find this new segment of users, and I had to push out the test for another two weeks.
- Make a schedule of the dates, times and questions in the screener.
- If you are doing a mix of remote and in person testing, make sure the person recruiting understands which days are set aside for each activity and that remote participants need to be told that they need to be near a computer at the scheduled time.
- A week before testing starts, ask to receive updates daily on how recruiting is going so there are no surprises by test time.
- Review the data from all recruits to make sure they are appropriate to the segment you are recruiting for.
- Start recruiting three weeks before the test and have someone doing it full time. Even if a client doesn’t think it will take that long, I would rather be proven wrong with two weeks to spare rather than rescheduling the test.
But really, when an experienced recruiting agency comes along and can provide me with tips for improving the screener, concerns about how difficult recruiting for a study can be based on my segment needs and can find the appropriate users without too much effort, it makes my job a lot easier.
Evantage works with The Leede Group to do all our recruiting. Their experience has made recruiting users so much better. Their estimates are spot on even when the recruiting itself is particularly difficult. They work quickly to find more users if there are more no-shows than expected or issues with one of the segments.
It’s rare when the users they have recruited don’t fit the screener (and if there are some we don’t have to pay for those users). Not only can they find us the right users nationwide, but if we are testing in another state they have a list of other firms with facilities in other cities and countries. The biggest time savings I see from the process isn’t just associated to the time it takes to call, it’s also the personal follow up they do calling the users and sending them a letter in the mail. By the time the user receives my reminder email, it’s already on their radar.
If you have a specific user list you want to recruit from, that doesn’t exclude you from hiring a firm to do the recruiting for you. If your users are based nationally it saves on the cost of buying lists and the recruiting firm won’t keep the list after recruiting is finished. If you do end up buying a list for nationwide locations, you can reuse it for up to a year without added cost.
The additional time it will take to do recruiting internally would best be spend in the hands of the professionals. Part of it is probably associated to how well I’ve been trained to have Leede refine the screener, set up a schedule and find the right people. I appreciate how quickly they can respond to my needs for a test, so testing can be completed the week I’ve planned it and not run into multiple weeks. I appreciate that I don’t need to spend so much time during the recruiting process making sure everyone knows the recruiting criteria. But most importantly, your staff person will appreciate you for not requiring them to do it.
Tags: Recruiting
