Fairly recently, a usability testing service was launched called Feedback Army. Unlike other testing services which generally have a number of people working for them doing the reviews, Feedback Army utilizes a service called Mechanical Turk from Amazon.
Mechanical Turk is a marketplace for work where you can post quick jobs and get them completed in minutes by any of the thousands of people who use it. Feedback Army uses an API to hook up with the service, post your usability questions there and quickly get feedback. The prices start at $7 for 10 replies to $33 for 50.
Raphael, the creator of Feedback Army, has contacted me a few days ago to see what I think. He gave me a free coupon for 10 replies, so I went ahead and tried it out. Here’s what I found…
I’ve used my recently launched logo gallery site, Logospire, as a test subject. The submission process is simple – just specify the url and list a few questions. The questions can be open or targeted, for example: “Did anything confuse you?” or “Can you quickly sort the entries by most recent first?”.
The questions
Here’s what I asked:
Hello, I’m collecting feedback on my website. Your answers let me know if my site is sending the right message.
- What does this site do?
- Can you easily find how to browse by latest submissions first?
- Is it clear how the site works?
- Any aspects of the site that confused you?
The results
I’ve received feedback within minutes of the submission, and the full 10 replies were filled up very quickly. Here’s a sample of three replies I received. All replies were fairly similar – some more thorough than others:
- The purprose of this website is for designers to have a feedback on theirs logo designs uploads.
- Yes, but the "upcoming" button is not clear for me...
- For me it is pressy clear.
- I don't manage to find a clear page/profile for a particular user, for example to contact him if i want to hire him for a logo job.
- It creates a gallery of logos that other people can rate.
- No. I can see that the logos can be sorted by popular and upcoming. I am assuming upcoming means gaining in popularity?
- Sort of--you register and then you can vote to rate the different logos.
- I see that you can add a comment about a logo but I only see the number of votes for each logo, no comments.
- This site posts logos
- It is very easy to browse
- The site is not very clear on how it works. Should include a title that maybe is more descriptive.
- It is hard to understand what the site is for by just going to it.
I think the feedback provided is actually pretty good value for money. The feedback clearly identified one or two issues I should improve – e.g. it’s not clear how to sort logos by most recent first and I should probably include a tagline somewhere in the header explaining what the site is about.
Obviously, a few cents for each reply isn’t going to get you thorough feedback, but people did seem to really try out the site and write what they think. They may not have spent much time on my site to really figure things out, but that’s likely how much time other people will spend on the site as well – if they can’t figure it out, they will just leave.
Conclusion
Overall, I liked Feedback Army and may use it in the future. This sort of service isn’t going to get you high quality feedback, but that doesn’t mean it has no use. On the contrary, for targeted and general things like testing out certain functions of your navigation or seeing if the visitors understand what your site is about it seems perfect – as long as each question doesn’t take much time to answer.
The speed of feedback is impressive, taking just minutes from the time of submission. The final thing to note is the demographic is going to be very mixed, so the people testing out your site may not be anywhere near your target audience, which may be an issue for niche sites.
Anyone used Feedback Army for their projects? What did you think?