We’re starting to see more and more new tools appearing for the purpose of conducting usability testing. A while back I wrote a review of UserFly, a web app that lets you record your visitors actions, from mouse movements to clicks and keyboard strokes, and then play it back as if you’re sitting behind the user’s screen.
The biggest problem with UserFly is that the information you receive is fairly basic. Yes, it’s objective – you see exactly what the users are doing, without the pressure of a developer standing behind their shoulder examining their every action. But at the same time, it’s very one dimensional – you don’t know what the user is thinking and you don’t know how they feel – you can only guess.
OpenHallway is a tool that lets you perform usability testing without having to be in the same room as the tester. How? By recording all of the users screen and sound. Users participating in OpenHallway tests are asked, just like in real usability tests, to speak their mind as they browse the website or use the app. They’re also asked to perform a series of actions – so everything they do is focused on certain things they have to accomplish. This makes OpenHallway a solution that comes very close to that of actually performing in person user testing.
Main screen of OpenHallway
So how does it all work? What the developer does is first set up a project and then a scenario on OpenHallway. The scenario is basically a list of instructions and a URL for the testers who will perform the test. You then get a link that you email out to your testers. When a tester clicks on that link they’re taken to a page with your instructions and a button to start recording their session.
The instructions page that testers will see
The link to the specified URL will be opened in a new window so they can always come back to check the instructions and also end the recording when they’re done. OpenHallway then is a simple way to record user tests without your testers having to install any software.
The video playback page
Is there any downsides to testing remotely? Yes. The main downside is the loss of two-way communication. You cannot ask the user questions or encourage them to speak out. Speaking out every thought as you browse a website isn’t the most natural thing in the world, so it often takes a bit of encouragement for this to happen. If the user behind the screen is holding back their feelings and thoughts, there will be no developer standing by to help bring them out. Neither will you be able to read the testers body language – although you may be able to pick up certain cues ustration.
The other thing I should note is that OpenHallway solves just one of the two parts of user testing – the other being the testers themselves, i.e. having to find them. There are some tools that make it easier, for example, Ethnio is an app that lets you to put a little widget on your site to recruit testers. It will be interesting to see if OpenHallway expands into this area by introducing some tools, or a marketplace, to allow developers and testers find each other with the ease of a few mouse clicks – then, the remote testing solution would really be complete.
But OpenHallway is not UserTesting.com who offer the whole solution by recording videos and writing reports on your site. OpenHallway is a focused tool that keeps your options open and allows you to recruit anyone you like. It also allows testers to do the tests without having to install any software, which means that anyone can do it.
Even though I mentioned a couple of issues, I really like OpenHallway. It’s very simple and it works. There is a free plan you can use to set up one test scenario and see how it works for yourself. Paid accounts begin at $49 , which will give you 1 GB of storage (about 20 ten minute tests), up to $199 a month for 9 GB (about 180 ten minute tests). Go ahead and check out OpenHallway site if you’re interested – there is a screencast there as well that shows the app in action.