In the hectic world of quickly evolving technologies and incredibly high competition between online businesses, success often depends on the convenience of web interfaces. Easy-to-use (“usable”) websites have higher conversion ratios and thus generate more revenue.
The ease of use and learnability of a software application is commonly known as “usability.” Usability takes into consideration the non-functional aspects of using software, in particular:
Effectiveness (task completion by users),
Efficiency (task in time), and
Satisfaction (response by user in term of experience) in a specified context of use (users, tasks, equipment, and environments).
There are three main evaluation criteria we used to assess all reviewed tools in this article:
Ability to record audio and video during the process of testing
Tools for post-editing of testing sessions
Ability to use mobile devices for testing
After completing this research and comparing ten different popular tools, we came to the conclusion that we still lack a comprehensive, fully integrated usability testing tool for mobile devices with the following features:
Capturing mobile device screen interaction
Recording all actions of testers as they work on given tasks (audio and video)
Performing testing anywhere the tester goes, at any time
Testing any smartphone or tablet (as long as it has a camera)
Real time streaming of the screen and audio/video of the tester
Currently, there are some very good ways for mobile usability testing, but we will focus on them more in our next post.
Tool/Feature
Tester Audio and Video Recordings
Recordings can be edit-ed
Mobile testing capable
Remote un-moderated testing
Testers available
Price
Ovo Logger
√
√
√
×
×
$3,000 and up depending on scope of associated usability lab
Ovo Solo
√
(but tool is PC only)
√
×
×
×
$99
Morae
√
√
×
×
×
$1,995
UserTesting
√
√
√
√
√
$49 per test with bulk pricing dis-counts available
Userlytics
√
√
√
√
√
$39 per month or more with bulk discount pricing available
Userfeel
√
×
×
√
√
$39 per test with bulk discount pricing available
OpenHallway
√
√
×
√
×
$49 per month or higher, de-pending on the plan
UsabilityTools
×
×
×
×
Yes, but on-ly paid pan-els
Starts from 300$ per month. 14 days free trial
TryMyUI
√
×
×
√
×
$35 per test with bulk discount pricing available
Ovo Logger by Ovo Studios is designed mainly for usability testing studios that conduct high volume testing with teams all over the world. Why? Basically because their purposes need a robust tool to capture, edit and share results. Ovo Studios also cre-ates custom usability testing labs, and the Ovo Logger is partly designed to fit those labs perfectly. Although the tool scores better than comparable tools because of its depth of technology, it can be a bit expensive to deploy.
Advantages:
Enterprise level tool
Records multiple inputs from PCs or Macs and allows sending streams to multiply reviewers
Editing tools enable real-time analysis of tests
Allows saving important points while reviewing the recording
Can capture untethered iPads, non-jail broken iPhones and Android devices if using the Logger mobile app
Disadvantages:
Designed specifically for larger firms with multiple locations and or usability teams
Requires a dedicated PC to run the software
Pricing is more expensive versus other usability testing tools
The tool is based on number of sources, associated lab requirements, etc.
Tester Audio and Video Recordings
Recordings can be edited
Mobile test-ing capable
Remote un-moderated testing
Testers avail-able
Price
√
√
√
×
×
$3,000 and up depending on scope of asso-ciated usabil-ity lab
The Ovo Solo tool by Ovo Studios is a solution for smaller corporations. We can even say it’s alite version of Ovo Logger that enables screen interaction video capture along with the testers’ faces and voices. Simple logging controls enable analysis of the resulting test videos. However, the tool only supports PCs, with no iOS support. On the bright side, its affordable price makes it a good choice if budgets are tight.
Advantages:
Relatively low price point makes this a very affordable way to capture, edit and display usability testing sessions
Disadvantages:
This tool is PC only
No ability to record Mobile device screen interaction directly
Editing tools are basic and may not meet more advanced usability tester needs
Morae from Techsmith was the first tool specifically created for usability testing and it still remains one of the best despite its age. It allows you to record, analyze, edit and share sessions. It resides on a practitioner’s computer (PC only, sorry iOS fans) as an application and records all screen interactions, including the tester’s voice and face if using a webcam. Testers can be on any Mac or PC when testing websites or applications. All documentation, editing and sharing tools are included.
Advantages:
Powerful screen capture and editing tools, so it’s good for larger scale projects
Includes additional tools that can track a variety of screen and click interac-tions
Able to incorporate survey- and opinion-based data
Sharing test sessions with other practitioners and testing results with teams is relatively easy and quick
Disadvantages:
No ability to record Mobile device screen interaction directly
Relatively expensive and PC-only (not Mac compatible) for the Manager and Recorder tools
UserTesting provides a network of pre-screened testers who can meet several specified demographics, but also allow practitioners to invite their own testers using the tool. Practitioners create a set of tasks for the testers to complete, either on a website or competitor site. Testers’ screen interactions and voices are recorded. UserTesting includes powerful and easy-to-use video editing tools to create highlight reels. UserTesting also provides mobile usability testing, which is a major feature considering the increasing utilization of mobile devices. Written responses to questions are included. UserTesting is one of the most robust tools to use for remote unmoderated usability testing.
Advantages:
Affordable way to conduct remote unmoderated usability testing
Records audio and video of the tester
Highlight reels are created easily and quickly by means of included editing tools
Testing session results are available in an hour or two as the network of testers is large
Demographic filtering tools are provided
Disadvantages:
Since UserTesting pays every user a small fee for every completed test, results may become skewed or inaccurate if testers rush through testing without paying much attention
You must to keep track of all passed tests and request a redo if you are concerned about the quality of the tester (however, using your own testers can alleviate much of this)
No ability to record Mobile device screen interaction directly
Tester Audio and Video Recordings
Recordings can be edited
Mobile test-ing capable
Remote un-moderated testing
Testers avail-able
Price
√
√
√
√
√
$49 per test with bulk pric-ing discounts available
Userlytics provides both PC and Mobile based remote usability testing that unfortunately cannot be moderated, although if you need mobile testing features, you have the ability to upgrade to the tool’s Gold or Enterprise plans. Testers may be sent either by Userlytics panel or by your own sources. Screen interactions and tester voice recordings are available, as well as webcam video images of testers along with written answers to post-test questions.
Advantages:
Relatively low cost
Easy to set up and use
Download of recordings for editing and sharing is available on upgraded plans
Disadvantages:
Unbiased questions can appear as in any other remote unmoderated usability testing tool
A rather confusing set of test functionality based on paid plans that will take time to get acquainted with
There are also limits in terms of number of post-test questions that can be asked based on plans
Tester Audio and Video Recordings
Recordings can be edited
Mobile test-ing capable
Remote un-moderated testing
Testers avail-able
Price
√
√
√
√
√
$39 per month or more with bulk discount pricing available
The biggest advantage of this tool is its provided assortment of testers that speak languages other than English. Needless to say that this feature is essential for websites that provide services targeted toward other countries. Written responses to post-test questions are also available. Videos of the test are available on the website, however you cannot download and edit them. The white label playback option of the video is free though, so if you lead testing in languages other than English, this could be very helpful.
Advantages:
A good tool for non-English language remote unmoderated usability testing
Demographic and language filtering tools are available
Pricing is relatively low
Post-test questionnaire functionality is available
Free white label playback of tests
Disadvantages:
Unbiased questions can appear or testers may try to quickly finish the test
No advanced video editing capabilities
Videos cannot be downloaded and edited
Mobile testing is not available
Tester Audio and Video Recordings
Recordings can be edited
Mobile test-ing capable
Remote un-moderated testing
Testers avail-able
Price
√
×
×
√
√
$39 per test with bulk dis-count pricing available
OpenHallway is a tool for PC- or Mac-based research. Although testers come from the practitioner, the tool doesn’t have the panel to manage users. The links to the testing sessions are generated by OpenHallway when everything is set up. It requires no code to use, which makes it a great tool for testing applications, websites, and even competitor websites. The voice of a tester, their screen interaction and their face are captured providing the user has a camera. The amount of available video storage is based on the plan that you use: Basic, Plus or Premium. And although an unlimited number of test sessions is available, video storage capacity will run out at some point.
Advantages:
Simple to use
Easy to set up
Handy tool for recording unmoderated remote usability tests
Disadvantages:
PC or Mac compatible, but does not support mobile
Java must be present on the tester computer for the tool to work
Videos are MP4 only, and only paid plans allow downloading and editing of the video
UsabilityTools is basically a number of usability, feedback and click tracking tools that use a single client interface. This makes creating tests that require multiple tools much easier. There are two versions, a UX Suite and Conversion Suite. It contains click testing, web testing, survey, card sorting, and a Persona Creator. You can set web tests that provide scenarios for website visitors, where the visitors are given two options: success or abandonment. A user clicks when they understand whether they can accomplish the task.
Advantages:
To fully understand the user experience you’ll need some additional tools
The process of creating and running multiple tool tests is easier because of the single client interface
The aforementioned set of tools can be more economical than using separate usability tools
Disadvantages:
Does not provide audio and or video recordings of tester sessions
Mobile interactions using audio and video recordings are also not available
TrymMyUI.com is SaaS-based remote unmoderated tool. Testers create their own tasks. Videos are provided which include screen interactions and testers’ voices. Written answers to post-test questions are also included.
Advantages:
Slightly cheaper than many other unmoderated tools
Provides video and audio recording of testers as they try to complete tasks
Disadvantages:
Necessary to create unbiased questions and carefully analyze results
No video editing feature
No mobile testing
Tester Audio and Video Recordings
Recordings can be edited
Mobile test-ing capable
Remote un-moderated testing
Testers avail-able
Price
√
×
×
√
×
$35 per test with bulk dis-count pricing available