Usability testing is an essential tool in any product life cycle, especially at startups, where you live and die by your product. It is the most effective technique to get real user data on how they are using and assessing your product–in real life. If the product stake holders are not already bought into user testing, you need to convince them, now. Usability tests are cheap, and have the potential to save development time, money, and headaches down the road.

1. Determine Your Personas

If you’re doing a startup, you should already have a good sense of who uses your site. Is it the normal internet user? Is it business people who are internet savvy? Is it your grandma? Make sure you pin down all the potential user personas before going any further.

2. Recruit Testees from Craigslist

Depending on your location, Craigslist has a rich ecosystem of people willing to give their time for usability tests. Here is a sample text I usually post:

We are a company creating an online consumer product that needs people for usability testing. The test will be no longer than 1 hour, and we'll pay you $35 for your time. Easy money! Requirements: List your demographic requirements here We are located at LOCATION, and we'll be conducting the tests on DATES. If you qualify, and are interested, please reply with the following information: - Name - Date and hours of availability - How much experience you have with internet sites, specifically searching the internet - How many hours a day you spend on the internet - The industry in which you work

Be as specific as possible with your requirements. You can always widen them later if you don’t get enough responses. In the San Francisco Bay Area, I received about 20 responses in the first day, most of them eager to help out and that fit our personas. Avoid putting any information about your company or website, as you want the testees to see your site for the first time during your test.

If Craigslist isn’t popular in your area, I suggest just grabbing some people from your local coffee shop. You’ll be surprised by people’s enthusiasm when they know they can help improve a product.

For your first battery of usability tests, take two days and block out 4 hours on each day, and try to set up times with 8 people. Some of them will invariably cancel or flake out, but, that’s okay, since you really only need about 5 people to have an effective test.

3. Prepare Your List of Tasks

I highly suggest picking up Don’t Make Me Think, which is a great introduction to web usability and usability testing. It’s short, and chock full of practical information.

Now, write down all the user tasks that are crucial to your site. This may be the signup flow, or the upload pictures flow, or the browse profiles flow. The point is, you should be testing flows that are most important to your business objectives.

Remember that testees will take much longer than expected to complete the tasks. It’s easy to lose sight of this, but as the founder or member of the product team, you’re an expert at using your product. Testees have never seen your site before, and they’ll have to figure it out all on the fly. Try to limit your first set of tests to 4 or 5 tasks.

Your list of tasks should be derived directly from the user flows on your site. It’s best for each task to have a very specific objective for the user with a plausible story behind it. For example, a task on YouTube might be: “You’re sitting at home relaxing with a friend, and you’re looking for videos of lolcats. You land on the YouTube homepage. Now, I want you to go and find some funny cat videos.” The more realistic the imagery you put into the testee’s mind, the more comfortable and natural they will be at completing the task.

Try as best you can to make these tasks funnel the user to do one type of action. Of course, you may not be able to avoid multiple paths–just be sure to take into account all them in your task list. In example above, the user might use the search functionality on YouTube, or, they may simply browse by category.

Make a spreadsheet with each tab being a separate task, and list each step of each task on a separate row. This makes taking notes easier during the test.

4. Download Silverback

Seriously, Silverback is the easiest and slickest video and screen capturing tool for usability testing. It captures the screen while also capturing the video from iSight and puts this all into the same video on export. You’ll be able to review the videos without missing a single thing the testee did during each task.

At $50, it’s cheap.

If you’re worried that Silverback is Mac only, and your application must be tested in Windows, you should be able to run Parallels.

5. Conduct the Test

You will need two people: one to take notes and one to direct the user to do the tasks. Here are some simple guidelines:

  • Make the testee as comfortable as possible. The less comfortable they are, the less likely the user will respond realistically to your site.
  • Many testees are nervous. Tell the testee that they are not being tested, rather, it is the product that is being tested.
  • Ask the testee to speak out loud during the session, explaining what they are thinking at all times.
  • When the testee first sees the site, ask them about what they think the site is. This is the best moment to get their genuine first impressions.
  • Start off with an open ended task that allows the testee to explore the site on their own.
  • Tell the testee to do each step of the task, but never tell the them how to do each step. This is by far the hardest part of usability testing. Remember: you need to let the testee struggle, or else the pain points in your product won't reveal themselves. If, after about a minute or two, they still have no idea, ask them, "Is there anything in particular you are looking for?". If it's a total no-go, mark the step as FAIL, and simply point to whatever the solution is and ask, "What about this option/checkbox/button/etc?" Note their response.
  • When taking notes, no detail is too small. Observe everything the testee does -- did they click on the image that was not hyperlinked? Did they frown when their upload aborted without a message? Were the confused by the order of things in a dropdown menu? It's the little nuances that will tell you the problem areas in your application.

6. Post Mortem

There’s no point in usability testing if you don’t analyze the results and make action items. Gather the team and go over the notes and videos. Everyone will probably be surprised by the number of small and big problems overlooked when designing the product.

Typically, the solutions to the problems seen in usability testing are obvious and easy to implement. Be decisive and move forward quickly.

I suggest doing usability tests periodically (once every few months) as you actively develop your product. Typically, there are no shortage of features and flows to test on your site, and it’s best to catch usability problems early.

I hope this post will encourage you to do your first usability test. A lot of people tend to dismiss conducting usability tests, saying, “I know exactly what the user wants in the design.” But, the fact is, in most cases, you’re not the target demographic, and your intuition about the design is vastly different than the average joe. It’s much better to have empirical evidence that users are responding well to your site, rather than falling back on your own gut feeling.

UPDATED 12/14/08: Thanks goes to Ron from Design Perspectives for his awesome feedback on this post.