User behaviour and interaction tracking with IOGraphica

During Sara Summers‘ talk Get Real! Sketch, Prototype, and Capture Great Ideas with Expression Blend and SketchFlow at MIX11 the question of recording user interaction and mouse behaviour on a prototype or a real application or website. In the talk Sara touched on how you can use SketchFlow to record user interaction with active elements such as buttons, but the question concerned live tracking of all mouse movements — i.e. where the user moves the mouse to, where she leaves it to rest and generally how she navigates the site. That made me realize I never got around to writing an article on how I do this with the free art tool IOGraphica so I guess it’s time I do just that.

What does your mouse tell you about your tools?

When you build applications or websites you (should) spend a lot of time thinking about how to make the user interface as intuitive as possible. That means placing navigation and key elements in the most obvious places and make them so that the user finds them easy to interact with. But there’s an additional layer to this process, and it is one that is usually ignored unless you are working for a large company: Actual live user behaviour.

Let me give you an example: Say you design a website with the main menu on the top left and a call to action button on the middle right. This configuration would likely lead to the user first moving her mouse up to the top left and then down to the middle right. Because most web users start their web surfing by placing the cursor at the front of the web address window it means you have created a situation in which they make a left reclining V movement to interact with the site. This is inefficient, but you are not likely to ever know because all you see are the actual actions (click) on the website. To discover this created behaviour you need to actually watch the mouse move in real time or record it.

Recording what matters

If you are working for a huge company with a UX department you likely have the fancy, advanced and expensive tools to do proper real-time interaction and behaviour tracking. But chances are you’re not and those tools are both well beyond your reach and also well outside of the scope of your work process. Typically this is solved by either doing live interaction testing where you literally sit and watch someone else work with the site Big Brother over-the-shoulder style or on a separate monitor or you record the session with a tool like Camtasia for review later. Both of these processes gives you some idea of what’s going on, but they are neither time effective nor conclusive. To be frank watching people surf the web and interacting with applications is mind numbingly boring and it’s exceptionally hard to glean anything from it.

More importantly though, the real-time watching procedure doesn’t actually provide what you need which is a map, taken over time, showcasing overall trends in behaviour. In other words, it’s really just a solid waste of time. This is where IOGraphica comes in.

Using an art tool to make valuable data

My experimentation with IOGraphica, a tool created by Anatoly Zenkov and Andrey Shipilov, actually started out of pure curiosity. I had seen an article on the tool hailing it as a way to create cool art from your day-to-day work routine. IOGraphica is a simple Java app that runs on your computer and records all mouse movements, pauses and clicks for however long you want it. You simply download it, boot it up, click the play button and work normally. It has some very basic settings to turn mouse pause recording on and off and to add the desktop as a background but beyond that it’s as simple as can be. If you take a break you can hit the pause button and when you’re done you simply stop it and save the image.

What you get from IOGraphica is a visually stunning image of your mouse behaviour over the recording time period. With the mouse pauses turned on it looks a bit like a highly organized random doodle with ink spots all over the place.

The description of the app as a tool for making art is accurate — the output has an artistic flare and you could definitely print and frame it, but there is a much more important use for it: IOGraphica gives you a clear and often quite surprising picture of how people actually behave on a site or application. And because it is recorded over an extended period of time you uncover strong trends and get a very clear idea of whether your tools, layouts and designs are organized in a way that makes sense to your users.

The image above shows my interaction with PhotoShop over a 3 hour period while I was working on a redesign of a site. I recorded it to test IOGraphica, but once I saw what came out of it I realized I’d gotten way more than I expected. Looking at the map I saw that my workspace layout was quite inefficient. The largest clustering was in the middle where the actual design work took place, but there was also a lot of traffic going to the top left and bottom right corners. This is where I had my tools and layers respectively. On closer inspection I also found that I often traversed the entire diagonal of the screen from tools to layers or vice versa, the longest distance possible without making any curves.

Two things became obvious: First of all I am far less good at using keyboard shortcuts than I thought. Secondly the default position of the tools is inefficient. For quicker work and less mouse movements I should place the tools and layers directly next to one another. Now imagine what the same process will say about your apps and sites.

IOGraphica user interaction recording scenarios

Incorporating the IOGraphica tool into your user behaviour tracking (and incorporating user behaviour tracking in your overall design process) is a simple process, and it’s one you can do either locally or remotely. It requires the Java Runtime Environment (JRE) which is about a 5 minute download and install. IOGraphica itself is a standalone application consisting of one executable file and requires no install. There are versions for Windows, Mac and Linux.

Once you have the application, simply start it, choose whether you want to record pauses or not and click the Play button. If the user needs a break or has to leave the application you can pause the recording temporarily. When the testing is over simply stop the recording and save the file. You can then choose to save it just as a behaviour map or as a behaviour map with the desktop as a background. Or you can do both. Since the application doesn’t record the desktop itself you simply place the application or website you are testing in the front view and save it to get the map superimposed.

The key to making this work is to ensure that the user only uses the application or website being tested and that the recording is done over an extended period of time. Longer time means a more statistically relevant result.

Because of the nature of IOGraphica you don’t actually have to be present when during the testing, and you don’t have to do it on your own computer. You can simply send IOGraphica or its link to your tester along with the application or site to be tested and some basic instructions and have them do the actual test on their own time. Because it runs in the background and is dead simple you’ll end up with valuable data from multiple sources and you can do some pretty advanced testing without ever leaving your office.

Cool, eh? I think so.