Measuring Inclusive Experiences

The USERindex is a benchmarking score created in 2014. To date, the USERindex has been used in over 700 studies across 50 countries and has been adopted by companies worldwide as their default benchmarking system.

The USER in USERindex represents 4 dimensions: Usefulness, Satisfaction, Ease of use and Reliability. Capturing these aspects of the experience has become crucial as digital technology is now more prevalent in consumers’ daily lives than ever.

One USERindex for All

The USERindex was originally developed with digital products such as websites and mobile apps in mind. Soon after, it was successfully used to measure both the digital and hardware experiences of IoT devices.

In the past few years, we have piloted the USERindex in inclusive design UX studies with participants with disabilities and people who use assistive technologies. Our conclusion is that the USERindex is suitable for capturing and measuring the experience of participants in inclusive design UX studies.

This is encouraging, since a significant portion of the world’s population has a disability of sorts; about 15 percent of the world’s population and 30 to 50 percent of older adults in the US. And let’s not forget people all over the world who break their arm or lose their glasses, resulting in a temporary disability.

Best Practices

Here are a few best practices for interpreting scores in inclusive design UX studies.

Track Disability and Assistive Tech

Benchmarks such as the USERindex rely on self-reported feedback as the user rates the statements. A person’s frame of reference can significantly influence their ratings. For example, screen reader users are accustomed to challenging user experiences. As a result, what may seem a poor experience to some is a “good” experience for screen reader users. This can skew the ratings and it is thus a good idea to track both the disability and the assistive technology used.

Define User Personas Based on Assistive Tech

The different types of disabilities (e.g. vision, cognitive, motor impairment) are broad categories that cover a number of conditions and severity of these conditions. This variety makes it challenging to define user personas based on condition. We have found that defining user personas primarily based on assistive technology works well to collect consistent data points since the experience heavily depends on the type of assistive tech used.

Learn Through Detailed Analysis

The USERindex is represented as a single score. Yet, reviewing the scores for each statement and dimension across participants can be very insightful and help identify specific areas where the digital product may be particularly lacking.

Our world needs to be inclusive; access to a benchmarking score to measure the digital experience for everyone is simply a must have.

Measuring Delightfulness

What is “delightful” UX?

Delightful UX is a user experience that literally puts a smile on your face (or an inner smile, if you’re not the type to show emotion easily ) simply because the interaction brings you joy. Feelings matter, and creating a positive emotional reaction influences the user’s behaviour and opinion towards the digital product they are using.

However, if the product itself falls short of the user’s needs and expectations, its delightfulness will not make up for it. This is expressed very nicely in Aaron Walter’s hierarchy of user needs, which is based on Maslow’s hierarchy of needs.

Walter remaps Maslow’s hierarchy of needs to the needs of users. He defines the most basic need a user has as functionality, followed by reliability, usability and then pleasure. Taking an app to purchase train tickets as an example: a user’s first need would be that the app does just that: allow the user to buy a train ticket (= functionality). The next need the user would have is that the app is reliable i.e. the data it provides is accurate, the payment is secure etc. and once these needs are met, the user will look for usability. Was it onerous to buy that ticket, or did the app make it seem effortless because it was so intuitive? Each of these needs is a potential exit for the user and a reorientation versus another app that might satisfy their needs better. And only once all of these needs are met, the user will appreciate if the app is pleasurable.

“Designers creating interfaces that are just usable is like a chef creating food that’s just edible.” – Aaron Walter

How do you measure delightfulness?

Nielsen differentiates between surface and deep delight; surface delight being local and contextual, while deep delight is more holistic and the one we should aim to achieve.

You can measure the surface delight simply by asking users to rate how delighted they were on a 5-point (or 7-point) labelled (or non-labelled) scale. Similarly, you could use a Likert scale and ask users to rate how strongly they felt delighted. Any of these would correctly measure the level of delight the users felt. But it wouldn’t inform you how the other aspects of the experience were perceived.

The USERindex measures a users’ subjective user experience by evaluating all 4 dimensions of their experience: Usefulness, Satisfaction, Ease of use and Reliability. These 4 dimensions map directly to Walter’s pyramid. The USERindex can thus be used to measure a product’s delightfulness as well as evaluate how the other 3 dimensions of the experience performed and identify on which level of the pyramid the product fails to live up to expectation. Try it out and do please share your thoughts!

—

Aaron Walter – Designing for emotion. http://www.abookapart.com/products/designing-for-emotion

Nielsen Norman Group – A Theory of User Delight: Why Usability Is the Foundation for Delightful Experiences https://www.nngroup.com/articles/theory-user-delight/