What is Heuristic Evaluation (HE) or Heuristic Research?
What is the importance of exploratory evaluation in product design?
Is exploratory evaluation one of the methods of user research (UX Research)?
What are the principles of exploratory evaluation?
How should he carry out an exploratory assessment?
User research is the most fundamental part of the interaction design and product design process.
The product design team cannot design successful digital products without having sufficient knowledge of the people who are going to use the product.
User research is done with different goals, methods, and methods. User research may be used at any stage of the design process to evaluate decisions and solutions.
Exploratory evaluation is a type of user research that is conducted with a specific purpose and provides decisive findings to the design team.
In this comprehensive guide, we answer all your questions about exploratory user research and its importance in the UI/UX design process.
Table of Contents
What is Heuristic Evaluation (HE)?
To have a proper understanding of exploratory evaluation, it is necessary to first look at the important role of user research in the UI/UX design process.
The design of digital products consists of different parts. In other words, experts from different disciplines collaborate to design a digital product, and each expert or team of experts in the field designs and builds apart.
Finally, with the implementation of all parts, the final product is available to users.
Just like a physical product whose different parts are designed and manufactured separately and then put together to form the final product.
The product design process usually starts with user research. Because it is necessary to identify the problems and needs of product users.
If the product cannot meet the needs of the users properly and they do not have a satisfactory experience interacting with the product, the product fails.
It is on this basis that at each stage of the design process, the designed parts and the product design are tested. It means that the team still researches to make sure that the end user of the product is satisfied with the design or if there is still a problem.
And this is why there are different types of user research, at different times (phases) in the design process and in different ways.
In some user research methods (qualitative research), real users are interviewed or observed while interacting with the product to identify their problems.
In others, users are not involved and experts are the ones who test the product to evaluate its performance and identify problems. Exploratory evaluation is a type of user research.
Exploratory evaluation is a type of user research in which experts, not users, evaluate the product based on guiding principles.
What does exploratory evaluation have to do with interaction design?
In the exploratory evaluation of interaction design and the design of user interfaces, the product is evaluated to identify any problems that interfere with user interaction with the product in user interfaces or other aspects of the design.
Interaction Design and User Interfaces/UI Design are key parts of user and product experience design.
In interaction design, the action and reaction that should happen between humans and the products (machines) are designed.
In other words, in interaction design, the points and dimensions that create communication and interaction between humans and systems or products (websites or applications) are identified.
Interaction design is not the same as user experience and user interface design and is an independent field. Human-machine interaction has 5 dimensions (the following 5 dimensions are taken from the comprehensive article on interaction design and its differences with UI/UX):
- Words: Human-machine interaction cannot happen without words. Words and texts, especially user experience texts such as error messages, are what give the user the information they need to interact with the product. Designing words is the specialty of UX Writer.
- Visual elements and UI: It is not just words and texts that guide the user and make him understand something. UI elements (menus, icons, etc.) and images are also important points of user interaction and communication with the product. Designing elements or user interfaces is the specialty of UI designers.
- Space and device: physical devices and devices (smartphone, laptop, computer, mouse, etc.) that primarily enable human-machine interaction. In addition to the device and its features, the actual place or space (work environment, home, while exercising, walking, etc.) where the interaction takes place is also effective in the quality of the interaction.
- Time: How much time does the user spend interacting with the product? How does time affect interaction? In addition, some elements (such as sound and animation) become relevant at a certain time of interaction with the user (for example, when the user enters a page, a pop-up with a certain sound is opened for him).
- Action and reaction: How does the product (the sum of the previous 4 dimensions) work as a whole?
What is the meaning of Heuristic and the exact definition of heuristic evaluation?
We said that exploratory evaluation is a type of user research that examines the efficiency and usability of interaction design and user interfaces.
The word “heuristic” is an adjective meaning exploratory, perceptive, and innovative. But in this combination, it has another meaning.
In the phrase Heuristic Evaluation, this word is synonymous with guiding principles or guidelines, rules or methods for evaluation.
In this type of user research, experts examine product design and user interfaces based on guidelines and predetermined rules and answer specific questions.
The reference website in the field of product design and user experience design (nngroup.com) has defined exploratory evaluation as follows:
A heuristic evaluation is a method for identifying design problems in a user interface. Evaluators judge the design against a set of guidelines (called heuristics) that make systems easy to use.
The guideline in exploratory evaluation is defined by each product design team based on the features and type of product being designed and the dimensions or dimensions in interaction design that should be evaluated.
Of course, a set of guidelines and principles have been proposed and published by experts and authorities in the field of human-machine interaction design, which any team can use for exploratory evaluation of its product.
The first and most famous set was proposed by Jakob Nielsen, the originator of exploratory evaluation in product design.
What are the most famous guiding principles for exploratory assessment?
Jacob Nielsen is a university professor in the field of human-computer interaction and is one of the most well-known figures in this field.
In 1990, Nielsen, together with Rolf Molich, who is also a well-known expert in the field of human-computer interaction design, used a new research tool (exploratory evaluation) using ten general principles and guidelines to measure human-computer interaction design.
In 1994, Nielsen fully explained the reasons and how heuristic evaluation was formed in his very famous article entitled The Theory Behind Heuristic Evaluations.
After that, other experts also published research and influential articles in this field. The point to be noted is that the exploratory evaluation measures the simplicity, fluency, and success of the interaction between the human and the system.
This is why experts from the fields of psychology and cognitive engineering have also addressed this issue.
UI/UX design is done to establish a successful interaction between humans and a non-physical product (website or application that is a system). So, product designers also used this tool to measure the design and usability of the user interface of digital products.
The most popular guiding principles for exploratory evaluation that product design teams can get ideas from or use for evaluation are as follows:
- Jacob Nielsen’s principles for evaluating user interface design
- Ben Schneiderman’s 8 golden rules for evaluating user interface design
- 10 Principles of Cognitive Engineering by Jill Gerhart Powells
- 18 ergonomic criteria by Christian Bastin and Dominique Scapin for evaluating human-computer user interfaces
- Fundamentals of Interaction Design by Bruce Tagnazzini
Jacob Nielsen’s ten principles for design evaluation and product user interface design
The ten principles of Nielsen and Molich are general criteria for evaluating the design and usability of a product’s user interface.
Evaluators and experts who are selected for exploratory evaluation, based on these evaluation principles, identify problems and propose solutions to the product.
Based on these principles, users interacting with a system (product) designed for successful and effective interaction with humans:
- They are quickly and appropriately aware of the events that happen during the interaction and their position and status in each interaction with the product and receive the appropriate feedback in time.
- They receive information in a logical way that is consistent with the order of things in the real world and is familiar to them. And the information is presented to them in their own language and in simple terms.
- Users are in complete control and can easily fix their errors or mistakes or cancel the interaction and process. In other words, the way out of each stage is clearly shown to them.
- They don’t get confused because all the texts, visual elements, and overall design are the same and have the same meaning everywhere.
- Prevents user errors or gives them timely warning (for example, displaying a “Are you sure you want to do this?” message before deleting an email or closing a page).
- Information, instructions, and interaction paths are recognizable to them, and they don’t have to remember anything or do trial and error.
- The product is designed to be flexible, meaning that as users gain experience interacting with the product, they have faster ways to achieve their goals that are hidden from novice users.
- Because the product design and user interfaces are beautiful and minimal and do not confuse users with additional and unnecessary elements. So, they are given only the information necessary to complete an action or achieve a goal at each stage, and nothing extra.
- Error messages are simply written and contain no technical or code information.
- They don’t need a guidance document or further explanation to interact achieve a specific goal or do something. If there is a guidance document for solving problems, it is concise, useful, accessible, and searchable.
When should an exploratory assessment be performed?
Exploratory evaluations are very useful and fruitful for identifying obvious problems in the user interface of a product.
Exploratory evaluation is used to measure the user interface of any product that users interact with.
User interface and interaction design prototypes, physical products, games, virtual reality, or audio interfaces can be measured with exploratory evaluation.
Contrary to what it seems, this evaluation is not only done in the final stage of design and when the product is finished and complete.
This method of user research is very helpful, especially in the early stages of design and helps to save costs and time for the product design team.
By doing this assessment early in the design process, the UI/UX design team will understand potential problems and obstacles without having to spend money on testing with real users.
However, exploratory evaluations are not a substitute for user research. User experience design Every product has its features and users. So, for UX design and successful product design, the product and its various aspects must be tested with real users.
Exploratory evaluation complements the research work of the design team. For example, if an exploratory evaluation is conducted before usability testing, the team uses the findings to identify design elements that should be given more attention in testing.
In addition, conducting exploratory evaluations is a good practice and training resource for novice user experience designers.
If you are a product designer or UX designer, evaluate different types of products. It doesn’t matter if you designed it yourself or not. It is important to try to identify problems with interaction and user interface design.
You can use one of the set of principles that we introduced a little higher to evaluate each product.
What are the advantages and disadvantages of exploratory evaluation?
Exploratory evaluation has important advantages:
It helps the design team to identify design problems in the early stages of the work by spending a small amount of money and without the presence of real users.
It complements user research and its findings can help the team to optimize other product evaluation methods.
Exploratory evaluation also has important disadvantages. Considering the disadvantages of the exploratory evaluation method, the product design team should choose this method to evaluate the product design.
The most important disadvantages of exploratory evaluation are:
- The results of exploratory evaluation depend on the knowledge and expertise of the evaluators. Training evaluators or hiring evaluators from outside the team and company may increase the time and costs required to conduct the evaluation.
- Exploratory evaluation is based on assuming a definition of good usability. But those assumptions and criteria cannot take the place of feedback, experience, and real feelings of users. As it is clear from the name of this research method, it is only a set of guidelines and nothing more.
- Errors may occur in exploratory evaluation. In their paper “Usability Testing Versus Exploratory Evaluation,” Robert Bailey, Robert Allen, and P. Railo found that 43% of problems identified in exploratory evaluations were not problems.
In addition, the evaluators identified only 21% of the actual usability problems, while the percentage of actual problems identified in the usability test was higher.
Therefore, it should be noted that exploratory evaluation is useful and helpful, but it is not a substitute for usability testing.
What are the steps of exploratory assessment?
The product design team must follow certain steps to perform exploratory evaluation. Design teams for exploratory evaluation usually go through the following steps:
1. Defining the scope and purpose of exploratory evaluation
The first step is to determine the scope of the evaluation and the purpose of the evaluation.
Is a whole product evaluated or only its mobile version or the design of a specific part or process in the product?
At this particular stage of the design process, the team must determine what it will measure and why using the exploratory evaluation tool.
At this stage, it is also determined how the exploratory assessment will be conducted (online and using tools such as Miro or with paper and pen and in the form of a booklet).
2. Comprehensive definition of product users
Evaluators must have a clear picture of the product’s users, their needs, and goals. Therefore, the team must have prepared and prepared audience personas.
3. Hiring evaluators or team training
Exploratory evaluation is better done by hiring 3 to 5 interaction design or usability experts. However the design team may not be able to afford to hire such specialists.
If there is a budget limit, team members can act as evaluators. In this case, the team should be trained about the exploratory evaluation and the set of guiding principles.
4. Selection of guiding principles
Usually, 5 to 10 principles are enough for evaluation. Of course, the exact number of guiding principles depends on the type of product or system and the dimension or aspect to be evaluated.
The product may be complex and detailed and require more principles. It is better to base Nielsen’s ten principles and add or subtract from them.
5. Justification of the evaluators and the first round of evaluation to get familiar with the product
Now everything is ready for exploratory evaluation. Before starting, evaluators need to be briefed so that they know exactly what to evaluate. Then it’s time for the first round of evaluation, which is for general familiarization with the product.
In the first round, evaluators only work with the product or system to get a general impression of it.
6. The second round of evaluation
The evaluators carefully check the design and all its details based on the guidelines and record any problems they encounter. Evaluators may also be asked to make suggestions for fixing any problems.
7. Summary meeting and announcement of findings and suggestions
After the second round is over, product managers and senior designers meet with evaluators to summarize and present their findings for design analysis, problems, and suggestions.