Deconstructing machine learning for product design
How data and design can collaborate in user-centric products
The intersection of AI and design is a unique one: on one hand are the data matrices and Ex Machina robots being programmed at 3am, and on the other are the color theories and UI mockups being concocted in Figma.
In the past, these two fields have been thought of as separate. Yet if the last decade of ML-powered products has taught us anything, it’s that machine learning is becoming more and more multidisciplinary. In his article about experience design in the machine learning era, Fabien Girardin argues that we are pioneering a new type of design, one that:
1. Creates new types of user experiences.
2. Redefines the relation between humans and machines.
3. Requests a tight partnership between designers and data scientists.
We’re heading towards a future where design and data reach a convergence, one that will guide how humans interact with AI as we know it.
This article is divided into 3 parts — a product-centric overview of machine learning, a guide to how product designers can identify ML problems, and some principles for designing ML-powered products. If you already have a comfortable understanding of ML, feel free to skip ahead to Part II :)
Part I: A product-focused rundown of machine learning
First, let’s recap what machine learning really is — it’s good to get the first principles straight before diving into how it can be incorporated into products. In a nutshell, machine learning (ML) is a branch of AI which, as Arthur Samuel defines, “gives computers the ability to learn without being explicitly programmed.”
Machine learning gives computers the ability to learn without being explicitly programmed.
— Arthur Samuel (coined the term “machine learning” in 1959)
This means that, unlike standard programs where every step must be explicitly coded out (“if this, then that…”), ML algorithms can “learn” the steps themselves.
So how do they learn? With data. A huge part of machine learning is the training data that’s fed into these ML models to, well, train them. This is followed up by testing data that’s given to check if the model actually learned something. Through this process, ML models can gain the ability to learn, make decisions, and improve on their own.
There are 3 main types of machine learning being used in products today — supervised, unsupervised, and reinforcement learning (we’ll save others, like semi-supervised learning, for another day). Let’s dive a little deeper to see what each one is about…
Supervised Learning: Predictions
Supervised learning essentially involves using labeled data that we’ve seen to predict the labels of unlabeled data that we haven’t seen. Think of it as predicting the labels we don’t know, or trying to figure out an unknown value for some piece of data.
Two main types of supervised learning are classification and regression.
In classification, the unknown value we’re trying to predict is a category (i.e. a discrete output). Say you’re trying to get a computer to label images: is it a dog, or not a dog? Is it a human, or a street lamp? Is it a benign tumor, or is it malignant?
In regression, on the other hand, the unknown value we’re trying to predict is a number (i.e. a continuous output). What will the price of a house be in 10 years? What is the probability of an event occurring? What will the height of a tree be given its characteristics now?
Supervised learning in products:
- Recommendation: e.g. social platforms like TikTok recommending content you may like on a “For You” page
- Classification: e.g. email providers like Gmail classifying mail as spam or not spam
- Ranking: e.g. search engines like Google or Yahoo ranking search results
Unsupervised Learning: Patterns
Unsupervised learning involves trying to discover the “structure” in a collection of data; in other words, we’re trying to find what the underlying patterns are. It’s particularly helpful when the data we have is unlabelled or when there’s not exactly a “correct” output that we expect.
One type of unsupervised learning is clustering. This involves grouping data points that are potentially similar, creating “clusters” of data that the computer thinks are closely related. You often see clustering being used in customer segmentation, fraud detection, and document classification.
Unsupervised learning in products:
- Clustering: e.g. e-commerce sites like Amazon with “Other customers also bought…” lists
- Anomaly detection: e.g. platforms like Twitter or Youtube uncovering trends for their “Trending” lists
Reinforcement Learning: Decisions
Reinforcement learning involves learning to choose an action based on trial and error. There is no existing data set; instead, an agent needs to collect its own data in an environment and makes decisions based on the state of the environment and the “rewards” or “punishments” from previous choices.
Think of it as similar to how humans learn: we have some actions to pick from, but we don’t know for certain what will happen with each action, so we pick based on our limited past experience. We receive a positive reward for picking good actions and negative rewards for picking bad ones, which informs how we’ll make our next decision.
While reinforcement learning is less widely seen in mainstream products, it has made waves in the real world, from being behind computers playing Go (AlphaGo Zero) to training self-driving cars in tasks like trajectory optimization and motion planning.
Part II: What can design do for machine learning?
Product design: The start of the ML development process
In order to design user experiences for ML products, you have to first understand the foundation of the problem needing to be solved. Notice how the first step on the diagram above is product design. You need to know the problem before focusing on the data, and product design is an integral part of this: the interaction between the user and the product is where the problem should be solved, so the design of that interaction is key.
The technical team members who are working on the ML product will be focusing on tasks like data analysis, feature engineering, avoiding overfitting, and algorithm optimization. However, those on the product team need to focus on organizing the needs and vision of a product. As Neal Lathia wrote in an article, this includes asking questions like: Does the ML fit the product goal? How does the product behave around the ML? What interactions, actions, and control do users have? And how could the product fail catastrophically?
Design makes AI smarter
If it isn’t obvious yet, machine learning models love data — they feast on lots of it to learn. Thus, our predictions can only be as good as the data we have. Arguably the most critical step of the machine learning pipeline is the collection of training data that will go on to influence the ML model — without good training data, the model can’t make good predictions. This is where design comes in.
Designers help artificial intelligence gather better data to understand user inputs. Most of the time, there’s a clash between the info that an AI needs and the info that a user is willing to give. Good design helps solve this by creating interfaces that seek to accurately capture a user’s preferences. This often involves designing efficient and effective feedback loops between the AI and the user. This way, teams can better understand user intent (almost like reading between the lines of a user’s actions) and build smarter products.
Establishing trust and clarity
Design can play a crucial role in establishing trust between users and machine learning products. A well-designed user interface helps users understand how the product works and what it is capable of; as a result, users feel more comfortable engaging with the product in a way that better teaches the AI (looping back to how good design makes AI smarter). Product designers need to focus on being user-centric and adding clarity to ML-powered interfaces, which will generate better user engagement for collecting the data that goes on to train ML models.
Google’s People + AI Research (PAIR), a multidisciplinary team that explores the human side of AI, explains the following:
Because AI-driven systems are based on probability and uncertainty, the right level of explanation is key to helping users understand how the system works. Once users have clear mental models of the system’s capabilities and limits, they can understand how and when to trust it to help accomplish their goals.
In other words, thoughtful UX design can help users feel more comfortable and confident in their interactions with ML products; this will be crucial if we intend to develop thoughtful, privacy-preserving machine learning in the future.
Part III: How to design ML-powered products
What is Machine Learning UX Design (ML UX)?
Intuitively, ML UX is the intersection between AI, user experience design, and human-computer interaction. It combines an understanding of artificial intelligence with the ability to design user experiences that are intuitive, efficient, and effective. ML UX designers need to design with data in a way that humanizes AI to work with humans — this includes designing user interfaces that are easy to understand and use, as well as designing interactions and workflows that make it easy for users to input data, receive output, and understand the results of machine learning models.
In the Google UX community, there’s an effort called “human-centered machine learning” (HCML), which emphasizes the importance of understanding how ML works, how algorithms can be used to create meaningful user experiences, and how to design for trust and transparency. Let’s consider some basic principles for designing ML-powered products that are informed by HCML and ML UX (these can be good reference points for product teams when productizing machine learning).
Principles for designing ML-powered products
Make sure you’re solving a specific (and real) human problem.
In designing AI systems today, the focus should be on addressing a specific issue or challenge, not getting caught up in the hype and buzz surrounding AI. Ask yourself if ML will uniquely address the problem — there are lots of problems that can be solved just as well (if not more easily) without machine learning. Human-centered machine learning is not about forcing ML into products to make them feel “smart” or “personal” — it’s about creating tangible value through solutions to human problems. The design of the AI system should clearly reflect this goal of improving the value it offers to users.
Ask the right questions.
Machine learning won’t figure out what problems to solve. Instead, itdesigners need to define the context and purpose behind a question in order to effectively apply machine learning to improve the user experience. This means considering the complexity and nuance of the human experience. For example, in platforms that use machine learning to recommend content to users (e.g. Spotify, Netflix, Instagram), product designers will need to ask questions like: What does it mean for a user to like a particular piece of content? How does a user’s context influence their content consumption decisions? What information does a user need to make a choice about content?
Good ML design shouldn’t get in the way.
Effective AI design should not interfere with the user experience. Product designers should focus on making it function subtly and discreetly, providing helpful signals or assistance without disrupting the user’s current activity. A well-designed AI system should be able to operate smoothly in the background, enhancing the user’s experience without being intrusive.
Don’t be constrained to historical context.
Emerging design trends show that design does not have to be limited by historical context. With the introduction of new technologies, it’s important for designers to embrace new and innovative ideas rather than sticking to what has been done in the past. This is especially important when designing “smart” products where it’s necessary to consider the potential capabilities and possibilities of the technology.
Design for future adaptation.
To improve AI systems over their lifetime, products need to have interactions that make it easy for users to provide feedback and show the benefits of that feedback. As users interact with these systems, they influence the outputs they see in the future, which in turn affects their future interactions with the system. This creates a feedback loop in which the system and users are constantly adapting and adjusting to one another. This helps differentiate good ML products from great ones — the most effective ML systems evolve in response to user interactions and feedback over time.
Conclusion
Design and ML are in a mutually-beneficial relationship: by using ML to inform product design and optimize the product development process, designers can create better products that meet the needs and preferences of their users; at the same time, good product design can help make machine learning more accessible and easier to use, potentially driving innovation and progress in the field.
Harry West, CEO of Frog, reasons in a recent article:
“Human-centered design has expanded from the design of objects (industrial design) to the design of experiences (adding interaction design, visual design, and the design of spaces) and the next step will be the design of system behavior: the design of the algorithms that determine the behavior of automated or intelligent systems. …The challenge for the designers is to tie the coding of algorithms with the experiences they enable.”
Thus, by creating products that are intuitive and easy to use, product designers can help users get the most out of their machine learning-powered products and make it more likely that they’ll continue to use them. And who knows — maybe one day we’ll have a machine learning algorithm that can design the perfect witty conclusion to this article.
Sources & further reading
References that provided guidance and inspo for this piece, as well as generally interesting resources to dive deeper
- Design of Artificial Intelligence Products (Jeffrey Chou): The class note of Design of Artificial Intelligence Products taught by John Zimmerman at Carnegie Mellon University
- ML + Design: A collection of resources for the intersection of design, user experience, machine learning, and artificial intelligence
- People + AI Guidebook (Google)
- When Design Meets Machine Learning — Lalo Martínez (Figma Global Design Conference)
- Machine Learning for Product Managers: A product-centric overview of machine learning (Neal Lathia)
- Design and Machine Learning (Matthew Ström)
- Three Principles for Designing ML-Powered Products (Spotify Design)
- Design Makes AI Smarter (Elaine Lee)
- Experience Design in the Machine Learning Era (Fabien Girardin)
- Why is Design Important for Artificial Intelligence? (Goodwork Labs)
- Human-Centered Machine Learning (Google UX Team)
- The Role of Design in Machine Learning (Owen Shoppe)
- The UX of AI (Josh Lovejoy, Google Design)
- What AI Can Learn from Design (Molly Wright Steenson, TEDx Carnegie Mellon University)
