Cris Cristina // November 02 2017

The Designer’s Guide to Machine Learning

This article is presented as a way for designers to introduce themselves to the concepts and applications of machine learning — a zero to 10 mph guide to working with developers and the broader product team to design applications with a machine learning component. It’s primarily a collection of aggregated articles with some annotation, in an effort to ease into a basic understanding of machine learning concepts. Designing with machine learning is exciting, but it raises certain questions and brings with it ethical and functional pitfalls. The good news is: good design principles translate perfectly to creating useful, usable, and desirable artificial intelligence (AI) projects, with just a little thought and preparation.


Introduction to the concept of machine learning

Let’s start by defining machine learning. Adam Geitgey, a machine learning consultant and educator, aptly states, “Machine learning is the idea that there are generic algorithms that can tell you something interesting about a set of data without you having to write any custom code specific to the problem. Instead of writing code, you feed data to the generic algorithm and it builds its own logic based on the data.” It’s really just an application of artificial intelligence algorithms that gives a computer (machine) access to large amounts of data and enables it to figure out solutions on its own (learning).

“Machine learning” as a term is quite near peak hype right now. As the label’s popularity wanes, the term “machine learning” may become less popular even as the implementation of such systems becomes more common. Additionally, a few of the terms on the upslope rely on this type of computing, so it’s likely the information here will remain relevant for some time. I tend to use “machine learning” and “artificial intelligence” nearly interchangeably in this piece. Machine learning is technically an application of artificial intelligence but for the purposes herein we can consider them as one technology.


Gartner’s 2016 Hype Cycle for Emerging Technologies. Copyright Gartner. 


Understanding machine learning basics

The best place to start to get a sense of how machine learning works is with this interactive visual guide by R2D3 collective. R2D2 walks us through the process of creating a machine learning model by comparing real estate in New York and San Francisco. They address basic concepts like the relationship between machine learning and statistics, the statistical vocabulary needed to map to machine learning algorithms, and how a model makes predictions that are helpful to humans.


R2D3’s Decision tree model for predicting home locations. 


To play around and get a sense of how this works in real time, Google has created a live demo at There you can train input — image or sound captured from your device — to effect the output…one of three cute, fuzzy animal gifs.


Google’s Teachable Machine (Google and the Google logo are registered trademarks of Google Inc., used with permission.)


Next, move on to this great seven part series from Geitgey called “Machine Learning is Fun!” A little bit of computer science background will help when reading this article, but it’s not necessary to glean a basic understanding. Geitgey gives the clearest definition of machine learning that I’ve seen, and proceeds to use simple, clear examples to show how machines “learn”.

Machine learning is the idea that there are generic algorithms that can tell you something interesting about a set of data without you having to write any custom code specific to the problem. Instead of writing code, you feed data to the generic algorithm and it builds its own logic based on the data.

Finally, to go a bit deeper, there’s a good sized O’Reilly report “Machine Learning for Designers” (free pdf download with email) that explores more of the history, considers future applications of the technology, and highlights how the field of design is both impacting and impacted by these advances. Here it helps to have a bit of electrical engineering background.


MNSIT dataset


The role of design in machine learning

Moving on to the practical side, we want to understand not only how machine learning algorithms operate, but also how the user is situated as an integral part of any machine learning system. Here are two great examples of design approaches for machine learning. One considered the user as an integral part of the system and one focused more on just the algorithm.

Algorithm-centered: Netflix treats all of its category rows in the recommendations homepage as variables in its algorithm, so things like “My List” or “Continue Watching” keep jumping position. Sometimes a particular category row can be first; sometimes it can be last; sometimes it can be in the hidden position “above” the starting position. There are a few nasty threads on Reddit about this (go figure), but they capture two essential frustrations: 1) users have no content anchor and 2) their highest priority categories keep moving, especially out of the top positions.

User-centered: Airbnb created a switch for their hosts that allowed the algorithm to automatically set prices for hosts’ units. What they found in talking with users (hosts) was that users were uncomfortable with giving up full control. The product team modified the design to add limits — minimum rent allowed and maximum rent allowed. Airbnb also added a setting that allowed hosts to set the general frequency of rentals (essentially low, medium, high but in more host-friendly language).

This user-centered example places the user as an integral part of the experience. It sets a great example for how to approach a machine learning design project.

Design challenges for machine learning

The Airbnb and Netflix examples provide a good lens to highlight top level AI-specific issues to tackle when designing for these systems.

  • Human-centered design — More so than other design exercises, it is important to consider the users when integrating AI components. It’s easier to think about the consequences of a design decision when we are essentially scripting the outcomes. With machine learning algorithms, however, comes greater lack of control. For instance, in a standard car, the driver has full control. In a driverless vehicle, the car is going to determine — at least in part — the route, speed, and other decisions, such as collision avoidance. As products add more and more automation to our lives, we must — as designers — proactively represent user concerns so users don’t lose their voice in the system.
  • Communication and collaboration — When adding another autonomous agent into the system, we have to address how the human agents and the autonomous agents communicate. When there are conflicts of interest, what determines priorities? How are decisions and intent communicated? Handoff, authority, and how the user and system understand each other’s actions becomes a design challenge in and of itself.
  • Transparency — The implementation of any machine learning project can be inherently opaque. The designers of the system — “designer” and developer alike — now have an additional layer between our anticipated audience and ourselves. It is our responsibility to overcome this opacity by representing the algorithm to the user. Showing what data the algorithm uses and demonstrating how the algorithm uses those data can go a long way to enabling trust between the users and the AI.
  • Inclusive design — We already know that what goes into designing a system comes right back at us: “garbage in, garbage out”, as the saying goes. As we construct new programs, we unintentionally build our own implicit biases right in, and that can have serious, negative side effects. For example, the Google image recognition app once labeled a black UK couple as “gorillas”. More recently, ProPublica reported that crime prediction software is biased against blacks. It’s easy to see how this can happen in retrospect, but harder to recognize it in the moment. On a project by project basis, it’s incumbent on all of us to proactively plan for these biases. On a larger level, we need to advocate for a broad range of voices contributing to the research and development of algorithms. And, we must keep an eye on the social impact of the AI systems we create. Uncontrolled input can create unintended output, as was the case with Microsoft’s Tay, a Twitter bot that started spewing racist tweets within a few hours of interacting with the Twitterverse.

Ethics of building AI

An even broader challenge than inclusive design is the ethics of building an AI system. You may have heard of Isaac Asimov’s three laws of robotics.

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.

2. A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.

3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Asimov later added a fourth law which superseded the original three.

0. A robot may not harm humanity, or, by inaction, allow humanity to come to harm.

While we all remember the actions of mutinous HAL 9000, it’s not strong AI we’re confronting today. Rather, systems simple things like chatbots are what we need to address now. It is more likely at the moment that the unintended consequences of an intelligent agent cause harm than the intended ones. The chatbot Luka was adapted to recreate a personality based on a lifetime of texts, tweets, emails, and the like. The issue? The data all came from a co-creator’s deceased partner. The creator didn’t quite think through the ethics of building the demo until after it was built. Did building a bridge to a dead person undermine the importance of connecting to the living? Since the deceased didn’t leave a digital will, how did the creator know with whom her partner would have agreed to share his information? We’re still a long way from an AI that’s able to address sophisticated ethical dilemmas. Until then, we all have to be the moral compass. We have to check those new, algorithm based dark patterns at the door. We might have some help soon, though, as there are researchers who are invested in placing AI applications in context by using machine learning to teach computers ethics.


Three things to keep in mind when designing AI

We’ve talked about the big challenges, but things get easier from a design side. All we have to do as designers is rely on design’s core strength, design thinking (or whatever you call your process,) and then take a step sideways to rethink how to address use cases when the outcomes are based on algorithms.

  1. Familiarize yourself with machine learning — Hopefully by now you’ve gotten a good, general sense of what it’s all about. If you need to start real design work, take some time to walk through the references listed above to gain a deeper understanding. If you were designing a website, it would be much easier to build a working solution when you know the constraints of Javascript. The same rules apply to AI.
  2. Do regular design stuff — Do what you would do for any project. Research the needs, goals, and desires of the users. Try to understand their behavior, culture, and the context of the environment. Look to see if there are any established design patterns. Sketch. Draw storyboards. Service design maps can be particularly useful as a precursor to helping the product team understand what algorithms to create.
  3. Rethink your role and your control — Design process should not be different on a machine learning project, but design’s role may change. As the services rely more and more on the algorithms, designers’ ability to control the outcomes diminishes. That means the designers will have to become more comfortable with designing for uncertain outcomes. Any good designer will tell you it’s impossible to build a foolproof system that controls for all possible outcomes, even without AI. With machine learning algorithms, though, the outcomes are based on probabilities. It’s important to consider how to architect and communicate a range of outcomes rather than specific ones.

This was just a taste of how to get started with machine learning design. It’s a nascent field, so there are lots of great opportunities out there. Jump in and experiment!

Cris is a product strategist, designer, researcher, and the Global UX Lead for the Digitalist Group. If you liked this article, check out Research is the Engine for Design and The Slightly Smarter Office. Comments and conversation always welcome.

Cover photo by Jason Leung

More from Cris Cristina