Embracing AI as a material for design
Now is the time for designers to learn about AI by using it
This year the news has been booming about AI. You can’t open a news website that doesn’t have an article on the front page about ChatGPT, Stable Diffusion, Midjourney or OpenAI.
Although artificial intelligence has been around since the 1950s, recent advancements have brought it to the forefront of technology. The first computers were great at performing repetitive tasks, but required us humans to communicate with them through programming languages. Over time, we developed increasingly sophisticated languages and interfaces, like the mouse and voice assistants, allowing non-programmers to use computers.
Today, a specific branch of AI called Transformers, and tools like ChatGPT, have exploded in the news due to their ability to generate content based on large knowledge bases. Many online tools have emerged that utilize generative AI, but their design seems to have taken things a step back.
So many new capabilities!
Generative AI has unlocked the door to many features that were previously unthinkable to just use in a design, since the development cost would simply be too high. With the help of generative AI however, your apps can now: help people create things, help people find things in unstructured data, answer complicated user questions, personalize experiences for each and every user and even communicate with your users on your behalf. Argodesign has a great webinar up on what generative AI can mean for businesses in the future.
Think by making.
I’ve always approached learning and understanding new capabilities by diving in and learning through hands-on experiences. At my work, we call this “Think by Making”. To get better at working with AI, I joined an internal contest to create movie posters using generative AI tools. Through this exercise, we gained valuable insights into the strengths and weaknesses of generative AI, seeing it as a material for designing with.
We used tools like ChatGPT to create movie titles, scripts and even magazine quotes. We then used tools like Midjourney and Stable Diffusion to generate images.
By working with all of the different tools available in a playful but professional contest, we learned a lot about how these tools work, what they are good at and most of all what they are bad at. While playing, we learned about generative AI as a material that we can design with.
What does this mean for designers?
First: Play responsibly.
First up, I’d like to caveat the rest of the article with a short word of warning. Currently, AI tends to give a lot of wrong answers and the quality of the output can be questionable in many cases. The legal stance on the intellectual property of AI-generated content is unclear and being publicly debated, and there are many ethical considerations on the material AI has been trained on and the general effects that generative AI can have on society.
As with any new technology, best practices for the UX of generative AI still have to be defined. Let’s continue to a couple of topics that are ripe for design:
Design for discoverability
This technology is so new that people don’t understand yet what it can do. The possibilities seem endless; summarize text, re-write this text in pirate speak, find repeating patterns in data or even create an image of a puppy on a skateboard wearing a baseball cap.
It’s imperative to help people discover these features, and give them quick access to what’s important. ChatGPT can do all these incredible things but OpenAI doesn’t provide feature documentation. This is why websites like Github have been exploding with resources for people to learn to write prompts. Essentially, people are writing the documentation for OpenAI and sharing it with each other.
The Midjourney image generation tool is a little more helpful, and has prompt sharing built right into its operating model. Whenever users generate an image using the tool, they’re added to a browsable catalog that allows other users to see what prompts and settings were used to generate that image. This way, users can learn from each other, rather than having to start from scratch.
Google has taken it one step further and has simplified text editing into easy-access buttons representing commonly used functions. This pattern removes the need for the user to write a prompt altogether. Rather than write a prompt that says “please summarize this text for me” the user can just click the button called shorten.
Takeaway: your users don’t know the power of your tools. Introduce them to what they can do, and help them use it.
Help people navigate the maze of possibilities
Generative AI makes it easy to create a lot of things in a short amount of time. It’s easy to re-write a paragraph in 15 different ways, it’s easy to create 30 variants of an image. But how do you help the user pick which variant is best? And how do you create separate concepts in an interface that is constructed as a single stream of thoughts?
ChatGPT’s chat style interface allows you to have multiple ‘conversations’ with the algorithm. Once you start a conversation, ChatGPT names it based on the contents to help you find it back. As cool as that is, it does have it shortcomings. What if your goal is to get help rewriting an article? You might want to have multiple variants that you want to compare. A chat interface is not ideal for this.
The same goes for Midjourney, that has the chat app Discord as its only way of interacting with it. Users have to open the chat app to send commands to a bot. The bot then in turn responds by adding the generated images to a continuous stream.
Luckily, dealing with collections of items is not a new UI problem. A great example that does allow you to curate can be found in Adobe Lightroom, Adobe’s tool for managing large photography archives. It has mechanisms baked in for filtering, sorting, rating, comparing and selecting. If you’re creating a tool that generates visuals for people, I’d definitely take a look at this tool.
Takeaway: when your app can generate lots of content at a low cost, it needs to help the user in curating and tracking variants.
Foster skepticism and allow people to check the work
Generative AI is good at the What, but less so at the Why. When you ask generative AI to answer a question, it will answer with full confidence, even though the answer can be fully hallucinated and not based on fact. Hallucinations are a side effect of generative AI that is hard to prevent. As such, how do we deal when designing apps that have this weak spot?
The example below is Google’s Bard LLM answering a question. We can debate if Bard is right or wrong here, since the used prompt contained an error which was successfully continued by the bot, the answer is still wrong in the real world. However, Bard gives no context on why or how the answer was given.
One way of tackling this issue is citing sources external to the LLM’s own model knowledge, so that the user can use these to verify facts themselves. Microsoft’s Bing is a great example of how this can be done. At the bottom of each answer they mention different sources the user can peruse to dig deeper on pages that it has based it’s answer on.
Another data source to pull from is called the Confidence Score. Many machine learning algorithms score how certain they are about a specific outcome. Below you can see a screenshot of this in action for OpenAI’s Whisper, a tool that can automatically transcribe audio files into text. By visualizing the confidence score, your user can easily identify parts of the result that need to be looked into, rather than just taking the complete output as truth.
Takeaway: When designing with generative or any other type of AI, make sure you give the user the tools to verify your answers. The machine will not always be right, and your user deserves to know.
Consider legal & ethical implications
With great power comes great legal and ethical responsibility! Before adding generative AI to your application, consider these questions:
What biases are present in the models you’re using?
There is bias in datasets. Midjourney prefers to generate images of men when you ask it to generate images of professionals at their job. What else can we find?
Are you dealing with sensitive or private information?
Most generative AI capabilities run in the cloud through what are called APIs. In essence, this means that to be able to use these services, you are sending data to another companies servers. In these cases it’s good to check what kind of data you are sending, and if it’s clear to your users that you are not the only recipient of their input.
In the case of ChatGPT for instance, their privacy policy clearly states (March 14, 2023) that they are allowed to use all of the user’s personal information provided to them as input. This is in conflict with Europe’s privacy law called GDPR, and it lead to Italy banning ChatGPT recently.
In any case, whenever you use cloud based generative AI, check the privacy policy and see if in your implementation it’s ethical and legal to use.
Is Intellectual Property of the generated output important for your tool?
The debate on who owns the intellectual property for something (partly) created by generative AI is running hot. Copyright has not been legally decided yet, but some cases are currently running that can decide the fate of things for at least the near future.
One of the first legal cases was that of the Zarya of the Dawn comic book. In a legal case, the US Copyright Office decided that it is impossible to copyright AI generated images.
More recently the Copyright Office released a more nuanced guidance that states that you only can’t copyright content that has been generated solely by using an AI prompt, with no further human involvement. As you can imagine, the next question is, how much human involvement is necessary? For now, the jury is out, and I wouldn’t bet on the ability to claim copyright as a certainty anytime soon.
Takeaway: You are the advocate of the user. Ask yourself these questions, ask yourself: am I taking care of the good of the user? And remember, all of these things are in flux, and will remain so for a while. Consider this when adding generative AI to your application, since it will probably not remain in it’s current form for very long.
Keep in mind:
Designers are the experts on design. Don’t let yourself be encouraged that a computer can now generate things you used to handcraft before, let it empower you to do more than you could before.
Humans are the experts on humanity. Computers can generate lots of things, but only you can see which of these things are worth creating, which things bring humanity forward instead of back.
And don’t forget: get your hands dirty and Think by Making!
Guus is a principal designer at argodesign, directing the design of AI enabled tools, and also responsible for experimenting with the latest technologies in design simulations.
