The best way to learn is to teach. I occasionally give lectures in universities, design schools and business schools about my 15+ years experience working at the crossroads of research and design in domains such as ubiquitous computing, big data, machine learning and futures design. This year, Delfina Moran, the Director of the Master Program in User Experience for Digital Products and Services at UDIT (University School of Design, Innovation and Technology in Madrid) proposed that I transform my talk into a full course on Emerging Technologies (i.e. digital technologies whose development and practical applications are still largely unrealized). Member of the Near Future Laboratory network and one of the sharpest creative technologists I know, Rohit Gupta, kindly agreed to play the assistant role and help me with that.

Many of the students at UDIT are part of a new generation of designers that refuse to be pigeonholed into roles for solving specific problems, designing apps or websites. Besides the increase in specializations in design, their role as young professionals is to be curious, to question the status quo, and above all, it is about “sense making” and understanding what to do with technology instead of blindly following a double diamond process. I have nothing against work packaged into a design process model, except that it often prevents designers from considering what happens when things do not work as expected. Following a clean path from one step to the next until the delivery of a “solution” does not leave room to anticipate its impact on individuals, society, and the environment after that. Truth is that there is nothing linear and procedural in working with emerging technologies because their results are too often hard to predict. It is common that the learnings gathered while experimenting require a team to step back and question what was agreed earlier in the process. Or as another member of the Near Future Laboratory and former Head of Design at Google X Nick Foster would say: “Design work is complex, chaotic and messy.”

It is how to engage with the high level of ambiguity inherent to emerging technologies that we wanted students to experience and learn.

The Objectives

With Rohit, our objective for this course was to teach students how to make sense and appropriate technologies whose development are still largely unrealized. We structured the course with a “prototyping studio” as a backbone for experimentation with new tools and frameworks. Students could understand first hand how data, algorithms, sensors and machine learning have become new design materials. For each topic, we invited professionals to share the recent evolutions at the edges of UX design. We wanted students to see opportunities but also to learn the implications of the presence of emerging technologies for their careers and the user experiences they will design. So we took time to discuss and practice field observations, design fiction and critique techniques. These were the objectives shared with students:

We hope you will…

• Learn how to make sense of “what’s coming next”.
• Experiment with new tools and frameworks.
• Discover the opportunities and implications for your career as UX designer.
• Discover the opportunities and implications for the user experiences that you will design.
• Learn by creating, presenting and critiquing prototypes.
• Have (serious) fun!

The Philosophy

We applied the spirit of a studio-based practice that encourages collaborative learning by talking, making, and critiquing. Call it “make-things-to-think” or “designing-by-doing”, we were inspired by Jon E. Froehlich course on Physical Computing at the University of Washington not only by the approach with hands-on exercises but also for his energy in sharing his passion for the topic. Rohit designed practical exercises as compasses on how a particular technology works. Many in-class exercises were done in small groups, and we mixed individual and small group projects for the homework assignments.

We made sure to use examples related to the student’s interests and close to our professional experience. Emerging technologies can be a pretty sober and humorless space. We believe it does not need to be. Rohit used a good dose of humor to keep each session fun and engaging. Humor can be a critical ingredient in any effective teaching, particularly to capture the attention at the moment of sharing key messages.

The Schedule

Class 1: Introduction

• Definitions
• Communities that study emerging technologies
• Theories and frameworks
• Mapping today’s emerging technologies
• Observing the presence of emerging technologies

Prototyping studio

• Observations of the presence of emerging technologies in your daily life
• Mapping impact of these emerging technologies on UX design

Assignment: Field observation

Sci-fi writer William Gibson famously said: “The future is already here — it’s just not evenly distributed.” Chances are, you have been in contact with emerging technologies without even realizing it. The objective of this exercise is to train your eyes in detecting the presence of digital technologies in a physical context and to produce clear descriptions of the user interactions that they produce.

Class 2: Big Data

• Definitions
• From user interactions to big data
• New/modified user experiences (e.g. Quantified self)
• Sensors and remote sensing
• Data structures
• Data storage
• Data preparation
• Data manipulation and visualization
• New tools and approaches (e.g. Data-driven/informed design)
• Cloud computing

Prototyping studio

• Let’s manipulate and visualize a dataset
• Introduction to Rawgraphs
• Introduction to Madrid Open Data
• Mapping Sensing stations to their locations
• Using a subset to explore air-quality data

Assignment: Data manipulation and visualization

You are part of the Spotify data+design team that prepares the Spotify Wrapped campaign for 2023. Some new ideas have emerged during an early brainstorming session. It is your mission with a colleague to visualize and prototype them for the next team workshop. You will manipulate a Spotify dataset from Kaggle to and answer key questions with visualizations.

Class 3: Machine Learning

• Definitions
• From big data to machine learning
• New/modified user experiences (e.g. Feedback loops, habit-forming products)
• How do machines learn? (e.g. Data labeling, biases)
• A bit of history
• New tools and approaches (e.g. Prompt engineering, generative tools, simulations, trust)
• Ethical considerations

Prototyping studio

• Terminology and concepts review
• Tour de Neural Networks
• Let’s build a Deep Learning model (Teachable Machines)
• Prompt engineering 101 (ChatGPT, DALL-E)

Assignment: Deep Learning model

Spotify has signed an agreement with Logitech to develop a smart camera for home parties. These cameras will connect to the Spotify app. Your assignment is to develop, prototype and test a hand-gesture language to interact with these cameras and communicate actions to the app.

Class 4: Smart Objects

• Past and present of ubiquitous computing (e.g. context-aware, ambient intelligence, calm computing, Internet of Things…)
• From machine learning to smart objects
• Touchless Interactions: beyond point-and-click and touchscreen interfaces
• Design the immaterial
• Design for privacy and security
• Design for hackability
• Design for ecological limits

Prototyping studio

• Let’s build a smart object
• Introduction to electronics
• Introduction to Arduino
• Programming in Arduino (the IDE)
• Connecting our AI model to Arduino

Assignment: Prototype a smart product

Say hello to the first prototype of the “home parties’’ camera that the Spotify Data+Design team received from Logitech. Your assignment for our final session is to prototype the user experience of that smart product. This includes:

• Create the visual feedback (the camera has 8 LEDs) the camera gives in response to each hand gesture
• Connect your hand-gesture language developed in Teachable Machine to an Arduino board and LEDs
• Create a demo video (1- min) that explains how the prototype works and presents results with different users. Show when it works well and when it does not work.
• Write and illustrate the Quick Start Guide that customers would receive with the webcam.

Class 5: eXtended Reality

• Definitions
• From smart objects to eXtended Reality
• Domains of application
• The design challenge of augmenting people’s vision
• The design challenge of virtual collaboration

Prototyping studio

• Let’s build an AR app (Adobe Aero)
• UI Overview
• Making your first scene in AR and previewing it
• Adding more objects
• Adding triggers and behaviors
• Brings models to Adobe Aero

Class 6: Beyond Tomorrow

• Design and the planet
• Design and the digital transformation (web3, quantum computing, robotics)
• Futures design
• “Open” prototypes. “Open” = They are not solutions but explorations
• Design fiction 101
• Prototype presentation and critique
• Backcasting

Prototyping studio

• Let’s create an FAQ from the future
• Let’s put ourselves in the shoes of users of our webcam and list all the questions they would have. (e.g. setup, key hand-gestures, privacy, potential issues, troubleshooting, offboarding, …)

Classe 7: Prototype Presentation and Critique

In a 10-minute presentation, each group used their prototype (including videos captured during development and testing and a printed quick start guide) to share their learnings:

• About the hand-gesture language and visual feedback they developed.
• When the prototype does not work and what could be done to mitigate the failures/limitations.
• About the key user experience question they expect from customers and how they answered them.
• About the key implications in introducing this new concept into the market (e.g. privacy, ecological issues, inclusivity).

The Lecture

I introduced each class with a 30 to 60 minutes lecture on a specific technology that aimed at providing context to a specific technology and to the hands-on work. Typically, I would offer definitions, narrate an historical evolution and show how different technologies interconnect. I always enjoyed professors who could transmit their knowledge from their professional experience. So, I relied as much as possible on using my projects to discuss the applications of a technology and the types of challenges that UX designers face. That made the conversation practical and when sharing the details of specific projects, I could use anecdotes to talk with nuances. With Rohit, when we felt we could not provide enough depth to a topic we invited a professional from our network to share their work that pushes the boundaries of UX design.

The Guest Practitioners

People are extremely generous with their time when it comes to showcase what they do. In most of the classes, we invited a guest practitioner (either in class or via Zoom) to talk about the practice as a UX designer on the specific topic covered. We allocated brief 30-minute sessions for them to present their recent projects in the domain and also reflect on their practice, its evolution, and the future. We wanted students to feel the day-to-day life of UX professionals who push the boundaries of the practice. We wanted them to feel inspired.

Juan Morales del Olmo talked about data visualization and the work at the crossroad of data science and UX design, Daniele Pezzatini shared the work at Bestiario that mixes creativity with data and algorithms, Tim Stutts gave us a deep dive into the design of assisted reality applications and Samuel De Sosa shared his pioneer design work in the Metaverse.

The Prototyping Studio

The application of emerging technologies comes with a level of ambiguity. When using data, algorithms, AI as design material the limitations, imperfections opportunities are not clear. Prototyping is a strategy for efficiently dealing with things that are hard to predict. We wanted students to be open to ambiguity and uncertainty. In this course, we used prototypes not as solutions but explorations. Prototypes as means to learn, to discover and demystify a technology instead of a means to propose a solution to a specific problem.

Rohit designed class-session prototypes for students to grasp key concepts and tools. He built a collection of resources and links to explore further up. And we used the homework assignments as the opportunity to design something that could belong to the students portfolio.

We focused on the mundane, day-to-day user experiences close to students’ interests (e.g. air quality, listening to music, gathering with friends, etc) to make the prototyping fun and engaging. The results were pretty amazing with students imagining a hand-gesture language, implementing a Deep Learning model to detect these hand gestures and connecting it to an Arduiono board that creates a feedback loop with a LED display.

Final Thoughts

We treated this first course as a prototype and we learned a lot particularly on trying to find the right balance between overwhelming and underwhelming students. Also, I came to realize that courses about technologies for designers often focus too much on the state of the art. Ours was no different. In the next iteration, I would probably dedicate less time coding in class and more time discussing the history of technology. My hypothesis is that professionals with a good understanding of the cultural and social contexts in the development of tools and techniques have a better chance in creating meaningful UX experiences with emerging technologies.

Teaching this course would not have been possible without the collaboration of Rohit Gupta. Many thanks to the 36 Master students in User Experience for Digital Products and Services at UDIT for their energy and curiosity. And special thanks to the director of the Master Delfina Moran for insisting on getting me onboard.

I am Fabien Girardin. I am a researcher, engineer, data scientist and futures designer with a Ph.D. in Computer Science and Digital Communication. You can get in touch with me through the regular internet endpoints. Each week I set aside a couple hours for Office Hours video call for anybody to talk about anything related to emerging technologies and design.