Here’s the text of my talk at SXSW Interaction 2011, March 12, 2011
It’s about time: visualizing temporality
Slide 1 Hi
My name is Joanna Wiebe; I’m an interaction designer and I work in e-commerce. My husband took this of me in Jamaica a couple of years ago. My web site is onemind.com, and my twitter hashtag for this event is #temporality.
Slide 2 It’s about time
Interaction designers can have an interesting and shamanic role in our global village, and that’s what I’m going to talk about.
Slide 3 What is time?
My immigrant Ukrainian grandfather arrived in Montreal without a watch. He walks up to a man on the street and asks, “Please, sir, what is time?” The man replies, “I’m sorry, you’ll have to ask a philosopher. I’m just a physicist.”
My favorite philosopher, Alfred North Whitehead said “It is impossible to meditate on time and the creative passage of nature without an overwhelming emotion at the limitations of human intelligence.” IT does boggle the mind. As a matter of fact, if you’ve read Zen and the Art of Motorcycle Maintenance, you’ll understand what I mean when I say I have had a few Phaedrus moments as I prepared this talk.
However, let’s give it a whirl. Let’s be armchair philosophers.
Slide 4 What is time?
Kids have no idea about time, nor do they care. I taught my children about clocks, and calendars, and timeliness, and it wasn’t easy.
Now that they are grown, they are time slaves, like me. But sometimes, doesn’t time seem to disappear? When you are very emotional, meditating or praying, listening to music or feeling the earth move?
A French geologist named Michael Siffre went underground for several months to live in a cave without clocks or calendars. He wanted to live “like an animal”, he said. His days gradually stretched out, starting with twenty-four and a half hours, then more and more until eventually he was sometimes living 48 hours days. He always thought the days were twenty-four hours, and even though he kept a diary to record his activities during a “day”, he couldn’t tell the difference between these long 48-hour days and a “normal” day.
In a recent interview he said, “It’s the problem of psychological time. It’s the problem of humans. What is time? We don’t know.”
Slide 5 A one-minute history of time
Humans have had many ideas about time over the centuries. In the Medieval world, time – and space — were just a static backdrop for the earth, the center of everything. This geocentric concept is shown in this astronomical clock, built in 1410 in Prague. The blue circle in the middle is the earth.
Copernicus, in the 16th century, taught us that the earth and planets revolve around the sun.
In the Renaissance, Galileo studied the phases of the moon, and developed the pendulum clock. In the 1700s, Newton described a mathematical, clockwork universe. People imagined God as the Big Watchmaker.
Slide 6 The arrow of time pointing to the end of time
Our next big idea was that the sun is part of a galaxy, among billions in the universe. The second law of thermodynamics gave us the concepts of entropy, an arrow of time leading to our universe’s eventual cosmic heat death, and an end to time.
Slide 7 The arrow of time and the evolution of time
Charles Darwin drew a picture of biological progress. The arrow of time is pointing forward, he said, and everything’s evolving! Philosophers, people of faith, and other scientists jumped on his bandwagon. Some people, like Tielhard de Chardin, said that even space-time will evolve into something else.
Slide 8 Mental models evolve
We care about time because we’re going to die. We keep track of time by the rhythms of tides, seasons, sunrise and sunset, heartbeats and menstrual cycles. These are all local clocks, relative to the observer. In fact, that’s all there is, according to Albert Einstein. There is no universal clock somewhere that keeps absolute time. He said, there’s nothing really, that anyone can point to and say, that’s the past, here is the present, there is the future. Past, present and future are just ideas. “Time and space are modes by which we think and not conditions in which we live, ” he said.
And our mental models of time also keep changing. This is not a picture of Einstein, but of Richard Feynman, a theoretical physicist who was an excellent drummer, and whose mental models of time are so evolved that I don’t really understand them.
Slide 9 Four-dimensional design
Some days, it seems like a big part of my work is thinking of ways to let people know that they are just going to have to spend some of their precious time on earth waiting.
As my friend Anne Kennedy says, the web is the only four-dimensional media. And as I say, time is the crux of interaction design.
Slide 10 Digital interaction happens in time
Every digital interaction is a series of actions over time, between two or more agents, human or nonhuman. A person’s keystroke takes a fraction of a second but a system response can take milliseconds, or on the other hand, several days. Uploading my entire hard drive to dropbox.com took four days. To research this talk, I completed a 15-year interaction by retrieving an article I wrote and published to my web site in 1994, by going to the Wayback machine at internetarchive.org.
Slide 11 Time-boxed interactions
Some interactions are time-boxed. My bank holds me in a live session for only a specified time before I have to login again. My search results at Kayak are only valid for a certain amount of time. Ticketmaster makes me race against the timer to enter my credit card information and get the seats I want.
Slide 12 Temporal interaction challenges
As interaction designers, we have some time-related challenges. Designing for different cultures, which hold diverse views of time. Designing interactions that span time zones. And inventing the path beyond technology. I’ll also talk about some cognitive tools that I hope will help you.
Slide 13 Playing nicely even though we see time differently
If we are designing for global audiences, it helps to know how people of different cultures visualize time.
Slide 14 Time flies like an arrow
Groucho Marx once said, Time flies like an arrow but fruit flies like a banana. I agree about the banana, but I am not sure about the arrow.
What is the shape of time? In our culture, time is an arrow. If you want to give someone the idea that you are goal-directed, that you are fast, that you get things done, then you include an arrow pointing forward – to the right — in your logo. Of course, the arrow may be pointing us toward cosmic heat death, but whatever.
Slide 15 Cycles, spirals and fractals
Some cultures see time like a wheel, with cycles of recurring events. Or it’s a spiral, and you can move up or down the spiral with each rebirth. Maybe –even — all of space-time is an evolving fractal, a simple algorithm that yields infinitely complex patterns!
Slide 16 Customer travel lifecycle
As interaction designers, sometimes it is useful to think of processes as cycles. This is a diagram of the cycle of tasks relating to personal travel, and the databases that track and store each person’s choices at each step in the cycle.
Slide 17 Time orientation
Some cultures have a long-term orientation, which is an outcome of being focused on the future, persevering, and being patient in waiting for the results, saving for the children, ten-year business plans. China is an example of a culture with long-term orientation. The United States is seventeenth in long-term orientation, last time I checked.
Last year, I redesigned the car rental flow for ebookers.com. The existing date picker for ebookers was designed by an American, and displayed one month at a time, with arrows to display future months. Because people rent cars a week or two ahead of time, right? But when I spoke with our UK product manager, I learned that our European customers rent cars, on average, about 56 days in advance.
Slide 18 Sixt calendar
In fact, some of our competition in Europe already had three-month calendars.
Slide 19 Sequential and synchronic cultures
Another dialectic in the way different cultures view time is whether they structure it sequentially, or synchronously. These terms are from Fons Trompenaars, a Dutch consultant in cross-cultural communication.
In a sequential culture, time is definitely an arrow. Time moves relentlessly forward, second by second, minute by minute, hour by hour, day by day, in a straight line, from the past to the future. Our day is sectioned, so we can do things one after another. Time is a commodity. Time is money. So a web site for our culture has to be navigationally efficient.
Then there are more synchronic cultures, which I am reminded of when I look at the snapshot on the right from my holiday in Jamaica. A relationship is valued more than staying on schedule. A date made for a specific time is more like a wish, or a temporary desired outcome based on the now. Designing for a culture with a flexible model of time, we might include browse, search or explore features rather than just linear navigation.
Slide 20 Living without time
It’s fun for me to think about designing software for cultures that think about time completely
differently. The traditional Hopi and the Mayan cultures, for example, don’t really have any words for “time”. The Hopi language just puts everything into two buckets – the manifest and the unmanifest. The Mayan cultures have two interlocking calendars, one with 13 months of 20 days and the other more of a solar calendar.
Slide 21 Days as gods
Mayan word for “day”, kin, is the closest thing they have – but this isn’t any abstract unit of measure. Each day has a unique personality, and is creative. In other words, the days are gods.
Another interesting aspect of Mayan time is that time stacks up. Time is cumulative.
Slide 22 Zero point
I cannot omit a mention of Terence McKenna’s view of time, in which time and space are the interplay of habit and novelty. At the “zero point”, or singularity, on December 21, novelty will achieve infinity and something will happen – I’m not sure what.
Also, if you’re interested in these things, Google “Schuman Resonance” for a rationale of how by the end of 2012, time will have sped up so much that “time will stop”, or potentially, be transformed.
Slide 23 Interactions among people who see time differently
If you are designing for a specific culture, you can ask: What is the shape of time for this culture? Is it an arrow? Cyclical? Does the culture have a long- or short-term orientation? Is it sequential or synchronic? Does this culture have a unique view of temporality?
If we are designing for a global audience, should we just use our own Western approach and force everyone to this standard? Or is there a way we can make the software work for people of many cultures?
Slide 24 An operational model of time that works across time zones
Think about what kind of an inner picture of time you can have, when you can get in a plane and travel anywhere in the world in 36 hours, your body adjusting to any number of the earth’s 24 time zones, not to mention daylight savings time, which starts early tomorrow morning, by the way?
Slide 25 Body time
Here I am, completely jetlagged, at the end of an 18-hour day of traveling. When my husband took this shot of me, I was no longer in any earthly time zone. I was in Body Time, the aggregate response of my own biology to the shifting temporal landscape of travel. Haven’t you had that experience of arriving at “breakfast time” when your body is pleading for “bedtime”? I want someone here today to make the tools, that would help people make sense of this kind of experience, to integrate the personal, bodily-felt experience of time with clock time? TripIt doesn’t do it for me.
Slide 26 Now zone
We’re here at SXSW, in this room, but I bet at least some of you are also connecting with family and friends in other places. With texts, Google calendar, Skype, and emails, we are living in a new kind of time zone: the Now zone. When we talk with one another, we are together at the same time, NOW — no matter that the clocks say that the times are different. But when we make plans with someone in another time zone, we always have to make it clear, is that your time or my time? Like the poet Rilke said, “the clocks move separately from our authentic time.”
Slide 27 Achieving an operational model of time that works across time zones
I read a tweet the other day, from a woman who was complaining that she suffered from “multiple temporality disorder” from living in two time zones six hours apart.
Time systems were set up to be local, and we don’t live local any more. We live global. We need to design systems to help us make sense of these experiences of time in our global village. Not just engines that translate one time zone to another, although that helps. I’m envisioning a new kind of calendar. A new language of time. Clocks that operate on our breath, our heartbeats? On the internal resonance of the earth? Got some better ideas?
Slide 28 Finding the path beyond technology
In my lifetime, I’ve gone from using a phone that was a wooden box on the wall, to this little powerhouse device called an EVO.
Slide 29 Finding the path beyond technology
I’ve gone from using a sequential TV schedule in the newspaper, to Amazon Instant video, where I can watch what I want, whenever I want. Our devices are miniaturizing. It’s up to you to make them cognitively invisible. Just the way you enter a room today, and the light automatically comes on, and you don’t even think about it, people will use your technology without being conscious of it. It will be a natural extension of themselves, just part of the ecosystem.
Slide 30 Real-time
Maybe when you think of real-time transactions you think of Google’s Instant Search or texting. Here’s a different kind of real-time I just read about a few days ago. In the past, farmers in the market for newborn piglets would have to drive to town to check the farmer’s coop bulletin board. Now, for newborn piglets, just search Craigslist.
Slide 31 Feedback loops
As real-time interactions become the norm for almost everything, we will be able to keep track of the health of people, populations, economies, and environments, and do something about problems more quickly. Like Katrina or the earthquake and tsunami in Japan. Real-time will give us the feedback loops physicist Friz Capra talks about, to regulate ourselves as a planet.
Physicist Fritjof Capra has said, “. . .a community that maintains an active network of communication will learn from its mistakes, because the consequences of a mistake will spread through the network and return to the source along feedback loops. Thus the community can correct its mistakes, regulate itself, and organize itself . . . The pattern of life, we might say, is a network pattern capable of self-organization.”
With everything you learn that makes you a better interaction designer, you are working toward building an active network of communication around the world. You are moving our systems toward the day when we can fluidly talk with one another in real-time. Of course, it will still be up to each one of us to decide to have goodwill and practice it with one another. But you can give us the possibility of economic and political systems in which everyone can participate in open, global interdependency. You’re moving the needle toward Marshall McLuhan’s “brand-new world of allatonceness” in which time has ceased and space has vanished, where we will be, as McLuhan has said, profoundly involved with one another.
Slide 32 Latency, session management and timeouts
But, we’re not quite there yet. Last night, I sent my husband a text message, and he responded. My phone indicated that he had texted back, in fact, two minutes before I had sent him the original message. Now, that’s latency!
To get to true real time, much has to be done on the front end AND our back end systems. And as designers, we need to understand our existing back end systems so we can make the best use of them.
Here’s my husband Tim, eating chips on the balcony of a hotel in Spain. The picture reminds me of the frustrating afternoon I spent trying to book that hotel, getting timed out over and over again.
The exciting part was that I worked at Orbitz, so I had a chance to help fix the timeout problem.
Why? I asked. Why these short sessions? Why can’t we just hold everyone in session indefinitely?
I learned that each hotel search called on a byzantine collection of global distribution systems for hotel inventory. Orbitz servers stored this collection of search results to create a stateful session from the essentially stateless behavior of the web. Our power wasn’t metered, so it wasn’t a matter of electric cost. But increasing session lengths or having more sessions had an impact on server performance. There were load issues at peak times of the day.
Also, it’s not safe to prolong everyone’s sessions indefinitely if credit card information has been entered. Travel data and prices quickly become stale, and there are latency issues with continually refreshing the data.
First of all, we increased session time across the board as much as we could without affecting performance or latency. Then we also timed out different pages in the booking path based on historical behavior of our key market segments. If the customer clicked on a select group of triggers, they would be asked if they would like to see refreshed results. We still had the same old awkward backend systems but we had improved the user experience through interaction design. Bookings went up.
Slide 33 Tools for time work
To do brilliant work with temporal issues in the user interface, here is a short checklist of cognitive tools you might find useful. By the way, it helps me to think of interaction design like composing music, because a lot of the same fundamentals apply.
– Pacing and rhythm
Slide 34 Sequence, and direction
To be an interaction designer, we have to be masters of sequence, and the direction of sequence, the subtleties of cause and effect. This is the cover of the graphic adventure video game Myst, in which the player explores an environment to see which causes will trigger helpful effects. As designers, how do we know when to make an effect explicit, and as discoverable as possible? How do we know when should we make the effect a learnable surprise, like in Myst?
Slide 35 Process flows show sequence
This is an example of a process flow I designed for a single screen of a back-end tool, to try to tease out all the conditional steps in a process, and which causes would lead to which effects. You don’t need to read all this – I am just showing you this to illustrate just how complex these box and arrow decision trees can get.
Slide 36 Functional descriptions that show cause and effect
And here’s a more traditional information architecture wireframe, again annotated with functional specifications so the developer knows what each trigger is supposed to do. Again, don’t try to read this.
Slide 37 Conceptual Graph Structures
Another tool for working with sequence is a cognitive task analysis method called Conceptual Graphing. In this method six kinds of nodes are connected by a formal grammar via 18 types of arcs, setting up semantic relationships. This method is based on research into how people tell stories. Here’s a hand-drawn example from my friend Scott, who introduced me to this method. For a deeper dive on this, check my web site. http://onemind.com/2010/01/27/conceptual-graph-structures-part-1/
Slide 38 Pacing and rhythm
Are you designing a roller coaster or a tea party? What’s the most efficient rhythm of work for the typical person? What pace is going to be the most fun? Every journey has its natural pace, including the paths through an e-commerce form, a game, or a web site. Push it too hard or go too slow, and it’s just painful. Or even impossible. Then also consider, are the people novices or experts? Are they multi-tasking or will they be giving this effort a single-minded focus?
The mind can absorb no more than the seat of the pants can endure. Is a group of tasks something that a person can do in a single sitting? Or are you going to make them get up several times to go find information that they don’t readily have at hand, interrupting their pace? Are you going to make someone click “Next” and “Next” and “Next” without giving them clues as to how long this is going to take? Are you going to interrupt them with interstitials and progress bars?
Slide 39 Duration
A time-related user interface problem, which I don’t think anyone has solved, is how to visually display an array of flights with their relative durations, as well as the durations of stopovers, overall elapsed time, time between flights, actual time in the air for each portion of the flight, time zone shifts, probability of delay, not to mention depart and landing times. This is a screen grab of an experiment in how to display flight duration.
Slide 40 Interval
When I began working at Orbitz, one of the first time-related issues I ran across was that there was not a common understanding of the intervals in a person’s itinerary. So I put together this diagram of flight components so we could start to think about these intervals.
Slide 41 Patterns
Time gives us context for understanding data. In time series visualizations, we can see how things change over time. Are there trends, patterns? What is the rate of change?
There are many time series display methods: from line graphs to heat maps. Stephen Few, Ben Shneiderman, Edward Tufte, and many others have done excellent work in the field of time series display.
Slide 42 Lifelines
Here’s an example of some conceptual work Shneiderman was involved in at the University of Maryland, to use interactive time series display for personal medical records. A single screen displays an expodable timeline of a person’s entire medical history.
Slide 43 Priority – implicit or explicit
Within the sequence, what should people do first? TurboTax gives you an implicit priority by the sequencing of the icons, and the 3-D, time-as-an-arrow diagram. An explicit priority of tasks can be shown through numbering. A wizard interface imposes priority.
Slide 44 The interaction designer as shaman
Speaking of wizards, this is Tim, the Shaman, from Monty Python and the Holy Grail. Tim says to the brave knights, “Follow. But. Follow only if ye be men and women of valour.“
I like Esther Dyson’s comment, “I’ve always done things I wasn’t ready for.” Whether or not you feel you have the skills, or the experience, or the background, be brave. Help people connect across cultural differences. Across time zones. Make technology that’s easy as breathing. Invent new calendars and clocks. This kind of work will help people do what they want what’s most important to them. Which is to love and be loved by others. To be in relationship. To have some fun. We want to work, play, support our families. And we want to do it in a way that we aren’t piling up five pounds of trash per person per day the way we are now. So that the polar bears can live, and the baby birds aren’t full of plastic bits, and the air is clean.
My vision is of a computing world where we are help be these kind of people – and ever more so – in a way that we’re not thinking to ourselves, “now I’m using a computer,” or, “now I’m running software applications now,” or, “now I’m using a wireless network device.” My vision is that everything’s going to be effortlessly integrated, so that we’re just doing the things humans like to do – work, play, learn — eat, pray, love – and the tools we are using to get things done just fade into the background of our awareness.
Slide 45 The power of a vision
A rock pile ceases to be a rock pile the moment a single man contemplates it, bearing within him the image of a cathedral.
– Antoine de Saint-Exupery
Looking back over history, we can see that new designs for clocks and calendars have often been the precursor of some kind of revolution or another.
Time is our sandbox. We can expand time. We can make time fly. We can bring people together in the Now zone, to heal and to help.
This kind of work we can do seems shamanic, to some, because we see a pattern where others only see a pile of rocks.
Click the image above to see the presentation on SlideRocket.
The image of the clock at the top of the post is the Question Mark Clock in Seattle, WA.