The Design of Future Things Read online

  After I had posted a draft version of this chapter on my website, I received a letter from a group of researchers who were exploring the metaphor of horse and rider to the control of automobiles and airplanes. The “H-metaphor,” they called it, where “H” stands for “horse.” Scientists at the American National Aeronautics and Space Administration research facilities at Langley, Virginia, were collaborating with scientists at the German Aerospace Center’s Institute for Transportation Systems in Braunschweig, Germany, to understand just how such systems might be built. I visited Braunschweig to learn more about their work (fascinating stuff, to which I return in chapter 3). Riders, it seems, delegate the amount of control they give to the horse: when using “loose reins,” the horse has authority, but under “tight reins,” the rider exerts more control. Skilled riders are in continual negotiation with their horses, adjusting the amount of control they maintain to the circumstances. The American and German scientists are trying to replicate this relationship with human-machine interaction—not only with cars but with houses and appliances.

  Symbiosis, in the sense meant by Licklider half a century ago, is a merger of two components, one human, one machine, where the mix is smooth and fruitful, the resulting collaboration exceeding what either is capable of alone. We need to understand how best to accomplish this interaction, how to make it so natural that training and skill are usually not required.

  Skittish Horses, Skittish Machines

  What would it mean for a car and driver to interact much as a skilled rider interacts with a horse? Suppose a car were to balk or act skittish when getting too close to the cars ahead or when driving at a speed it computed to be dangerous? Suppose the car responded smoothly and gracefully to appropriate commands and sluggishly and reluctantly to others? Would it be possible to devise a car whose physical responsiveness communicated the safety status to the driver?

  What about your house? What would it mean to have a skittish house? I can see my vacuum cleaner or stove acting up, wanting to do one thing when I wanted it to do another. But my house? Today companies are poised to transform your home into an automated beast, always looking out for your best interests, providing you with everything you need and desire, even before you know you need or desire it. Many companies are anxious to equip, wire, and control these “smart homes”—homes that control the lighting according to their perception of your moods, that choose what music to play or that direct the television images to move from screen to screen as you wander about the house. All these “smart” and “intelligent” devices pose the question of how we will be able to relate to all this smartness. If we want to learn to ride a horse, we have to practice or, better yet, take lessons. So, do we need to practice how to use our home, to take lessons on getting along with our appliances?

  What if we could devise natural means of interaction between people and machines? Could we learn from the way that skilled riders interact with horses? Perhaps. We would need to determine the appropriate behavioral mappings between the behaviors and states of the horse and rider and those of the car and driver. How would a car indicate nervousness? What is the equivalent for a car to a horse’s posture or skittishness? If a horse conveys its emotional state by rearing back and tensing its neck, what might the equivalent be for a car? What if suddenly your car reared back, lowering its rear end while raising the front, perhaps moving the front end left and right?

  Natural signals akin to what the horse receives from its rider are actually being explored in research laboratories. Research scientists in the automobile companies are experimenting with measures of emotion and attention, and at least one automobile model sold to the public does have a television camera located on the steering column that watches drivers, deciding whether or not they are paying attention. If the automobile decides that a crash is imminent but the driver is looking elsewhere, it brakes.

  Similarly, scientists are hard at work developing smart homes that monitor the inhabitants, assessing their moods and emotions, and adjusting room temperature, lighting, and background music. I’ve visited several of these experiments and observed the results. At one research facility at a European university, people were asked to play a stressful video game, then allowed to rest afterwards in a special experimental room equipped with comfortable chairs, friendly and aesthetically pleasing furniture, and specially equipped lighting designed to relax the inhabitants. When I tried it, I found it to be a calm and restful environment. The goal of the research was to understand how to develop room environments appropriate to a person’s emotional state. Could a home relax its inhabitants automatically when it detected stress? Or perhaps the home could take on a zingy, upbeat mood with bright lights, lively music, and warm colors when it determined that the inhabitants needed an energy boost.

  Thinking for Machines Is Easy; Physical Actions

  Are Hard; Logic Is Simple, Emotion Difficult

  “Follow me,” says Manfred Macx, the hero/narrator of Charles Stross’s science fiction novel Accelerando, to his newly purchased luggage. And follow him it does, “his new luggage rolling at his heels” as he turns and walks away.

  Many of us grew up with the robots and giant brains of novels, movies, and television, where machines were all-powerful, sometimes clumsy (think of Star Wars’ C–3PO), sometimes omniscient (think of 2001’s HAL), and sometimes indistinguishable from people (think of Rick Deckard, hero of the movie Blade Runner: is he human or replicant?). Reality is rather different from fiction: twenty-first century robots can’t conduct any meaningful communication with people; indeed, they are barely capable of walking, and their ability to manipulate real objects in the world is pathetically weak. As a result, most intelligent devices—especially in the home, where costs must be kept down and reliability and ease of use kept up—concentrate on mundane tasks such as making coffee, washing clothes and dishes, controlling lights, heating, and air conditioning, and vacuuming, mopping, and cutting the grass.

  If the task is very well specified and the environment under control, then intelligent machines can indeed do a reasonable, informed job. They can sense temperature and moisture, as well as the amount of liquid, clothing, or food, and thus determine when the laundry is dry or the food is cooked. The latest models of washing machines can even figure out what kind of material is being washed, how large the load is, and how dirty the clothes are, and adjust itself accordingly.

  Vacuum cleaners and mops work as long as the pathway is relatively smooth and clear of obstacles, but the luggage that follows its owner in Stross’s Accelerando is still beyond the capability of affordable machines. Nonetheless, though, this is precisely what a machine might be able to do, for it doesn’t require real interaction with people: no communication, no safety-related issues, just follow along. What if someone tried to steal the freewheeling suitcase? It could be programmed to scream loudly at any attempt, and Stross tells us that it has learned the owner’s “fingerprints, digital and phenotypic”: thieves might be able to steal it, but they wouldn’t be able to open it.

  But could the luggage really make its way through crowded streets? People have feet, the better to step over and around obstacles, to go up and down stairs and over curbs. The luggage, with its wheels, would behave like a handicapped object, so it would need to seek out curb cuts at street intersections and ramps and elevators to maneuver within buildings. Human wheelchair users are often stymied: the wheeled luggage would be even more frustrated. And beyond curbs and stairs, navigating through city traffic would likely defeat its visual processing systems. Its ability to track its owner, avoid obstacles, and find paths navigable by a nonlegged device, while avoiding collisions with automobiles, bicycles, and people, would surely be compromised.

  There is an interesting disjunction between the things people and machines find easy and hard. Thinking, which once was held up as the pinnacle of human achievement, is the area in which machines have made the greatest progress, especially any thinking that requires logic and attention to
detail. Physical actions, such as standing, walking, jumping, and avoiding obstacles, are relatively easy for people, but difficult if not impossible for machines. Emotions play an essential role in human and animal behavior, helping us judge what is good or bad, safe or unsafe, while also providing a powerful communication system for conveying feelings and beliefs, reactions and intentions among people. Machine emotions are simplistic.

  Despite these limitations many scientists are still striving to create the grand dream of intelligent machines that will communicate effectively with human beings. It is in the nature of research scientists to be optimists, to believe that they are doing the most important activity in the world and, moreover, that they are close to significant breakthroughs. The result is a plethora of news articles, such as this one:

  Researchers say robots soon will be able to perform many tasks for people, from child care to driving for the elderly.

  Some of the country’s leading robotics experts gathered here Saturday at the annual meeting of the American Association for the Advancement of Science to present their latest research and talk about a future rife with robots. . . .

  [Y]our future could include: a huggable teddy bear that tutors your kids in Spanish or French; an autonomous car that drives you to work while you nap, eat or prepare your PowerPoint presentation; a Chihuahua-sized pet dinosaur that learns whether you like to cuddle, play or be left alone; a computer that can move its screen to help your posture or match your task or mood; and a party-bot that greets your guests at the door, introduces them in case you’ve forgotten their names, and entertains them with music, jokes and finger food.

  Many conferences are held to discuss progress on the development of “smart environments.” Here is the wording of one invitation among the many that I receive:

  Symposium on Affective Smart Environment. Newcastle Upon Tyne, UK.

  Ambient Intelligence is an emerging and popular research field with the goal to create “smart” environments that react in an attentive, adaptive and proactive way to the presence and activities of humans, in order to provide the services that inhabitants of these environments request or are presumed to need.

  Ambient Intelligence is increasingly affecting our everyday lives: computers are already embedded in numerous everyday objects like TV sets, kitchen appliances, or central heating, and soon they will be networked, with each other. . . . [B]io-sensing will allow devices to perceive the presence and state of users and to understand their needs and goals in order to improve their general living conditions and actual well-being.

  Do you trust your house to know what is best for you? Do you want the kitchen to talk to your bathroom scale, or perhaps to have your toilet run an automatic urinalysis, sharing the results with your medical clinic? And how, anyway, would the kitchen really know what you were eating? How would the kitchen know that the butter, eggs, and cream taken out of the refrigerator were for you, rather than for some other member of the household, or for a visitor, or maybe even for a school project?

  Although monitoring eating habits wasn’t really possible until recently, we can now attach tiny, barely visible tags on everything: clothes, products, food, items, even people and pets, so everything and everybody can be tracked. These are called radio frequency identification (RFID) tags. No batteries are required because these devices cleverly take their power from the very signal sent to them asking them to state their business, their identification number, and any other tidbits about the person or object they feel like sharing. When all the food in the house is tagged, the house knows what you are eating. RFID tags plus TV cameras, microphones, and other sensors equals “Eat your broccoli,” “No more butter,” “Do your exercises.” Cantankerous kitchens? That’s the least of it.

  “What if appliances could understand what you need?” asked one group of researchers at the MIT Media Lab. They built a kitchen with sensors everywhere they could put them, television cameras, and pressure gauges on the floor to determine where people were standing. The system, they said, “infers that when a person uses the fridge and then stands in front of the microwave, he/she has a high probability of re-heating food.” “KitchenSense,” they call it. Here is their description:

  KitchenSense is a sensor-rich networked kitchen research platform that uses CommonSense reasoning to simplify control interfaces and augment interaction. The system’s sensor net attempts to interpret people’s intentions to create fail-soft support for safe, efficient and aesthetic activity. By considering embedded sensor data together with daily-event knowledge, a centrally-controlled OpenMind system can develop a shared context across various appliances.

  If people use the refrigerator and then walk to the microwave oven, they have a “high probability of reheating food.” This is highfalutin scientific jargon for guessing. Oh, to be sure, it is a sophisticated guess, but a guess it is. This example makes the point: the “system,” meaning the computers in the kitchen, doesn’t know anything. It simply makes guesses—statistically plausible guesses based on the designer’s observations and hunches. But these computer systems can’t know what the person really has in mind.

  To be fair, even statistical regularity can be useful. In this particular case, the kitchen doesn’t take any action. Rather, it gets ready to act, projecting a likely set of alternative actions on the counter so that if by chance one of them is what you are planning to do, you only have to touch and indicate yes. If the system doesn’t anticipate what you had in mind, you can just ignore it—if you can ignore a house that constantly flashes suggestions to you on the counters, walls, and floors.

  The system uses CommonSense (any confusion with the English term “common sense” is deliberate). Just as Common-Sense is not really a word, the kitchen doesn’t actually have any real common sense. It only has as much sense as the designers were able to program into it, which isn’t much, given that it can’t really know what is going on.

  But what if you decide to do something that the house thinks is bad for you, or perhaps simply wrong? “No,” says the house, “that’s not the proper way to cook that. If you do it that way, I can’t be responsible for the result. Here, look at this cookbook. See? Don’t make me say ‘I told you so.’” This scenario has shades of Minority Report, the Steven Spielberg movie based upon the great futurist Philip K. Dick’s short story by that name. As the hero, John Anderton, flees from the authorities, he passes through the crowded shopping malls. The advertising signs recognize him, calling him by name, tempting him with offers of clothes and special sale prices just for him. A car advertisement calls out, “It’s not just a car, Mr. Anderton. It’s an environment, designed to soothe and caress the tired soul.” A travel agency entices him: “Stressed out, John Anderton? Need a vacation? Come to Aruba!” Hey, signs, he’s running away from the cops; he isn’t going to stop and buy some clothes.

  Minority Report was fiction, but the technology depicted in the movie was designed by clever, imaginative experts who were very careful to depict only plausible technologies and activities. Those active advertising signs are already close to becoming a reality. Billboards in multiple cities recognize owners of BMW’s Mini Cooper automobile by the RFID tags they carry. The Mini Cooper advertisements are harmless, and each driver has volunteered and selected the phrases that will be displayed. But now that this has started, where will it stop? Today, the billboard requires its audience to carry RFID tags, but this is a temporary expedient. Already, researchers are hard at work, using television cameras to view people and automobiles, then to identify them by their gait and facial features or their model, year, color, and license plate. This is how the City of London keeps track of cars that enter the downtown area. This is how security agencies expect to be able to track suspected terrorists. And this is how advertising agencies will track down potential customers. Will signs in shopping malls offer special bargains for frequent shoppers? Will restaurant menus offer your favorite meals? First in a science fiction story, then in a movie, then on the city s
treets: look for them at your nearest shops. Actually, you won’t have to look: they will be looking for you.

  Communicating with Our Machines: We Are Two

  Different Species

  I can imagine it now: it’s the middle of the night, but I can’t sleep. I quietly get out of bed, careful not to wake up my wife, deciding that as long as I can’t sleep, I might as well do some work. But my house detects my movement and cheerfully announces “good morning” as it turns on the lights and starts the radio news station. The noise wakes my wife: “Why are you waking me up so early?” she mumbles.

  In this scenario, how could I explain to my house that behavior perfectly appropriate at one time is not so at another? Should I program it according to the time of day? No, sometimes my wife and I need to wake up early, perhaps to catch a morning flight. Or I might have a telephone conference with colleagues in India. For the house to know how to respond appropriately, it would need to understand the context, the reasoning behind the actions. Am I waking up deliberately? Does my wife still want to sleep? Do I really want the radio and the coffeemaker turned on? For the house to understand the reasons behind my awakening, it would have to know my intentions, but that requires effective communication at a level not possible today or in the near future. For now, automatic, intelligent devices must still be controlled by people. In the worst of cases, this can lead to conflict. In the best of cases, the human+machine forms a symbiotic unit, functioning well. Here, we could say that it is humans who make machines smart.