September 2003
http://www.coe.berkeley.edu/labnotes/0903
Commuter-friendly version

Professor Kris Pister is currently on industrial leave from the University, working at his start-up company Dust Inc. Peg Skorpinski photo

"For fourteen years, I've had this dream of making silicon walk," says UC Berkeley professor Kristofer Pister of the Department of Electrical Engineering and Computer Sciences.

It's a startling idea: Swarms of ant-size robots burrowing through the rubble of a building after an earthquake searching for survivors or crawling onto the hull of a spacecraft to repair damage in-flight. But perhaps the most amazing thing about Pister's dream is that it's not as far off as one might think. Already Pister and his graduate students have built simple solar-powered microrobots just 8.5 millimeters long and less than 4 millimeters wide.

"It only has two legs and its tail is too heavy, so it can't quite walk," Pister says. "But it does push-ups."

Still, the solar-powered bug demonstrates that tiny autonomous robots can be fabricated using the same technology used to manufacture integrated circuits. The current microrobot prototypes were built in three separate pieces in UC Berkeley's state-of-the-art Microfabrication Laboratory. The first component is a digital logic chip that controls the walking motion of the two legs. The second includes the near-microscopic solar cells, designed by Pister's former graduate student Seth Hollar. Those two components are then combined with the legs, motors and frame of the robot. Eventually, all three steps could be combined into a single process, enabling the microrobots to be cranked out in bulk at costs of perhaps less than $1 each.

Graduate student Sarah Bergbreiter examines a microrobot using a high-powered magnifying glass. David Pescovitz photo

Key to the microrobot's locomotion is the novel MEMS (micro-electromechanical systems) the researchers developed during the last several years. Aptly-named "inchworm" motors work by repeatedly engaging a shuttle that pulls the leg forward a minuscule amount, releases it, and then engages it again to move it a bit more. Similar to the way a person climbs a ladder, the repetition of small steps will provide the legs with enough force and displacement (distance of travel) to carry the microrobot along.

"Traditionally with MEMS, you get either high force or large displacements," says lead graduate student Sarah Bergbreiter. "With the inchworm motors, you have both."

Right now, the microrobot is crippled by a clutch that slips when the leg pushes against the ground. The problem, Bergbreiter says, should be solvable with minor adjustments to the design of the device before the next batch of microrobots is fabricated.

A scanning electron microscope (SEM) image of the microrobot's legs. Photo courtesy the researchers

The microrobot research preceded Pister's Smart Dust motes, tiny wireless transceivers outfitted with sensors for myriad applications. A key component in the efforts of the Center for Information Technology Research in the Interest of Society (CITRIS), Smart Dust has a multitude applications— from diagnosing a building's structural integrity to measuring light and temperature for energy use monitoring. Outfitted with their own TinyOS operating system, the motes self-organize into ad hoc wireless networks and pass their data from one to another bucket-brigade style until the information reaches a central computer for processing. Now, Pister and Bergbreiter, with private sector collaborator Anita Flynn of MicroPropulsion Corporation are finally able to realize the full potential of the Smart Dust platform.

A full view of the two-legged microrobot. Image courtesy the researchers.

"MEMS technology has shrunk so much that we can start to make microrobots which are essentially smart dust with legs," Bergbreiter says.

What applications do you think will benefit from the microrobot? We want to hear from you...

Video: The microrobot doing "calisthenics." (courtesy the researchers)

The Smart Dust-style sensors and on-board computer processing are what will provide the microrobots with their autonomy, Bergrbreiter says. While each robot will control its own activity, the power will come from deploying them in swarms. For instance, much like ants build a nest, a swarm of microrobots dropped on Mars could work collaboratively to construct a satellite antenna so they can transmit their environmental readings to an orbiting spacecraft. Even more amazing, Bergbreiter says, they could crawl on top of each other to build a larger silicon structure out of themselves.

"In my opinions, robotics has always meant big lumbering machines," Bergbreiter says. "I like the idea of making very simple robots, but a lot of them. If one component of a big robot fails, the robot is finished. But if one microrobot dies, the rest of them continue to function and the task can still be completed."

UC Berkeley professor Jitendra Malik, Associate Chair of the Computer Science Division

UC Berkeley professor Jitendra Malik, associate chair of the computer science division, has become an expert in the graceful human dynamics of ballet. Malik is not a male ballerina though, nor a particularly big fan of classical dance. He's had to study the subtleties of plies and releves and a host of other human motions in order to teach a computer to identify what a person is doing just by watching him or her. His novel approach to computational analysis of human movement has myriad applications, from ultra-realistic videogames, where the players control human actors on the screen to, surveillance.

"A big aspect of human intelligence is vision - how we, using our eyes, understand the world around us," says Malik, who is also a researcher with the Center for Information Technology Research in the Interest of Society (CITRIS). "But it's not just that we need to recognize a tiger in front of us. We need to recognize that the tiger is jumping toward us."

To empower computers with that same capability, Malik and graduate students Alyosha Efros, Greg Mori and Alex Berg have developed a software system that instantly classifies the videotaped actions of a human figure in various settings, from the World Cup to a ballet recital. They've also used their system to create a video sequence where a World Cup player appears to mimic the motion of one of the graduate students showing off his soccer skills on videotape. The researchers will present their results in a scientific paper at the IEEE International Conference on Computer Vision in France next month.

Here's how the underlying technology works: The researchers provide the computer with a digitized video clip of, for example, a televised soccer game. Even though the players are quite small, the pattern of motion--a run toward the goal, for instance— is easily identifiable by the cyclic motions of the hands and legs.

The software captures that pattern and computes the "optical flow," the local movement of each pixel in the part of the image being analyzed. Those optical flow vectors are then compared to the data in a library of predetermined patterns, representing walks, jumps, ballet movements, and other human motions shot from a variety of angles. "The computer finds the best match and identifies the action," Malik says.

Given a video input sequence of frames depicting a soccer player (above), the software identifies the most closely matching frames in a database of 3D motion capture data (below). The motion capture data is rendered from many viewing directions using a stick figure. Image courtesy the researchers.

While the optical flow technique is ideal for wide shots where the action may be far away, the researchers also developed a related technique for close-ups. In this version, the computer detects the outlines of the person and, Malik says, "fits the equivalent of a human skeleton inside." The software then observes how the motion of the skeleton evolves over time. Because the computer can shift the skeleton in three-dimensions, the database does not need to contain multiple angles of the same movement. This technique, Malik says, could someday enhance computer speech recognition systems with "gesture recognition."

How do you think computer recognition of human motion will impact your neighborhood? We want to hear from you...

Of course, computer recognition of human motion has surveillance applications as well. According to Malik, speedier hardware could potentially enable his system to detect burglaries or crimes in progress. On the civilian side, he adds, it could also be used to monitor swimming pools, keeping a constant vigil for actions that are consistent with drowning.

More than surveillance though, Malik is particularly excited about how computational motion analysis and "Do As I Say" could be applied by the entertainment industry. For example, a videogame player could potentially control the motion of a human "character" using a joystick. Or, the researchers write in their scientific paper, a filmmaker might "collect a large database of, say, Charlie Chaplin footage and then be able to 'direct' him in a new movie."

Sara McMains, professor of Berkeley mechanical engineering Angela Privin photo

At an automobile manufacturing facility in Japan, a large computer-generated model of a sedan floats in space in front of a product manager's eyes. Holding a stylus in her hand and pressing a button at her fingertip, she begins to draw on the surface of the vehicle. As she traces the lines around the wheel wells, she feels resistance against the stylus corresponding to the curves of the steel. It's as if she's dragging a magic marker along the body of a real car. Simultaneously in Los Angeles, a car designer sees lines appearing on the same virtual vehicle and creates a digital post-it note, reminding him to reconsider whether the fender may be too close to the tire. This isn't a science fiction vision for the future of automobile design. It's the very real virtual reality research of Berkeley mechanical engineering professor Sara McMains.

"Instead of having to bring everyone who has a stake in a design together in one physical location, they can be anywhere and still talk about the particulars and provide valuable feedback during the design process," McMains says.

A student uses the Virtual Reality workstation running the CHaMUE system. (The automobile's 3D effect is simulated to depict what the CHaMUE user sees.) Image courtesy the researchers

McMains, graduate students Youngung Shon and Irena Nadjakova, and professor Carlo Séquin are already testing this novel system for collaborative design review at a distance. Users of their CHaMUE (Collaborative Haptic Mark-Up Environment) system don stereo glasses that provide a three-dimensional view of a computer-generated model displayed on a drafting table-like display screen. The glasses track the wearer's head so he or she can view the model's different angles simply by looking "around" the image. The user then grasps a stylus suspended from a force-feedback, or haptic, device called a PHANTOM. The PHANTOM and its accompanying software convert digital information about a virtual model into force feedback to provide the sensation of actually touching a real object, or in this case drawing on it. CHaMUE enables users at similar workstations anywhere in the world to interact with each other and the computer model via the Internet.

Professor Sara McMains and graduate student Youngung Shon demonstrate the CHaMUE system. Many undergraduate students also contributed to the project through the College of Engineering's Undergraduate Research Opportunities (URO) Program and Undergraduate Research Apprentice Program (URAP). David Pescovitz photo

While CHaMUE has applications across design disciplines, McMains points to the automobile industry as a driving force behind her work. Automobile companies are increasingly off-loading the manufacturing of various car parts to different suppliers. The problem is that it's difficult to share feedback throughout the design and manufacturing process when the interested parties may be thousands of miles away from each other.

Virtual collaborative development environments are not new, McMains points out. But traditionally, the models that are discussed in these virtual conference rooms are either two-dimensional, or complicated CAD designs, or both. These models are often difficult for product managers or other non-engineers to understand. That's why she's exploring whether haptics can make virtual collaborative environments more user friendly.

To determine whether haptics help the design process, the researchers had to develop a system that could display the complex models required by many design applications. The problem is compounded by the fact that in CHaMUE, the model is dynamically changed by the user's interactions. One approach the researchers are exploring is enabling the CHaMUE system to change the level of detail in the model based on the requirements of the task-at-hand, the bandwidth available to share designs online, and the speed of the hardware powering the workstation.

"In IEOR, the devices in the systems we work on are on the scale of factories, hospitals or airports," Professor Lee Schruben says. "Our units of measure are not Joules or Ohms, but social resources such as time and money."

When Berkeley professor Lee Schruben attended a conference celebrating the opening of Berkeley's new Department of Bioengineering, he was duly impressed. One after another, researchers highlighted new research on methods to treat myriad diseases that could someday save "millions of lives." But as much as Schruben, the chair of the Department of Industrial Engineering and Operations Research (IEOR), was impressed by the presentations, he was also concerned. A new drug to combat multiple sclerosis, for example, is only a lifesaver if it gets to the patients who need it at a cost they can afford. This is a problem, Schruben realized, that falls squarely in the IEOR domain.

"The question is how do you efficiently and cost-effectively mass produce high quantities of a high-quality product and get it into the supply chain," he says. "Those are traditional industrial engineering research topics. But in academia, very few people are applying IEOR methods to biotechnology."

To that end, Schruben and IEOR professors Robert Leachman and Philip Kaminsky are undertaking a UC Berkeley initiative in bioproduction, the way in which biopharmaceutical firms manufacture and distribute their wares. Schruben and Leachman are experts in semiconductor manufacturing while Kaminsky is a recognized authority in supply-chain management with experience in the pharmaceutical industry. Eventually, Schruben hopes the College will offer a certificate program or even a degree in the field. Kaminsky is spearheading an effort to co-host a National Science Foundation-sponsored bioproduction symposium later this year, the first step in Schruben's vision for a Berkeley-based international consortium of biopharmaceutical production researchers and manufacturers.

The IEOR Department's bioproduction research efforts have already begun through collaborations with scientists at the bio-pharma plants of Chiron Corporation and Bayer. The Berkeley researchers agree that biotechnology is at a similar crossroads as the semiconductor industry faced several decades ago. Amazing new products are being discovered so fast that the cost and quality issues associated with good production and distribution methodologies are now secondary concerns. When production ramps up to high volume though, those issues become critical.

"The issues of quality, production, materials, and supply-chain management that we have had great success with in semiconductor operations appear to be very similar to those in bioproduction," Schruben says. "But the biotech companies also have the Food and Drug Administration looking over their shoulders at not only product quality but their manufacturing methods. There are also some new and challenging quality control issues here."

Schruben points to a 2002 article in Fortune magazine detailing a hit Bayer took several years ago when FDA inspectors found major faults with the manufacturing methods the company was using to produce a certain hemophilia treatment. The necessary corrective steps drastically slowed production of the drug, leading to a rationing and a cloud of fear falling over the patients who counted on the drug for their very survival.

"Because we were very enamored of our science, we weren't necessarily paying attention to good manufacturing practice," a Bayer vice president was quoted as saying in the Fortune article.

Bayer of course bounced back, but the tale, Schruben says, proves that bioproduction is tricky business. The facilities grow their products from living cells, often using large bioreactors where protein drugs are fermented. Alternately, they employ a continuous-flow process where the products are drawn from smaller reactors over a period of months. In either case, as in microchip fabrication, even the tiniest contaminant can ruin an entire batch.

Do you think bioproduction research will lower health-care costs? We want to hear from you...

"Because you use one batch to make the next batch, a product may be well into its production process before you find out it's bad," Schruben explains. "So you need to take a risk, but you need to do it without going too far down the production process. In IEOR, we specialize in quantifying such production risk trade-offs."

The Berkeley researchers believe that they can make big improvements in this kind of quality assurance. Like integrated circuit manufacturing, the end products, in this case proteins, are not visible to the naked eye so all inspections must be done indirectly using high-tech metrology, or measurement, devices. Additionally, the researchers are exploring ways to improve the production schedule of bio-pharma facilities. The aim is to improve the plants' capabilities to efficiently manufacture several products using the same equipment while efficiently scheduling in necessary decontamination cycles.

Drawing from his experience in semiconductor manufacturing, Schruben's goal is to create a bio-pharma version of SEMATECH, the Austin, Texas-based consortium for the international semiconductor industry where he served as visiting Distinguished Professor in 1992. A bioproduction consortium would share best manufacturing practices while driving necessarily cross-disciplinary manufacturing and production research in all areas of the industry.

"In addition to the economic benefit similar to what SEMATECH provides to the semiconductor industry, this effort has a moral imperative," Schruben says. "It's the only way bio-pharma is really going to save millions of lives."

David N. Kennedy, California's "water czar," will deliver the keynote address at the "Celebrating Engineering Excellence" symposium on September 13, 2003. Peg Skorpinski photo

For 15 years, David N. Kennedy (CE BS' 59, MS '62) was known to some as California's "water czar." Beginning in 1983, Kennedy directed the State of California's Department of Water Resources (DWR), the organization that oversees the water needs of more than 30 million people. During the longest tenure as Director in DWR's history, Kennedy rode the ebbs, flows, and tsunamis of California's delicate water issues.

He will return to his alma mater September 13 to deliver the keynote address at the College of Engineering's "Celebrating Engineering Excellence" symposium and Distinguished Engineering Alumni Awards luncheon. Kennedy himself was a recipient of the award in 1997.

Kennedy's immersion in water began in earnest after graduate school when he took an engineering position in the DWR's statewide planning office, helping plan water development facilities throughout the State. From there, Kennedy relocated to the southern part of the state where he was involved in the planning, hydrology, and operations work of the Metropolitan Water District of Southern California. In 1974, he was promoted to the position of assistant general manager.

In 1983, Governor George Deukmejian brought Kennedy back to Sacramento, appointing him Director of the DWR. When Governor Pete Wilson took office in 1991, he announced that Kennedy would be one of the few hold-overs from the previous administration. Under Wilson's governorship, Kennedy managed a $900 million annual budget with 2,500 employees.

"Early in my career I realized that I enjoyed public service and being involved in public works," Kennedy says. "However, I never had a career plan as such. Each of the career changes happened in unforeseen ways and I never looked much beyond the particular job I was in."

As director of the DWR, Kennedy battled the impact of the 1987-1991 drought by organizing the state emergency drought water bank program, the first of its kind in the country. As the drought ended, Kennedy drafted Governor Pete Wilson's long-term plan for the State's water resources, a 10-point policy to ensure that California's water needs would be met in the future. He also completed SWP's Intake Pumping Plant and oversaw the construction of a Coastal Aqueduct to bring water to the parched San Luis Obispo and Santa Barbara counties.

David Kennedy, 3rd from left, receiving the Distinguished Alumni Award in 2001. Peg Skorpinski photo

Surprisingly, it was not the drought Kennedy found most challenging but the three major floods during his years as director of the DWR.

"Decisions about reservoir releases and levee repairs have to be made in real-time with incomplete information and many different things going on at once," Kennedy says. "Those were pretty hectic times."
In 1994, Kennedy was involved in negotiating the Monterey Agreement, a drastic modification to the SWP's contracts with long-term water contractors to enable the contractors to increase their water supply reliability. That same year, he helped devise the Delta Accord, designed to improve environmental protection for the area's wildlife while also tackling water supply problems.

When he was elected in 1998 to the prestigious National Academy of Engineering, Kennedy was lauded "for his ability to nurture consensus on challenging water issues, working cooperatively with legislators, water users, regulatory agencies, environmental and business groups to formulate and put into action sound water resource policies, programs and projects."

Kennedy retired in 1998 but remains professionally active on the board of the California Water Service Company, which provides through its subsidiaries water utility services to 1.7 million people in 99 California communities. Kennedy and his wife Barbara reside in Sacramento and have three children, one of whom continued the family's Berkeley lineage by earning his BS and MS degrees mechanical engineering at the College.

The College of Engineering hopes you'll join us in welcoming our esteemed alumnus back to campus.

UC Berkeley wireless sensors networks now monitoring the microclimates of California's giant redwoods. Original article: Downsizing Sensor Software (April 2002) http://www.coe.berkeley.edu/labnotes/0402/tinyos.html In August, tiny wireless sensors developed by researchers at UC Berkeley's Center for Information Technology Research in the Interest of Society (CITRIS) were deployed in redwood trees at the UC Botanical Garden to measure environmental variables like light, temperature, and humidity. The devices, called motes, run their own tiny operating system (TinyOS), developed by UC Berkeley computer science professor David Culler. TinyOS enables the sensors to form their own networks, bouncing bits of data from neighboring node to neighboring node until the information reaches its desired destination for processing. The Redwood effort is a collaboration between Culler, formerly the director of the Intel Research Laboratory at Berkeley, and UC Berkeley professor integrative biology Todd Dawson. "We'd like to better understand why the natural range of the coast redwoods is largely restricted to the fog belt," says Dawson. "We have a lot of questions, and these new micromotes are the solution to getting them answered." Previously, studying the redwood canopy involved lugging 30 pounds of gear up the massive trees using pulleys. "We worked with Todd's team to design a system that would generate trustworthy data and withstand the harsh environment in the forest, while making it easy to install many sensors in each tree," Culler says. "The network of sensors will provide a web of data for environmental scientists." Other members of the project team include Robert Szewczyk and Joe Polastre, UC Berkeley graduate students in electrical engineering and computer sciences, and Wei Hong and David Gay of the Intel Research Laboratory at Berkeley. The redwoods project was recently profiled on CNN.com and in the pages of the San Jose Mercury News and the Sacramento Bee. "Redwood go high tech: Researchers use wireless sensors to study California's state tree" by Sarah Yang, Media Relations

Anthony Ambrose (right) makes preparations to mount miniature wireless sensors Noah Berger photo