Graduate student brings books to life

With just a zap of a handheld computer, oral histories can now talk and move, thanks to the work of a Berkeley engineering doctoral student.

Doctoral student Scott Klemmer synchronized video and text to create a multimedia experience that adds a compelling, personal dimension to reading oral histories. "Building technology based on what people do is a huge improvement over ignoring people," he says. ANGELA PRIVIN PHOTO

Scott Klemmer’s Books with Voices is an interface that uses an enhanced personal digital assistant (PDA) and barcodes to link transcripts with corresponding video from an historical interview. Klemmer and his collaborators will present the project this spring at the 2003 Conference on Human Factors in Computing Systems, the largest meeting worldwide on human-computer interaction.

A Ph.D. student in EECS, Klemmer worked with the prestigious Berkeley Regional Oral History Office (ROHO) and office technology company Ricoh Innovations to develop the system. Books with Voices introduces into the reading experience the primary source materials of oral history — the videotaped subject interviews — which until now might sit gathering dust on a library shelf.

"Predictions of a paperless office in the information age didn’t happen, and more paper is generated now than ever before," says James Landay, EECS professor and Klemmer’s faculty advisor. Landay also co-directs the Group for User Interface Research (GUIR).

Here’s how Book with Voices works: The printed transcript is tagged with barcodes, like UPC codes on grocery store products, which correspond to the original videotape of the subject interview. Using the PDA — which is augmented with a barcode reader and a tiny 2-gigabyte hard drive — the reader clicks to scan the code beside a passage and, within seconds, the corresponding video clip plays on its screen.

The Books with Voices PDA is used to scan a barcode in the oral history text and display the corresponding video with a single click. Users tend to reference the video component to get a sense of the subject’s personality or intonation. PHOTO COURTESY OF SCOTT KLEMMER

Books with Voices and a companion project, The Designers’ Outpost, inspired Klemmer’s dissertation, Papier-Mâché, a toolkit for building tangible interfaces to link physical and electronic media.

"The paper-saturated office," Klemmer writes, "is not a failing of digital technology; it is a validation of our expertise with the physical world." He describes himself as a technologist working with user-centered design research methods that focus on the needs of a particular community.

In his research for Books with Voices, Klemmer picked the brains of ROHO’s professional oral historians and took training in the field. He conducted oral histories with computer sciences professors David Patterson and Carlo Séquin and generated Books with Voices transcripts. To get an idea of how it works, see these oral histories at http://guir.berkeley.edu/oral-history/.

Through feasibility testing with 13 people experienced in oral history or computer technology, Klemmer found that readers accessed the video an average of 10 times per hour. "It’s typical to access the video in the beginning to get a sense of the subject’s character," he says. Users also consulted the visual component for details about the subject’s facial expression or intonation.

Jamey Graham and Gregory Wolff of Ricoh spent a year researching the barcode integration technology for the project. As ROHO moves from analog to digital transcription for its oral histories, Books with Voices will be incorporated into the archive.

David Pescovitz, editor of the College’s online publication Lab Notes, and Angela Privin, editor of Engineering News, contributed to this story.

FOREFRONT takes you into the labs, classrooms, and lives of professors, students, and alumni for an intimate look at the innovative research, teaching, and campus life that defines the College of Engineering at the University of California, Berkeley.

Published three times a year by the Engineering Public Affairs Office. Have a comment about Forefront? E-mail your letter to the editor. Click here to learn more about the magazine.