Asking Software to Recommend a Good Book
The New York Times, June 20, 1998
by Samuel G. Freedman
Clerking at a New Haven book shop one afternoon 15 years ago, a novelist named Kitty Florey watched two regular customers approach the display racks, their faces furrowed with consternation. "I get tired of these after a while," one said, indicating a stack of best sellers, "but I don't know what else to read." Ms. Florey bounded across the Oriental rugs to propose a few titles by Anne Tyler and Barbara Kingsolver. Her manager, just a pace behind, suggested Ms. Florey's own latest book.
On a recent morning in the TriBeCa offices of a software company called Muze, the same Kitty Florey typed into a computer the catalog information and plot summary of "The Man in the Box," Thomas Moran's novel about an Austrian family that shelters a Jewish doctor from the Nazis. Then she listed the book under such categories as "caring and caretaking," "fugitives" and "Holocaust survivors." All the material became part of Muze's data base, which some 300 bookstores offer to customers.
Ms. Florey's evolution parallels American society's: from an old-fashioned, personal brand of taste-making to a high-tech variety made possible by the advent of what is known as agent software. This software, sometimes described with the more specific term "collaborative filtering," compiles information that consumers provide about their taste in the arts and past buying patterns and analyzes the data to recommend books, videos, films and CDs. It is used by such major retailers as the online bookstore Amazon.com and the Columbia House music club. Microsoft recently paid $40 million to acquire the agent-software company Firefly, and Amazon.com just leased Muze's software for use in its new online music store.
As such developments demonstrate, artificial intelligence is offering an alternative to -- or even a replacement for -- flesh-and-blood arbiters of culture. The consumer who decades ago took cues from a critic like Edmund Wilson or a mass-audience magazine like the original Life can now click on Muze or Firefly.
"All of us feel a little uncomfortable," says Ms. Florey, the senior fiction editor at Muze, "because we all come from a background of studying literature and believing great books are beyond categorization. At the same time, you feel like you're opening it up to people who wouldn't go down that path otherwise."
Ms. Florey's own divided feelings frame the larger debate. For the creators and buyers of agent software, it gives consumers a matchless tool for navigating through an overwhelming number of choices to find art they will cherish. For critics, including those conversant with computer technology, collaborative-filtering software threatens to take the idiosyncratic process of forming an esthetic and replace it with the soulless logic of code.
"People are losing their ability to really reason for themselves, to look at things for themselves," says Patricia Galvis Assmus, a fine-arts professor who directs the Center for Research in Art and Technology at the University of Massachusetts. "In one sense, collaborative filtering is not that different from a bibliography at the end of a scholarly report. And that's fine. The problem is that people rely on it solely and don't take the initiative beyond the software. In this country, people are so enthused with technology that browsing a library just isn't as attractive."
Assmus' vision does not persuade fellow scholars like David Thornburn, a professor of literature and director of the Communications Forum seminar series at the Massachusetts Institute of Technology. He has been pleased to click onto Amazon.com's Web site and find its Netperception software presenting him with book titles based on those he most recently bought.
"I don't feel as if cultural value is threatened by this software," Thornburn says. "It seems not to have a judging function but a consumer-advice function. I had bought 'Cold Mountain' for a gift, and the software was intelligent enough not to offer me every other best seller. It didn't recommend Tom Clancy. It recommended 'The God of Small Things,"' also a first novel of literary distinction."
"It made that distinction," he added. "It was accurate about the things I enjoy reading."
Agent software arose in the late 1980s from a mixture of commercial and cultural motives. John Hey, later one of the founders of the software company Likeminds, started out as an MIT student and movie buff who wanted to be steered to films without having to learn plot details from reviews or conversations. Hey invented two forms of collaborative-filtering software, installed them in kiosks and sold the kiosks to video stores in the Cambridge area. The store's customers put in their ratings of films they rented, forming a reservoir of data that could be matched to the personal preferences entered by any user.
Muze developed a different approach. It began by leasing and digitizing printed catalogs of music and books. Then it hired people like Ms. Florey to cross-reference the entries by themes, categories, genres and subgenres. These range from such conventional terms as literary/avant-garde to those Muze itself has coined like galactic empires and space opera. Lately, the Muze staff has passionately debated whether to add the designation angst bunny for what Ms. Florey described as "young women in the Village with black clothes and lots of body piercings."
Such discussions underscore the fact that the intelligence behind the software is not artificial; the raw material was provided by human beings, whether customers typing in their book preferences or editors in computer companies deciding how to categorize a particular book. What the software does, in computer jargon, is "find patterns in the noise" -- use mathematical formulas to determine which books or films or CD's had the greatest probability of matching either a consumer's stated preferences or actual record of purchases. Although not all retailers choose to do so, they can have the software adjusted to put extra emphasis on particular titles.
In whatever variation, collaborative-filtering software surged with the refinement of the Internet and the World Wide Web in the early 1990s. Likeminds, for instance, went on line for such clients as Columbia House, the cable network Cinemax and the Web site Moviecritic.com. What customers got was convenience in choosing a product; what retailers got was an identity and a marketing profile of each buyer.
Technological innovation, though, tells only half the story of the agent-software bonanza. The software entered the cultural marketplace just as the power of highbrow critics, general interest magazines and television networks unchallenged by cable was dissipating.
"One of the reasons for the mandarins and the shapers of opinion in the past is that there were limitations on the availability of information to people," says Stephen Kanzler, the president and chief executive officer of Likeminds. "Fifty years ago, even 20 years ago, how many people actually got to go to the Louvre or the Metropolitan Museum of Art? So if there was an opinion needed on a work of art or a book or a cultural trend, we were dependent on the people who had access.
"Now the information is ubiquitous. You can look at the Louvre collection on the Net. The sifting of information and opinion is the problem. It's the old drinking-from-the-fire-hose problem. How do you sort through it? An agent can help a consumer get what he wants."
Ms. Florey of Muze speaks more bluntly. Speaking of the New Haven bookstore, she says: "At a place like the Foundry, we didn't need machines. The guy who owned it prided himself on knowing. At the superstores, there's nobody you can ask. They don't have booksellers who make their career out of loving books."
But precisely because agent software intends to cater to personal taste as a commercial strategy -- Likeminds' slogan is "Every individual a market" -- scholars like Carol Gigliotti of Ohio State University criticize it.
"If you look at the Firefly Web site, you'll see this phrase, 'loyal and dynamic communities built on shared interest,"' says Gigliotti, a professor in the Advanced Computing Center for the Arts and Design and the project director for an online journal about computer ethics. "What these dynamic communities are about is brand loyalty. They're not about an idea, a value, a spiritual concept. They're about the market economy."
In cyberspace, where the prevailing political philosophy seems to be libertarian, the notion of a vast bazaar of information is populist. Even as a literary critic himself, Thornburn of MIT contends that the pre-eminent cultural critics of mid-century America performed a disservice by creating "a false dichotomy between high culture and popular culture," stranding art forms like jazz. Agent software, in comparison, weights any and every opinion equally.
Which is part of what disturbs Dr. Gigliotti. "There's a big difference between knowledge and wisdom," she says. "And a great deal of what's on the Web has to do with information, which is confused with knowledge, which is confused with wisdom. Wisdom can understand knowledge and decipher information. We're allowing a very random, nonexperiential machine to say what we should like."
As the debate proceeds, Kitty Florey writes novels by night and enters data by day, her office shelves crowded with the London Review of Books, the Threepenny Review and a cheese sandwich on whole wheat. A few carrels over works Molly Siemers, the genre-fiction editor. As an English major at Vassar, Ms. Siemers wrote a thesis on the author Djuna Barnes' refusal to be categorized for her blackness or lesbianism.
On occasion, Ms. Florey and Ms. Siemers have paused in their categorizing duties to determine which books carried the most cross-references. The winner, as well as they can reckon, is "Tom Jones," without about 150. Proust also ranks high, as does "Middlemarch." And there are several unexpected competitors.
"I've given my own books dozens of themes," Ms. Florey says. "I'm shameless."