The Promise of the Daily Me
From My News to digital butlers: An in-depth look at the different flavors of personalization
This in-depth report appeared Aug. 2, 2001, in the Online Journalism Review. Here’s the version on the OJR site. It was republished in the Law Library Resource Xchange. An earlier version was commissioned for inclusion in an online journalism textbook by McGraw-Hill.
For a look at the latest developments in personalization at media outlets, see the companion column, The second coming of personalized news.
By J.D. Lasica
No trend threatens the guardians of old media more than personalization. The very notion challenges the philosophical underpinnings of traditional media: We, the gatekeepers, gather the news and tell you what’s important. Under this chiseled-in-stone setup, editors sort through and rank the news, controlling everything from the assignment of stories to their tone, slant and prominence on the page.
Personalized news reduces the role of editors in the news equation. The reporter writes the story, the copy editor (if there is one) edits it, another person indexes it for easy retrieval, and the user decides what’s important.
Personalized news tips the balance of power toward the news consumer. If I have breast cancer, I may want to read not only your medical writer’s story on new research developments but reports from other news services, too. If I’m a walnut farmer, I want access to all the agricultural news from the wire services that doesn’t make it into my hometown paper or onto its Web site. If I’m a restaurateur, a competitor’s plans to open a business down the street is major news to me, far more important than the latest doings inside the Washington Beltway.
True personalization, in short, augurs a revolutionary shift in the balance of power between news provider and news consumer. Traditional assumptions about who gathers the news and who consumes it go out the window. Journalists now entering the field may be collecting, processing and disseminating the news in completely novel ways. How we think about news itself may be transformed, from a solid if predictably plodding product, scripted by a professional priesthood for a mass audience, to a more free-flowing fount of information that serves the individual needs of consumers.
The challenge facing the next generation of journalists is to find a way to hold onto the time-honored values of traditional journalism — accuracy, reliability, fairness, accountability — while embracing the changes in their craft that elevate the reader. If they accept the new realities dictated by the Internet, the new journalists will be light years ahead of publishers and old-guard editors who continue to think within the box of mass media. Mass media are about reaching large audiences and target demographics. Mass media control the content, form and distribution of the message. Mass media serve each person’s general interests while serving no individual’s specific needs.
The Internet, however, is not a mass medium. It’s a medium for the masses. The Net blows away the top-down, one-to-many model that governs old media. Instead, it encompasses one-to-one, one-to-many and many-to-many communication, with the individual firmly rooted at center stage. On the Internet, control of the content, form and distribution of the message passes back and forth between publisher, user and other participants. The user may adopt the mantle of reporter, editor and publisher, creating new forms of individualized content.
Personalization is not just a cool feature of new media — it’s intrinsic to new media.
Personalization is not just a cool feature of new media — it’s intrinsic to new media. Unlike radio, television or print, the Internet is the only medium that is inherently personalizable. Users can be reached simultaneously with one-of-a-kind messages. The old formula of editors and news directors having the lone say in determining what’s important has become an anachronism in cyberspace. The user, after all, is in the best position to know what he or she finds most interesting, valuable, useful — or newsworthy.
For print and broadcast media — even those with online presences — that’s a difficult proposition to accept, given the historical baggage of top-down media.
“When these players take their businesses to the Web, they bring along the mind set of a medium that delivers content to a passive audience. They do the sifting and choosing, and they give the customer what they hope is of interest to the readers,” observed Henry Sohn, vice president and general manager for network services at Yahoo! “The Web requires a different mind set. It’s an interactive medium, and everything else is secondary to that fundamental principle.”
Where does this leave the online journalist? In a role that’s changing — but more critical than ever.
Personalization does not mean that journalists should abandon their role of sorting, filtering, prioritizing and making sense of the news. But it does mean that users need to be brought into the process in a direct and meaningful way.
People want broad-brushstroke news to find out what’s taking place in the world at large, but they also want narrowcast news to find out what’s going on in their own world: the latest news in their specific field, news affecting their company, breaking news about people or sports teams important to them, local and community news (assuming that someone has done the reporting). News operations that share power and influence with individual users will be the big winners in the new information economy. Companies that cling to a one-way gatekeeper mindset will become increasingly marginalized.
Personalization is a slippery concept to get our arms around. As Sohn of Yahoo! pointed out, “Just clicking on a hyperlink is personalization — you’re deciding where you want to go.” Under that expansive view, every time you read a newspaper and toss the business section aside for the world news pages, you’re engaging in a personalized news experience. But it’s also a limiting experience: You can read only those stories that a team of news professionals has selected.
True personalization requires an extra step: a recurring set of interactions between news provider and news consumer that permits you to tailor the news to your specific interests. Imagine a publication made up entirely of articles of special interest to you: stories about your hometown, your college, field of study, hobbies and interests, favorite bands, TV shows and sports teams, along with coupons and discounts for all the stuff you need to buy.
Call it the Daily Me.
Is such an endlessly fascinating publication really possible? It is, although the early releases suggest that its full realization remains many years away.
An early experiment: fishWrap
Many of the current working versions of personalization can be traced to a class project by eight freshmen at the Massachusetts Institute of Technology’s Media Lab in late 1992. “Our class assignment was to design a personalized news system for MIT’s incoming freshmen,” recalled one of the students, Brad Bartley, now a programmer at a high-tech firm. “To make them less homesick, we provided content geared to their specific backgrounds.”
For Bartley, that meant getting news updates about his hometown of Quapaw, Oklahoma, population 985 — or, at any rate, from his home state, given that neither Quapaw nor Oklahoma turned up with any regularity in the Boston media. To solve that info gap, Bartley, his fellow students and researcher Pascal Chesnais developed a customized personal news service called fishWrap.
Within a short time, the experiment spread from the freshmen class to the entire MIT community. It worked like this: A computer program asked participants for their hometown zip code, academic interests and personal interests. From that profile, the program looked for news from your town, region and state. Somewhat more clunkily, it matched keywords in your profile against news stories coming in from the Associated Press, Reuters, the Knight Ridder chain, Boston’s newspapers, foreign wires and other news providers. Then it bundled it all into a nice customized package and dropped it on each subscriber’s digital doorstep each morning. After a reader scanned a summary of stories in each news category, he could summon up the full text with accompanying graphics or audio on his computer screen.
With fishWrap, the community — not editors — decided which stories appeared at the top of the front page based on how many readers had called up the story.
FishWrap allowed subscribers to change their preferences or add new topics to their personal choices. Bartley, for example, built a personal page that zeroed in on computer technology, architecture, book reviews and photo essays. The program also detected changes in reading habits: It would pay attention to the kinds of news stories a person chose or sports teams she followed, and those items became more prominent on subsequent viewings. Finally, fishWrap ranked the order of the “page one” stories each day based on how many of the 700 or so subscribers had read the article, so that the most popular stories appeared at the top of the page. “No editors made those decisions,” Bartley said. “The community served as editor.”
Of the approximately 15 stories on fishWrap’s page one each day, a third were similar to what you would see on the front page of a newspaper, a third were technology-related (this was, after all, MIT), and a third were stories that the mainstream media generally didn’t consider newsworthy: coverage of rock music, movies, youth culture and other matters of special interest to college students.
Jack Driscoll, former editor of the Boston Globe and now a visiting scholar at the Media Lab, recalled the disdain with which the journalism establishment greeted fishWrap and similar personalized news projects, such as Create Your Own Newspaper (CRAYON), an online tool created by two students at Bucknell University. “Some naysayers in my profession didn’t like the whole concept, suggesting that it would turn readers into self-absorbed myopes. But I always argued that this provided a valuable service.”
Critics of the Daily Me have faulted personalized news services for filtering out news that’s important to us as members of the local, national or global community. The more personalized the news became, they feared, the more isolated we would become from one another. I beg to disagree. We live in a world where it’s impossible to become closed off from the world at large. News assaults us from all directions. What we can do, however, is build constructs to make sure the news I care about most deeply breaks through the noise and grabs my attention.
For the most part, personalized news is not about filtering out; it’s about funneling in. It’s about broadening the reader’s content choices and navigation options so that she gets the news she needs and cares about.
Different kinds of personalization
We can think of personalization in several ways:
• Personalized news content: The most common use of the term personalization refers to filters that give us control over the content selection process. By whatever means — a Web site that remembers who you are, a bot that trolls the Internet at your command — you have a greater say over the kinds of news, headlines and information that come streaming into your life. Think of this as the “what” of personalization.
• Personalized news experience: If personalized news selection is the main meal, personalized news experience is the tablecloth, candlelight and violin music. Users decide on the setting of their news experience, the frequency, and the method. We might access our personalized information via e-mail, pop-up screen alerts, pager, cell phone, mobile palm device, or Web page whose appearance we can alter. Think of this as the “where,” “when” and “how” of personalization.
• Personalized news services: Service journalism shines when it hands us tools to make the news more personally relevant and when it simplifies our lives. Personalized services might be about connecting workers, sharing products, or solving a chore or project. Think of this as the next stage of personalization, where an individual’s personal needs or work tasks are met, sometimes for a price.
All three of these categories overlap. What they all have in common, however, is an inherent bias toward empowering the user. Let’s look closer at these different approaches to personalization.
Personalized news content
We swim in data. Media have brought us such an abundance of information sources that we now face the problem of information overload. How to sort through the info glut that comes cascading into our lives every day? Media and technology companies have tackled the problem in widely varying and imaginative ways.
One spectacular early failure was PointCast, a company emblematic of high-tech publishing — and of the Internet gods’ fickle natures. The Silicon Valley company burst into prominence in May 1996 with the release of its personalized-news software. Within a year, more than 2 million people had downloaded the free computer screen saver. When a user dawdled too long between mouse clicks or chose the PointCast icon intentionally, the program took over his screen, turning it into a whirling kaleidoscope of personalized news, sports, weather, stock quotes and company information (not to mention animated ads), much of it delivered to his computer’s hard drive in real time if he was online.
But PointCast overreached itself, and within a year the magic had vanished. Though it had brought aboard a stellar list of content partners — including The New York Times, Los Angeles Times, Knight Ridder, CNN and Time magazine — the PointCast Network billed itself as “the first broadcast network on the Internet” at a time when the Internet’s broadband pipeline was woefully thin. Many users (including me) pulled the plug, fed up with its ceaseless appetite for bandwidth as well as the intrusive, nonstop advertising; others became disenchanted with the lack of specificity in the program’s broadly defined categories. In the end, usage plunged and the company, which had spurned a $500 million takeover offer by the News Corp. in 1997, ultimately sold at the fire sale price of $7 million two years later.
While PointCast failed to deliver the goods in its execution, its early success suggested that users hungered for news and information tailored to their individual needs. The search engine portals saw an opportunity and stepped into the breach.
The portals get personal
Following close on the heels of PointCast, Web portals such as My Yahoo, My Excite, My Netscape, My Lycos and My Snap (now My NBCi) launched personalization services. Each portal offered slightly different variations on the same pick-and-click theme: Users could choose favorite news topics, stocks, sports teams, horoscopes, TV listings, movie show times, lottery results and other interests. Many of the services also featured message boards, chat rooms, daily health tips, favorite Web sites, travel alerts and convenient reminders of friends’ birthdays or relatives’ anniversaries. Millions of us now use these personalized pages, often as launch pads for our online forays every morning.
To get their personalized profiles, users fill out online questionnaires, checking off subjects of interest — for example, political news, human-interest stories, romance movies — and entering more specific terms such as parasailing or telephony. The services then go to work, trolling for the requested news and information from their databases, their partners’ news feeds and the Web. Users can easily update their profiles and customize their choice of news providers.
Such services have had some measure of success: Mention personalization today and most people think of My Yahoo or My Excite. But only 5 to 10 percent of users at the portals register for the My services, perhaps because these first-generation tools only scratch the surface of personalization. Invariably, they rely on the same predictable wire services for general interest news. They require a tedious form of checkbox personalization that drives away most users. And they fall short of the promise, resulting in customization for the masses but little in the way of true one-to-one personalization.
Web news publications join the fray
A few online news organizations have followed the lead of the portals — but only a few. The Christian Science Monitor, Wall Street Journal, Los Angeles Times, MSNBC and CNN are among those that have offered personalized news in some form. A few others, like the Washington Post, are just now arriving at the party. (See The Second Coming of Personalized News.)
The Wall Street Journal‘s Personal Journal lets users customize to see certain features of its online edition every day as well as tracking specific topics. Neil Budde, publisher of WSJ.com, said, “The most popular thing on our site is the personal portfolio, which is a level of personalization that ties into the news with a level of granularity that’s meaningful to people.” A user can track how her stocks are performing throughout the day, and the Journal flags any story that mentions a company in her portfolio.
The Christian Science Monitor‘s Monitor Extra, which also requires a subscription, lets users select from customized news categories including art, religion, higher education, immigration, careers, media, education, gardening, book reviews, music, food, animals and geographic regions such as China, Mexico and the Middle East. In most cases, a staff member assigns each story that streams into the newsroom to a corresponding news category. “It’ll be a while before we trust the software to figure this out for us,” said associate editor Tom Regan.
Regan said the paper’s personalization efforts were modest but growing. “We’ve got to start reaching users who have time to read only a few high-value stories a day. Personalization lets you zero in on the news that matters most to you.”
The online operations of ABC, CBS and NBC have all taken a small step toward personalization by letting users choose a local TV affiliate. Plunk in your zip code and, voila!, up pops local news, weather and sports. Trouble is, the affiliates’ news feeds are basically shovelware and rarely contain detailed reporting.
‘Search-term searches are notoriously flaky, and users want to search the news in a baffling number of ways.’ – Travis F. Smith
Why haven’t online news operations made greater use of personalization? For a variety of reasons, including an unproven business model. But another obstacle is a technological one: The simple truth is that, financial news aside, most news doesn’t sort well.
Travis F. Smith, who helped set up the Los Angeles Times’ trailblazing personal search agent, Hunter, in the mid-1990s, put it this way in an e-mail interview: “Search-term searches are notoriously flaky, and users want to search the news in a baffling number of ways: by date, by major subject, by author, by type of story (opinions, letters, obituaries, reviews, profiles), by geography, and by topic area (crime stories, consumer safety, high tech, women’s issues).”
None of the current models fulfills personalization’s ultimate promise to deliver the news that’s most meaningful to each individual consumer.
“A lot of the efforts we’re seeing are brute-force customization at a base level,” said Daniel Farber, editor in chief of ZDNet, an online technology news company. “Most sites are providing customization to subsets of the population rather than true personalization targeted to individuals’ tastes and interests. If you’re doing true personalization of a higher order, you need much more intelligence involved and you need to be looking at a much larger data set of information provided by each user.”
That’s a formidable challenge. One answer touted by proponents as “The Next Big Thing” — and derided by detractors as “The Next Big Bust” — involves intelligent agents.
Bots and intelligent agents
Imagine this: a digital butler that roams the Internet, intuitively knowing your likes and dislikes, retrieving perfect strands of news and information that you never would have discovered through old-fashioned surfing. That’s the holy grail of personalization. At this point, it’s still a fantasy.
But give the tech-heads points for imagination, if not execution. Bots, also called intelligent agents or personal profile agents, are poised to help you find a job, download software, follow your stocks, search for bargain air fares, bid for auction items, grab image files or audio news clips and perform other feats of digital magic. Over time, your bot could be trained to learn that you love “Ally McBeal” and the Green Bay Packers, that you need the stock price of Intel every morning, and that Sheryl Crow bores you to tears.
Advocates such as Marcus Zillman, who founded and sold BotSpot, a Web site that tracks dozens of kinds of Internet bots, predict that these electronic serfs will replace both online newspapers and portals as the primary source of users’ online news in just a few years.
“For journalists in the very near future, people won’t be going to portals or online papers to get their news. They’ll be using bots,” Zillman said. “News in the future may be very different from what you’re seeing in the daily newspaper and on your local TV stations.”
‘For journalists in the very near future, people won’t be going to portals or online papers to get their news. They’ll be using bots.’ – Marcus Zillman
Perhaps so. But it begs the question of where the bots will find this wealth of relevant, fact-checked news if journalists aren’t part of the equation. More likely, in my view, is a landscape where bots play a growing role as a precision tool that helps people assemble personalized, highly targeted packages of news and information. That’s hardly cause for alarm inside the newsrooms of America. If anything, bot programmers will likely try to strike deals with new media managers to tap into the rich wealth of content processed by a daily newsroom.
In the Net’s early days, the most popular bots were shopping comparison bots that fetched the best price on consumer products such as CDs, software and games. Shortly after the online auction house eBay was founded in 1995, people used bidding bots that haggled to get the best price in a virtual transaction. Other bots obediently zip through cyberspace looking for any mention of your name, school, company, hometown — any term you designate. These bots have pretty much taken over the function of the old-time clipping services that send notices by mail whenever an individual’s name appears in a print publication.
Bots are commonly thought to search the entire Internet, but most of them troll only the major search engines or select Usenet newsgroups. The next generation of bots, proponents say, will be smarter, slicker and more comprehensive. While some bots grab entire stories, most retrieve only a synopsis or a few introductory sentences and provide a link for the user to access the full story.
Free-lance journalists ought to fare well in a bot-populated world. One can easily imagine automated bidding software programmed to scour the Net for articles on niche subjects. “People with bots will discover stories you’ve written,” Zillman said. “Their bots will strike a transaction with yours, allowing the bot’s owner to access your article for, say, a dollar or two.” Multiply that by hundreds or thousands of users and it can turn into a nice chunk of change.
That’s still a few years away. Today, bots may be good at finding things, but they’re lousy communicators. Experts and business interests have begun hammering out an agent communication language as well as developing tags or conventions that will help categorize content — a sort of digital Dewey decimal system.
Once they’ve licked the translation problems, though, there remains a larger problem: Bots tend to be dumb as a stick. Ask a bot for stories about mountain biking and you’ll get stories about the Himalayas thrown into the mix. Ask for an image of President Bush and you might get pictures of leafy, green plants.
“It’s easy enough to say, ‘Give me every wire service story that mentions the words Taiwan and semiconductors,'” said Andrew Leonard, author of the book “Bots” and a reporter for Salon. “Then you get fifty stories in your mailbox and you don’t read any of them. What you want is the one really good story about Taiwan and semiconductors, and Natural Language Processing technology just isn’t good enough yet for a bot to do that kind of selection without human help.”
One obstacle is that bots can’t yet distinguish between quality journalism and dubious sources of information. “The bots may work, but most people will choose to go to trusted names and sources of information,” said The Wall Street Journal’s Neil Budde. “So if you train it to fetch the news from these five reliable publications every day, I suppose that would work. But people like to read things in context, and I’m not sure they’ll like a news experience that blends Wall Street Journal articles with articles from other sources.”
Will we soon enter a golden age of news retrieval? John Funk, founder of the personalized e-mail service Infobeat, has his doubts. “I’ve been reading about bots for 15 years. The reality is, people are lazy, and training a bot is very hard. If the definition of news is something you didn’t know yesterday, how do you tell a bot to fetch you something you don’t know about?”
A first-time user at an art Web site might browse its collection of pop art. After two or three more visits, the site can predict with a fair degree of accuracy what other galleries he might like after comparing his behavior to the browsing habits of previous visitors. A message pops up on his screen: “Visitors with tastes similar to yours have enjoyed our abstract expressionism collection. Click here to visit our Jackson Pollock gallery.”
Creepy? Or cool? Depends on your point of view. “Personalization on this scale provides users with a higher level of personal service,” said Steve Larsen, founder of the annual Personalization Summit and head of the consulting firm Personalization.com. “A site that collects and stores personal profiles is treating you as an audience of one, not as part of an anonymous mass audience.”
Collaborative filtering works like this: You visit a site and fill out a form listing films you liked or hated. The site’s recommendation engine then compares your responses to, say, 20 other people who gave similar answers. Then it instantly suggests other titles based on movies they gave a thumbs-up to. No Roger Eberts, no expert film critics, just the shared experiences of an intelligent online collective.
Welcome to the Borg.
The entire enterprise cuts against our fiercely individualistic culture, but consider: Don’t we invoke this same process whenever we ask a trusted friend whether she liked a new movie? If you work in an office or spend a lot of time online, don’t you circulate the URLs of articles you find provocative or amusing? Collaborative filtering automates the word-of-mouth process, finding people who like the same kinds of movies, CDs, books — and news — that we do.
Sometimes, the results are laughably wrong; mathematical formulas are only as good as the laws of probability. But more often than not, the recommendations can be surprisingly dead on, and the more you use the tool, the more accurate the suggestions become.
Web sites such as Amazon and CDnow have used collaborative filtering technologies almost from day one, personalizing the book- and CD-buying process for millions of users. With enough technical resources, a CD site could take on a hipper look if it knew that a teenager was visiting; a jazz fan would find racks of Sonny Rollins or Roy Hargrove awaiting him, not Moby or Garth Brooks.
To date, no online news sites have put collaborative filtering to the test. After all, isn’t it the job of editors to recommend stories? But possibilities loom:
• “I can see a button at the bottom of a Web page that said, ‘People who read this column also seemed interested in this editorial or feature story,’ ” said Budde of WSJ.com.
• Publications like The New York Times Book Review enjoy a built-in community ripe for personalized recommendations. If you like novels by Tom Wolfe, short stories by Joyce Carol Oates, thrillers by John LeCarré and nonfiction works by John McPhee, chances are that a recommendation engine — based on the collective wisdom of Book Review readers and writers who’ve shared their likes and dislikes — could make winning suggestions far better than the Times’ own book critics.
• Larsen sees the day when a user can designate his or her own personalized circle of trusted advisers — sort of a daily What We’re Reading — to see which two or three online articles or news stories especially caught their fancy. It’s not implausible that we might be peeking over the shoulders of a Michael Kinsley, Maureen Dowd or Camille Paglia one day.
Collaborative filtering holds out great promise for personalized news and information. All that remains is for someone to figure out how to harness that potential.
NEXT PAGE | Personalized news experience & services