OTH search engines article

Searching for Bobby Fisher
(Or Anything Else, For That Matter):

A Cry for Intelligent Search Agents in a Cyberspatial Morass

Andy Carvin
Corporation for Public Broadcasting

(Note: This article is to appear in On The Horizon. Copyright 1996 Jossey Bass Publishing. May not be copied and/or retransmitted without the the expressed permission of Jossey Bass.)

When I first started using the World Wide Web in the spring of 1993, the whole notion of 'Searching the Web' had yet to make much sense. Back then, the total number of Websites was limited to several score at best, and since a significant number of them offered hypertext links to each other, it was easy to get caught in an online loop, wandering around in circles without ever bumping into anything new.

In the scant three years since then, the web community has exploded, having expanded exponentially to no less than 12 million known individual web pages. Whatever your interest, fancy, or obsession might be, chances are you'd be able to find something related to it on the Web.

Of course, the major dilemma that all WWW users must face is how to track down the right information in the first place. Currently, the most popular way to seek out online content is through what is known as a search engine. A search engine essentially is a computer program that a Web user accesses remotely by way of a special web site. For example, one of the largest search engines, Lycos (http://www.lycos.com), offers a small field on your screen in which you can type a word or a phrase. You then tell the computer to 'search' for that word or phrase, and the search engine responds accordingly by giving you a collection of links to pages that happen to contain the information requested.

In theory, it's a great service. Tens of thousands of Internauts access Lycos each day to track down everything from Unix tutorials to amateur erotica. Unfortunately, search engines are plagued by an inherent flaw: they're not very smart. For instance, let's say I'm a junior high student writing a research paper on dolphins and I decide to access a search engine like Lycos to help find online resources. After inputing the word dolphin, Lycos responds by presenting me with 7,351 references to marine biology pages, homepages of kids who like dolphins, football fanatics from Miami, recipes for Mahi Mahi, and Flipper fan clubs.

To Lycos, they're all the same thing: information related to the word 'dolphin,' with no delineation of context whatsoever. And I can't get always recognize what each page is about unless I search through descriptions of each page, site by site. As a junior high student with limited time and slow Internet access to boot, 7,351 references is a bit more than I'd be willing to deal with.

On the other hand, I could also search for 'dolphin' at a site like Yahoo (http://www.yahoo.com). Though it's often referred to as a search engine, Yahoo is really an enormous hierarchical catalog of thousands of content-infused websites. It doesn't have the same gargantuan breadth as a site like Lycos, but Yahoo arranges its data in special subject areas so you always know what kind of Web page you're going to. So, as I look for dolphin at Yahoo, I get an annotated list of 53 websites, as well as two general indices, one for the marine mammal, the other for the football team. These annotations allow me to go directly to the known dolphin sites that are apropos to my work.

But again, the number of sites I find through Yahoo can be limited. Since Yahoo is a relatively smaller catalogue of web resources, for all I know there might be some great new dolphin sites that have yet to be archived - which means, unfortunately, I could only stumble on to them with the assistance of a general search engine like Lycos.

It's a Catch-22, to say the least. Currently, there are at no less than half a dozen large search engines available, including Yahoo and Lycos, as well as AltaVista (http://altavista.digital.com), Excite (http://www.excite.com) Inktomi (http://inktomi.berkeley.edu), Open Text (http://www.opentext.com) and several others. But as of yet, none of these sites successfully combines the sheer size of Lycos with the user-savvy organization of Yahoo. Perhaps more importantly, none of them is very adept at informing the user if any of the websites referred to are out-of-date, obsolete, or patently false. Just because it exists doesn't mean it's a worthy resource.

With hundreds of new websites coming online each day, it's becoming ever more imperative that users have access to smart search technology that accounts for individual needs, as well as accurate accountings of what content is available online. Inspiration for an intelligent search engine might come from an unusual source.

For instance, one might look at a free online service called Firefly (http://www.agents-inc.com/), formerly known as both HOMR and Ringo. With Firefly, users can learn more about certain bands or musicians by telling the computer about their personal taste in music. Firefly contains descriptions of hundreds of musical groups and CDs, from Martinu to Muddy Waters to Metallica. Users rate music on a scale of one to seven, thus giving Firefly a 'sense' of your musical taste. With this information, Firefly will recommend other music to you, which you can sample and even purchase online. And you can rate the music it recommends to you, which in turn increases Firefly's knowledge of your personal interests. In other words, the more you use it, the smarter it gets.

As for the next step in search engines, it would be most intriguing to see someone apply Firefly's intelligent agents to a searchable, annotated web directory. Not only would it let you search for information efficiently, it would also allow you to build a profile of your personal web tastes. This would prove especially useful for online research: as more content related to a given topic goes online, a smart directory would be able to 'refer' that information to the user, knowing that user's interest. Instead of having to waste time weeding out occasional good content from a surplus of online junk, an intelligent directory would help bring the content to you.

A common cry on the Internet has always been "Content is King." In a sense, that will always be true, because without good content, there's not much of a reason to go online in the first place. But as more and more self-proclaimed monarchs litter the Net with terabytes of content, it would seem that informational anarchy will continue to reign. That is, until someone sits down to build a better (and intelligent) search engine that makes content gathering efficient, enjoyable, and even profitable. Cybrarians of the Net, unite!, for there's still a lot of work to do.

Return to the EdWeb Home RoomI'd like to see the Edweb DictionaryI'd like to send comments to EdWeb

EdWeb: Exploring Technology and School Reform, by Andy Carvin. All rights reserved. puzzle gamesadventure gamesword gameshidden object gamesmanagement gamesbest pc games