Meet BESS

We hear so much about collaborative web-sites, wikis, user-generated content, the vast dialog that is the blogosphere, social media and social bookmarking, online networking, the whole web 2.0 revolution, that it is hard to imagine a time when finding useful information meant simply tapping in a few keywords into a search engine. But, I hear you scream, Google still reigns supreme and the vast majority of web users are still doing just that.

Yes, you’re probably right, there is still a lot of educating to do, to persuade the average non-techie user, present excepted, of course, that collaborative methods of finding the best, most relevant, most informative, and perhaps even the most entertaining, information is the future.

Part of the problem lies with the whole notion that beauty, or relevance more precisely, is in the eye of the beholder. Web-spam aside, who’s to say that the first page of search engine hits is going to be relevant to members of a particular niche. Even with specialist search engines for movie buffs, photographers, artists, scientists, musicians, etc, there is still the problem of information overload. No search engine yet invented provides the perfect answers for everyone, each and every time.

Now, Roman Shtykh and Qun Jin of the Networked Information Systems Laboratory at Waseda University, in Japan, have also recognized that a one-size-fits-all to dealing with information overload is not the most effective way of satisfying anyone’s individual information needs. But, rather than simply griping about the problem, they propose a new a new form of collaborative personalized search that attempts to understand your search.

They have developed a web information retrieval framework called Better Search and Sharing (BESS) that captures interactions between users and the system. This can then be used to produce user profile and then tailor results to personal interests dynamically, the same mechanism could be used to co-evaluate documents found valuable within a specific search context by users with similar interests, their subjective index.

Currently, systems such as Swiki (social wiki, Eurekster was unavailable at the time of writing), Rollyo, and the Google Custom Search Engine correspond to the vertical and mostly community-oriented approach to search personalization. They allow communities to create personalized search engines around specific interests, Shtykh and Jin say. However, the advent of collaborative web 2.0 sites that favor the transition of each person’s activities from passive browsing to active participation have changed the search situation radically.

BESS is different. It is a community-oriented system, but has the features of a horizontal search system, like Google personalized search and a vertical search system like Google custom search rolled into one. “It performs searches on the information assets of both horizontal (the objective index unedited, non-controlled) and vertical (subjective index, evaluated, commented).

The notion of the subjective index in our research is similar to the “social search” of the vertical community-oriented systems presented above, but differ in the higher degree of personalization for every user, the high granularity of the vertical search model and, finally, the way of collecting and (re-)evaluating the information pieces.

In other words, groups of users are formed dynamically without user intervention, based on matching interests and expertise. The role of the community becomes indispensable for improving search quality and the evolution of the system, in general, the researchers add.

So, how does BESS work and what does it (she?) do? The proposed system consists of a contribution software component on the client side and the BESS collaborative information retrieval framework on the server side. The user contribution component would be a Firefox browser extension that allowed new elements (html) to be embedded into the search (engine) result pages (SERPs) next to every result.

On the server side, BESS will utilize a web proxy, software to analyze user behavior (not for spyware purposes, of course, but to allow the system to function), a profile construction engine. It then captures user search and post-search behavior, analyzes them statistically generates updated profiles and then searches the growing subjective index to find the optimum hits.

The team has demonstrated proof of principle using the example of a search for Mitsubishi air conditioners. They assume that user Annabelle is looking for information on air-conditioning units in Mitsubishi cars. She searches for “Mitsubishi air conditioner”. Annabelle, however, is unaware that Mitsubishi Electric produces air-conditioning systems for buildings too and is thinking only of her car.

Conventional SERPs will be a mixed bag of reviews, information, and documents about both vehicular and building A/C units. BESS, however, knows that Annabelle is keen on her car and has already been searching for information she flagged as interesting in her subjective index covering other option extras in cars. So, BESS presents hits on in-car air-conditioners from Mitsubishi instead of its domestic appliances entries.

Annabelle’s friend Carl who is in the same automobile enthusiasts group may subsequently be looking for car A/C units but searching by company name and the phrase air-conditioners too, but BESS, aware of their shared interest will again fire up the car equipment instead of the domestic units. The flagging and tagging of positive hits could be automatic (done by BESS) or deliberate on the part of Annabelle and Carl. Either way, BESS learns about their behavior and builds their community profiles accordingly.

There is a drawback to BESS. “Any web personalization system requires storing and analyzing personal information,” the researchers say, and “this is seen to be a problem by privacy advocates.” They point out that client-side personalization can alleviate some of the privacy concerns but would preclude BESS from working well with a community. “We have to consider how to ensure users’ privacy, probably by combining the client-side and server-side approaches for storing and processing user-sensitive information,” they conclude. However, there are millions of people using the likes of Google custom search and countless web 2.0 sites, so while privacy is a concern, the benefits of a more powerful and useful search system might outweigh such concerns for many users.

Harnessing user contributions and dynamic profiling to better satisfy individual information search needs, in International Journal of Web and Grid Services, 2008, vol. 4, pp 63-79