Publication:
Incorporating variability in user behavior into systems based evaluation

dc.contributor.coauthorKanoulas, Evangelos
dc.contributor.coauthorCarterette, Ben
dc.contributor.kuauthorYılmaz, Emine
dc.date.accessioned2024-11-09T23:11:43Z
dc.date.issued2012
dc.description.abstractClick logs present a wealth of evidence about how users interact with a search system. This evidence has been used for many things: learning rankings, personalizing, evaluating effectiveness, and more. But it is almost always distilled into point estimates of feature or parameter values, ignoring what may be the most salient feature of users - -their variability. No two users interact with a system in exactly the same way, and even a single user may interact with results for the same query differently depending on information need, mood, time of day, and a host of other factors. We present a Bayesian approach to using logs to compute posterior distributions for probabilistic models of user interactions. Since they are distributions rather than point estimates, they naturally capture variability in the population. We show how to cluster posterior distributions to discover patterns of user interactions in logs, and discuss how to use the clusters to evaluate search engines according to a user model. Because the approach is Bayesian, our methods can be applied to very large logs (such as those possessed by Web search engines) as well as very small (such as those found in almost any other setting).
dc.description.indexedbyScopus
dc.description.openaccessYES
dc.description.publisherscopeInternational
dc.description.sponsoredbyTubitakEuN/A
dc.description.sponsorshipSpecial Interest Group on Information Retrieval (ACM SIGIR)
dc.description.sponsorshipACM SIGWEB
dc.identifier.doi10.1145/2396761.2396782
dc.identifier.isbn9781-4503-1156-4
dc.identifier.quartileN/A
dc.identifier.scopus2-s2.0-84871091208
dc.identifier.urihttps://doi.org/10.1145/2396761.2396782
dc.identifier.urihttps://hdl.handle.net/20.500.14288/9669
dc.keywordsParameter values
dc.keywordsPoint estimate
dc.keywordsPosterior distributions
dc.keywordsProbabilistic models
dc.keywordsSalient features
dc.keywordsSearch system
dc.keywordsSingle users
dc.keywordsTest Collection
dc.keywordsTime of day
dc.keywordsUser behaviors
dc.keywordsUser interaction
dc.keywordsUser log
dc.keywordsUser models
dc.keywordsBayesian networks
dc.keywordsBehavioral research
dc.keywordsKnowledge management
dc.keywordsProbability distributions
dc.keywordsSearch engines
dc.language.isoeng
dc.publisherACM
dc.relation.ispartofACM International Conference Proceeding Series
dc.subjectComputer engineering
dc.titleIncorporating variability in user behavior into systems based evaluation
dc.typeConference Proceeding
dspace.entity.typePublication
local.contributor.kuauthorYılmaz, Emine

Files