Publication: Incorporating variability in user behavior into systems based evaluation
dc.contributor.coauthor | Kanoulas, Evangelos | |
dc.contributor.coauthor | Carterette, Ben | |
dc.contributor.kuauthor | Yılmaz, Emine | |
dc.date.accessioned | 2024-11-09T23:11:43Z | |
dc.date.issued | 2012 | |
dc.description.abstract | Click logs present a wealth of evidence about how users interact with a search system. This evidence has been used for many things: learning rankings, personalizing, evaluating effectiveness, and more. But it is almost always distilled into point estimates of feature or parameter values, ignoring what may be the most salient feature of users - -their variability. No two users interact with a system in exactly the same way, and even a single user may interact with results for the same query differently depending on information need, mood, time of day, and a host of other factors. We present a Bayesian approach to using logs to compute posterior distributions for probabilistic models of user interactions. Since they are distributions rather than point estimates, they naturally capture variability in the population. We show how to cluster posterior distributions to discover patterns of user interactions in logs, and discuss how to use the clusters to evaluate search engines according to a user model. Because the approach is Bayesian, our methods can be applied to very large logs (such as those possessed by Web search engines) as well as very small (such as those found in almost any other setting). | |
dc.description.indexedby | Scopus | |
dc.description.openaccess | YES | |
dc.description.publisherscope | International | |
dc.description.sponsoredbyTubitakEu | N/A | |
dc.description.sponsorship | Special Interest Group on Information Retrieval (ACM SIGIR) | |
dc.description.sponsorship | ACM SIGWEB | |
dc.identifier.doi | 10.1145/2396761.2396782 | |
dc.identifier.isbn | 9781-4503-1156-4 | |
dc.identifier.quartile | N/A | |
dc.identifier.scopus | 2-s2.0-84871091208 | |
dc.identifier.uri | https://doi.org/10.1145/2396761.2396782 | |
dc.identifier.uri | https://hdl.handle.net/20.500.14288/9669 | |
dc.keywords | Parameter values | |
dc.keywords | Point estimate | |
dc.keywords | Posterior distributions | |
dc.keywords | Probabilistic models | |
dc.keywords | Salient features | |
dc.keywords | Search system | |
dc.keywords | Single users | |
dc.keywords | Test Collection | |
dc.keywords | Time of day | |
dc.keywords | User behaviors | |
dc.keywords | User interaction | |
dc.keywords | User log | |
dc.keywords | User models | |
dc.keywords | Bayesian networks | |
dc.keywords | Behavioral research | |
dc.keywords | Knowledge management | |
dc.keywords | Probability distributions | |
dc.keywords | Search engines | |
dc.language.iso | eng | |
dc.publisher | ACM | |
dc.relation.ispartof | ACM International Conference Proceeding Series | |
dc.subject | Computer engineering | |
dc.title | Incorporating variability in user behavior into systems based evaluation | |
dc.type | Conference Proceeding | |
dspace.entity.type | Publication | |
local.contributor.kuauthor | Yılmaz, Emine |