Please enable JavaScript.
Coggle requires JavaScript to display documents.
7 [IR] Relevance Feedback (Relevance Feedback (How to use relevance…
7 [IR] Relevance Feedback
Relevance
1
Relevance is the basis for evaluating IR
Both recall and precision depends on "relevance"
Relevance is difficult to define precisely
The full set of relevant documents is never known
2
A relevant document is one that a person judges as useful in the context of a specific information need
Two related concepts
Topical relevance
assume that relevance is
solely a property of the internal mechanism of the retrieval system
is related the content of the document, so it is objective
is the result of the match between a query and the document representation
virtually ignoring the role of user
limitations
assume relevance to a query = relevance to a need
Assume relevance to be objective
assume relevance is static, not change
But is useful for evaluating IR system and algorithms
No user around, so cheap to run experiments
Relevance of docs is static, so can run experiements many time
Relevance of documents only related to the content and the mechanisms inside IR system, so provide basis for comparing effectiveness of different retrieval systems
Topical relevance is an important factor in users' relevance judgment
concentrate on "aboutness"
Utility
not concentrate on "aboutness", but on "usefulness"
Relevance Feedback
Basic settings
Search system heavily reply on queries for finding relevant docs
But a query only approximates user's information need
User initial query is often short and poor approximation
People can improve query when seeing relevant and non-relevant docs
Procedure of relevance feedback
Basic procedure
a user issue a (short, simple) query
The system returns an initial set of retrieval results
The user marks some returned documents as relevant or not relevant
The system computes a better representation of the information need based on the user feedback
The system displays a revised set of retrieval results
Types of relevance feedback
interactive relevance feedback: feedback information obtained from the user
Explicit relevance feedback
let user mark relevant and irrelevant documents
Implicit relevance feedback
system attempt to infer user intentions based on observable behavior
Blind relevance feedback or pseudo relevance feedback (doesn't always help)
feedback in absence of any evidence, explicit or otherwise
system assumes that the top ranked documents as relevant docs
How to use relevance feedback
assume that there is an optimal query
relevance feedback helps to bring user's query closer to the optimal one
how
Term reweighting
boost weights of terms from relevant documents
query expansion
add terms from relevant documents to the query
Rocchio
Goals
know the basic idea of relevance feedback and query expansion
know the formulas of relevance feedback in vector space model and probabilistic model