[week 7] Reading Notes

Relevance Feedback

• In the section 9.1, the authors wrote the following: “RF can also be effective in tracking a user’s evolving information need”. I am not so sure of this, because if the user understands better her information need, will probably reformulate her query instead of waiting for the system to recalculate a result set.

Foundations of the Rocchio Algorithm

• I don’t understand how the formula (9.2)

is derived from the formula 6.10 of vector similarity.

The Rocchio Algorithm

• The book states that Relevance Feedback has shown to be useful for increasing recall in situations where it is important, and one of the reasons is because the technique expands the query. Is this always true? When I expand the query and I do a Boolean match of the query terms with AND, adding more terms can increase precision but not necessarily recall. Now, I know that in this case this is a VSM and not a Boolean one, but it is always true that adding terms to query the recall is increased?

• Besides, I don’t understand clearly the IDE DEC-HI concept.

[week 6] Muddiest Points

This week's lecture was about Evaluation of IR Systems. In addition to the weekly book reading, we read two papers:

• Karen Sparck Jones, What's the value of TREC: is there a gap to jump or a chasm to bridge? ACM SIGIR Forum, Volume 40 Issue 1 June 2006
• Kalervo Järvelin, Jaana Kekäläinen. Cumulated gain-based evaluation of IR techniques ACM Transactions on Information Systems (TOIS) Volume 20 , Issue 4 (October 2002) Pages: 422 – 446

TREC
Through the IIR book chapter and the first paper, I was introduced to TREC (Text REtrieval Conference) :
1. I was I little surprised when reading the paper about TREC with the affirmation of the author: <<...this progress... has not so far enabled the research community it represents to say:'if your retrieval case is like this, do this' as oposed to 'well, with tunning, this sort of thing could serve yo alright'. >> Despite Google's success, is this still the feeling about the IR community?
2. Which are the sources used to generate content in the Biological and Law tracks of TREC?
3. How do they (TREC consortium) decide to stop a track or to create a new one?

Interpolation on P-R graphs
About the Precision-Recall graphs, I wonder if the Interpolation process may produce, in some cases, a false interpretation of the performance of an IR system.

[week 6] Reading Notes

After reading chapter 8th of IIR and the paper about the TREC collection, I started to look more deeply about the different domains covered by this project. It was specially interesting finding a TREC Genomics data and also a TREC law.

About the first one, I read some papers about it and I found out that the project had been running from 2003 to 2007, and in this paper written by the leader of the project,
William Hersh, he states that the project was a success but there was not too much advancement over the state of the art of IR. In his words:

<<...As with all TREC activity, the short cycle of experimentation and reporting of results has prevented more detailed investigation of different approaches. However, there emerged some evidence that some resources from the genomics/bioinformatics could contribute to improving retrieval, especially controlled lists of terminology used in query expansion, although their improvement over standard state-of-the-art IR was not substantial. ...>>

Which is the real gain in creating this test datasets? Is there any new algorithm based on experiments using TREC data, for example?

Other question I raised was if there is some research about creating queries automatically using NLP based on the description of the user need .

[week 5] Reading Notes

2 questions have raised this week:

1. The binary independence model assumes that terms occur in in documents independently and the authors say that nevertheless the assumption is not right, in practice the models perform satisfactorily in some occasions. Is there any explanation for this result? this practical evidence is just in English language or it also occurs with Chinese and Arab languages?

2. In chapter 12, the authors say that most of the time the Stop and (1 - Stop) probabilities are omitted from the language model. If this situation incurs in not modeling a well-formed language (according to Equation 12.1), why do authors do this?

See you on Thursday in class!

[week 4] Muddiest Points

I always write long posts, yet this week I'll make the long story short:

• In slide 47, professor presented the Lucene term weighting and the SMART term weighting.. which is the term weighting for Lemur?
• I have been investigating about Gene databases, I wonder whether boolean search or vector space model are used for searching data in these collections. Is there any other popular way to index this data in particular?
  
• In slide 67, weights for terms in a query are "guessed"... which are the most popular ways to guess their importance, i.e., their weighting? Besides, is there any common scale (0-1, 1-10) used for this purpose?

[week 4] Reading Notes

In the last paragraph of chapter 1, the authors talk about the practical usage of regular expressions for searching. I wanted to try the behavior of web search engines (google, yahoo, msn) in their response to a simple query involving a regular expression: "hello" and then "hel*o". In general, the * means 0 or more characters. The results are presented in the tables below (showing the first two results per each query in each search engine)

Query "hello":


Query "hel*o":
(sorry for posting screenshots, but blogger doesn't have a good table tool in its WYSIWYG editor)

First, I saw that there is some treatment of the * symbol, since there is no result with the exact match hel*o. Now, the question is if it is replaced just as in a regular expression match.

If the regular expression just as we know them had been working in these search engines, I would have expected more results for the search hel*o than for hello, but it was the contrary.

It seems that search engines replace the * for characters different than just one alphanumeric. The results showed that they looked for especial characters, such as - (hyphen) or for more than one concatenated with alphanumeric characters. However they don't seem to replace * per each letter in the alphabet, to then perform successive queries such as helao, helbo, helco, etc

Finally, I note the differences in the amount of returned results between search engines, it's incredible (Yahoo returned more than 1 and a half billion pages to query hello, whereas google 476 million and msn 81 million).

[week 3] Muddiest Points

My first muddiest point is referred to the data structures used for Access Methods. Are Hash table, Binary Trees and B-Trees the only options? There is no other structure like a linked list or a double linked list commonly used for this purpose?

About the compression methods, I just wonder how the indexers deal with the characters codification, for example Unicode UTF-8 characters, UTF-16 or ISO-8859. Should these codifications change the way IR systems index or compress the data? in which way?