UJM at ImageCLEFwiki 2008

Christophe Moulin (1) , Cecile Barat (1) , Mathias Géry (1) , Christophe Ducottet (1) , Christine Largeron (1)

1 LHC - Laboratoire Hubert Curien

This paper reports our multimedia information retrieval experiments carried out for the ImageCLEF track (ImageCLEFwiki). The task is to answer to user information needs, i.e. queries which may be composed of several modalities (text, image, concept) with ranked lists of relevant documents. The purpose of our experiments is twofold: firstly, our overall aim is to develop a multimedia document model combining text and/or image modalities. Secondly, we aim to compare results of our model using a multimedia query with a text only model. Our multimedia document model is based on a vector of textual and visual terms. The textual terms correspond to words. The visual ones result from local colour descriptors which are automatically extracted and quantized by k-means, leading to an image vocabulary. They represent the colour property of an image region. To perform a query, we compute a similarity score between each document vector (textual + visual terms) and the query using the Okapi method based on the tf.idf approach. We have submitted 6 runs either automatic or manual, using textual, visual or both information. Thanks to these 6 runs, we aim to study several aspects of our model, as the choice of the visual words and local features, the way of combining textual and visual words for a query and the performance improvements obtained when adding visual information to a pure textual model. Concerning the choice of the visual words, results show us that they are significant in some cases where the visualness of the query is meaningful. The conclusion about the combination of textual and visual words is surprising. We obtain worth results when we add directly the text to the visual words. Finally, results also inform that visual information bring complementary relevant documents that were not found with the text query. These initial results are promising and encourage the development of our multimedia model.

Domains: Machine Learning [cs.LG]; Image Processing [eess.IV]; Information Retrieval [cs.IR]

Christophe Moulin, Cecile Barat, Mathias Géry, Christophe Ducottet, Christine Largeron. UJM at ImageCLEFwiki 2008. ECDL 2008 - Workshop CLEF, Sep 2008, Aarhus, Denmark. ⟨ujm-00326228⟩