January 2009 - Februari 2010

Course: PhD project
Software: C++, with cximage library

Description

This project provides a comparative study of content-based copy detection methods, which include research literature methods based on salient point matching, discrete cosine and wavelet transforms, color histograms and a biologically motivated visual matching method. The evaluation focuses on large-scale applications, especially on performance in the context of search engines for web images. For our experiments, original images have been altered by a diverse set of realistic transformations and have been embedded in a collection of one million web images and one million Flickr images.

Copy detection methods

We have selected four copy detection methods from recent literature, each of which uses a different representation as basis for detecting copies, namely discrete cosine transform, discrete wavelet transform, color histograms and interest points. In addition we have developed a method ourselves, which is based on the human vision system. We have implemented the four existing copy detection methods to the best of our ability, based on the sequence of steps and values used as described in their respective papers. In order to determine the accuracy relative to varying descriptor sizes, we also created variations of the original methods.

Tourist database

This database consists of 6000 color photos taken at various touristic locations around the world. Of each of these images several copies are created by applying 60 different transformations on the original query image. We performed a survey of common image manipulations we encountered on the internet, and have reflected these in the set of copies we created. Common transformations are for instance the use of various levels of compression when saving an original or scaling the original image up or down. In Figure 1 we can see some transformation examples.

a) b) c) d) e)

Figure 1. Several transformations: a) the original image, b) black border added, c)increased brightness, d) increased saturation, e) copyright logo and text added

Web database

Using our noteworthy crawler we downloaded two million images from the internet, see for example Figure 2.

Figure 2. Example images from the web database. Note that for copyright reasons we do not show any advertisements or celebrity photos, even though these are well-represented in the web collection.

Flickr database

Using our flickr crawler we downloaded two million images with high 'interestingness' value from the Flickr website that all have a Creative Commons license. See Figure 3 for some example images.

Figure 3. Example images from the flickr database.

Experimental setup

Our goal is to mix the tourist images (originals and their copies) with the web images and run each of the copy detection methods to see if they can find all copies of each original image. We also perform the same experiments using the mix of tourist images with the flickr images. We keep the tourist+web mix separate from the tourist+flickr mix, because we want to analyze the differences in performance of the copy detection methods on these two image collections. In contrast to the web images, which may contain all sorts of images (e.g. logos), flickr generally only hosts images that are photographic of nature. Since our tourist images are also photographic, we expect it to be more difficult for the copy detection methods to find the copies of an original in the tourist+flickr mix than in the tourist+web mix.

The tourist images are known to not be published anywhere else on the internet, ensuring that the only copies in the test set will be the images generated by the transformations on the original query image.

To evaluate the accuracy of the copy detection methods we use a testing framework that for each query image measures the distances to all images in the collection. Ideally, all copies have small distances to the query image, whereas all other images have large distances. One type of results we present is average precision at different recall levels versus descriptor size, since we are specifically interested in the accuracy of a method with respect to its computational requirements. The results allow us to quantify how well a copy detection method is able to detect the copies.

Publications

For more information and experimental results, take a look at the MIR2008 paper in the publications section. This paper compares the tourist images with the web images. Extensive experiments on the tourist+flickr images have finished and the results have been submitted to the TOIS journal.