Current Projects:
Web-scale Image Understanding
This is an extension work to Search-based Image Annotation system. We attempt to leverage Web-scale image data to build up an effective and efficiency image understanding framework. New ideas need to be proposed on high-dimensional indexing, content analysis, and concept modeling, etc.
Former Projects:
Search-based Image Annotation
Image annotation nowadays is still far from practical and satisfactory given so many computer vision and machine learning
approaches. Possible reasons are: 1) it is still unclear how to model the semantic concepts effectively and efficiently; 2) the lack of training data to bridge effectively the semantic gap.
With the explosive development of the Web, it has become a huge
resource of all kinds of data and has brought about possible solutions to many problems that were believed to be “unsolvable.”
In this project, we propose to leverage the huge number of images existing on the Web and mine the annotations for an image. The key idea is to find a group of similar images both semantically and visually, extract key phrases from their textual descriptions, and select the highest-scored ones to annotate the query image.
A notable advantage is that the proposed approach is entirely unsupervised. No supervised learning approach is required to train
a prediction model as a traditional approach does. And as a direct
result, this method has no limitations on vocabulary, making it fundamentally different from the previous works.
Multi-Modal Image Retrieval
Rather than the traditional content-based image retrieval and text retrieval techniques, we attempt to leverage the richly structured, heterogeous datasets, in which images, hyperlinks, surrounding texts are inter-related with each other, to model the intinsic image similarity structure for a better retrieval performance. Some similarity propagation algorithms are proposed for this project.
Reference Case Finding
Due to the high turn-over rate in call centers, most of agents are inexperienced. To improve agents' productivity, it is important to provide them reference cases, i.e. the digested solutions to the current customer's problem, which are retrieved/mined from knowledge bases. Other than the high precision, this project also requires real-time service.