Date: Wednesday, July 2nd, 2014
Time: 4-5pm
Place: Informatics Teaching Lab (ITL) — Top floor meeting room
Speaker: Zhiyuan Shi
Title:Weakly Supervised Learning of Objects, Attributes and their Associations

Abstract:

When humans describe images they tend to use combina-
tions of nouns and adjectives, corresponding to objects and their as-
sociated attributes respectively. To generate such a description auto-
matically, one needs to model objects, attributes and their associations.
Conventional methods require strong annotation of object and attribute
locations, making them less scalable. In this paper, we propose to model
object-attribute associations from weakly labelled images, such as those
widely available on media sharing sites (e.g. Flickr), where only image-
level labels (either object or attributes) are given, without their locations
and associations. This is achieved by introducing a novel weakly super-
vised non-parametric Bayesian model. Once learned, given a new im-
age, our model can describe the image, including objects, attributes and
their associations, as well as their locations and segmentation. Exten-
sive experiments on benchmark datasets demonstrate that our weakly
supervised model performs at par with strongly supervised models on
tasks such as image description and retrieval based on object-attribute
associations.​

Bio : Zhiyuan Shi is currently a 3rd year PhD student in our group.