Picture Tags and World Knowledge
Lexing Xie (ANU Research School of Computer Science)
ARTIFICIAL INTELLIGENCE SEMINARDATE: 2012-08-24
TIME: 15:00:00 - 16:00:00
LOCATION: RSISE Seminar Room, ground floor, building 115, cnr. North and Daley Roads, ANU
CONTACT: JavaScript must be enabled to display this email address.
ABSTRACT:
The proliferation of tagged social media data brings a challenge to understanding collective tag use at scale -- which tags are used, which aren't, what is the language of multimedia? Image tagging has been extensively studied in recent research literature, while visual tags are not typically connected to real-world knowledge. We propose a new method to connect three distinct large online resources: Flickr, ImageNet/WordNet, and ConceptNet. This allows us to quantify the visual relevance of tags and tag-tag relationships, which in turn lead to a learning algorithm for image annotation that takes into account both image and tag features. We analyze thousands of image tags on millions of visually labelled photos. Their statistics allow us to observe tag utility and meanings, and in particular, get insights about collective categorization. Such observations lead to a design of tag features and a tag annotation model with competitive performance. Other applications of this work can be in generating natural language descriptions of pictures, validating the quality of commonsense knowledge, and generalizing to unseen tags.
