About ME

Ting Yao (姚霆)

Google Scholar
tiyao@microsoft.com

Ting Yao is currently a Researcher in the Multimedia Search and Mining group at Microsoft Research, Beijing, China. His research interests include video understanding, large scale multimedia search, and deep learning.

Dr. Yao is an active participant of several benchmark evaluations. He is the principal designer of several top-performing multimedia analytic systems in worldwide competitions such as COCO image captioning, ActivityNet Large Scale Activity Recognition Challenge 2017 and 2016, THUMOS Action Recognition Challenge 2015, and MSR-Bing Image Retrieval Challenge 2014 and 2013. He is one of the organizers of the MSR Video to Language Challenge 2017 and 2016. For his contributions to Multimedia Search by Self, External and Crowdsourcing Knowledge, he was awarded the 2015 SIGMM Outstanding Ph.D. Thesis Award.

He completed a Ph.D. in computer science at the City University of Hong Kong, advised by Prof. Chong-Wah Ngo. He received the B.Sc. degree in theoretical and applied mechanics, B.Eng. double degree in electronic engineering, and the M.Eng. degree in signal and information processing all from the University of Science and Technology of China, Hefei, China. He joined the multimedia search and mining group at Microsoft Research as a research intern during 2012 and 2013, advised by Dr. Tao Mei. He was also a software engineer at the Alibaba Company, Beijing, China, in 2008-2010.

Highlights

  • Rank 1 in the Dense-Captioning Events in Videos and Rank 2 in Temporal Action Proposals of ActivityNet Large Scale Activity Recognition Challenge 2017.

  • Rank 1 in the COCO Image Captioning Challenge.

  • Two papers are accepted by ICCV 2017.

  • Three papers are accepted by CVPR 2017.

Publications

Boosting Image Captioning with Attributes

Ting Yao, Yingwei Pan, Yehao Li, Zhaofan Qiu, Tao Mei
IEEE International Conference on Computer Vision (ICCV), 2017

Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks

Zhaofan Qiu, Ting Yao, Tao Mei
IEEE International Conference on Computer Vision (ICCV), 2017

Deep Semantic Hashing with Generative Adversarial Networks

Zhaofan Qiu, Yingwei Pan, Ting Yao, Tao Mei
ACM conference on Research and Development in Information Retrieval (SIGIR), 2017 (oral)

[pdf], [bibtex]

Incorporating Copying Mechanism in Image Captioning for Learning Novel Objects

Ting Yao, Yingwei Pan, Yehao Li, Tao Mei
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017

[pdf], [bibtex]

Deep Quantization: Encoding Convolutional Activations with Deep Generative Model

Zhaofan Qiu, Ting Yao, Tao Mei
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017

Video Captioning with Transferred Semantic Attributes

Yingwei Pan, Ting Yao, Tao Mei, Houqiang Li
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017

Highlight Detection with Pairwise Deep Ranking for First-Person Video Summarization

Ting Yao, Tao Mei, Yong Rui
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016 (spotlight)

Jointly Modeling Embedding and Translation to Bridge Video and Language

Yingwei Pan, Tao Mei, Ting Yao, Houqiang Li, Yong Rui
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016 (oral)

You Lead, We Exceed: Labor-Free Video Concept Learning by Jointly Exploiting Web Videos and Images

Chuang Gan, Ting Yao, Kuiyuan Yang, Yi Yang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016 (spotlight)

MSR-VTT: A Large Video Description Dataset for Bridging Video and Language

Jun Xu, Tao Mei, Ting Yao, Yong Rui
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016 (poster)

Share-and-Chat: Achieving Human-Level Video Commenting by Search and Multi-View Embedding

Yehao Li, Ting Yao, Tao Mei, Hongyang Chao, Yong Rui
ACM Multimedia (MM), 2016 (oral)

Deep Semantic-Preserving and Ranking-Based Hashing for Image Retrieval

Ting Yao, Fuchen Long, Tao Mei, Yong Rui
International Joint Conference on Artificial Intelligence (IJCAI), 2016

Learning Deep Intrinsic Video Representation by Exploring Temporal Coherence and Graph Structure

Yingwei Pan, Yehao Li, Ting Yao, Tao Mei, Houqiang Li, Yong Rui
International Joint Conference on Artificial Intelligence (IJCAI), 2016

Action Recognition by Learning Deep Multi-Granular Spatio-Temporal Video Representation

Qing Li, Zhaofan Qiu, Ting Yao, Tao Mei, Yong Rui, Jiebo Luo
ACM International Conference on Multimedia Retrieval (ICMR), 2016 (Best Paper Candidate)

Learning Query and Image Similarities with Ranking Canonical Correlation Analysis

Ting Yao, Tao Mei, Chong-Wah Ngo
IEEE International Conference on Computer Vision (ICCV), 2015 (oral)

Semi-supervised Domain Adaptation with Subspace Learning for Visual Recognition

Ting Yao, Yingwei Pan, Chong-Wah Ngo, Houqiang Li, Tao Mei
IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2015

Semi-supervised Hashing with Semantic Confidence for Large Scale Visual Search

Yingwei Pan, Ting Yao, Houqiang Li, Chong-Wah Ngo, Tao Mei
ACM conference on Research and Development in Information Retrieval (SIGIR), 2015 (oral)

Click-through-based Cross-view Learning for Image Search

Yingwei Pan, Ting Yao, Tao Mei, Houqiang Li, Chong-Wah Ngo, Yong Rui
ACM conference on Research and Development in Information Retrieval (SIGIR), 2014 (oral)

Click-boosting Multi-modality Graph-based Reranking for Image Search

Xiaopeng Yang, Yongdong Zhang, Ting Yao, Chong-Wah Ngo, Tao Mei
Multimedia Systems, 2014

Circular Reranking for Visual Search

Ting Yao, Chong-Wah Ngo, Tao Mei
IEEE Trans. on Image Processing (TIP), vol. 22, no. 4, pp. 1644-1655, 2013

Unified Entity Search in Social Media Community

Ting Yao, Yuan Liu, Chong-Wah Ngo, Tao Mei
International World Wide Web Conference (WWW), 2013 (oral)

Annotation for Free: Video Tagging by Mining User Search Behavior

Ting Yao, Tao Mei, Chong-Wah Ngo, Shipeng Li
ACM Multimedia (MM), 2013 (oral)

Click-boosting Random Walk for Image Search Reranking

Xiaopeng Yang, Yongdong Zhang, Ting Yao, Zheng-Jun Zha, Chong-Wah Ngo
International Conference on Internet Multimedia Computing and Service (ICIMCS), 2013 (Best Paper Award)

Predicting Domain Adaptivity: Redo or Recycle?

Ting Yao, Chong-Wah Ngo, Shiai Zhu
ACM Multimedia (MM), 2012

Context-based Friend Suggestion in Online Photo-sharing Community

Ting Yao, Chong-Wah Ngo, Tao Mei
ACM Multimedia (MM), 2012

Co-reranking by Mutual Reinforcement for Image Search

Ting Yao, Tao Mei, Chong-Wah Ngo
ACM International Conference on Image and Video Retrieval (CIVR), 2010 (oral)

Workshops

Seeing Bot

Yingwei Pan, Zhaofan Qiu, Ting Yao, Houqiang Li, Tao Mei
ACM conference on Research and Development in Information Retrieval (SIGIR), 2017 (demo, accepted)

[pdf], [bibtex]

Video ChatBot: Triggering Live Social Interactions by Automatic Video Commenting

Yehao Li, Ting Yao, Rui Hu, Tao Mei, Yong Rui
ACM Multimedia (MM), 2016 (demo)

MSR Asia MSM at ActivityNet Challenge 2016

Zhaofan Qiu, Dong Li, Chuang Gan, Ting Yao, Tao Mei, Yong Rui
In CVPR ActivityNet Challenge Workshop, 2016 (3nd place in Untrimmed Video Classification task)

MSR Asia MSM at THUMOS Challenge 2015

Zhaofan Qiu, Qing Li, Ting Yao, Tao Mei, Yong Rui
In CVPR THUMOS Challenge Workshop, 2015 (2nd place in Action Classification task)

Click-through-based Subspace Learning for Image Search

Yingwei Pan, Ting Yao, Xinmei Tian, Houqiang Li, Chong-Wah Ngo
ACM Multimedia (MM), 2014 (Multimedia Grand Challenge)

Image Search by Graph-based Label Propagation with Image Representation from DNN

Yingwei Pan, Ting Yao, Kuiyuan Yang, Houqiang Li, Chong-Wah Ngo, Jingdong Wang, Tao Mei
ACM Multimedia (MM), 2013 (Multimedia Grand Challenge)

Video Concept Detection by Learning from Web Images: A Case Study on Cross Domain Learning

Shiai Zhu, Ting Yao, Chong-Wah Ngo
ICME workshop on Media Fragment Creation and reMIXing (MMIX), 2013 (oral)

VIREO/ECNU @ TRECVID 2013: A Video Dance of Detection, Recounting and Search with Motion Relativity and Concept Learning from Wild

C. W. Ngo, F. Wang, W. Zhang, C. C. Tan, Z. H. Sun, S. A. Zhu and T. Yao
NIST TRECVID Workshop (TRECVID'13), 2013

VIREO @ TRECVID 2012: Searching with Topology, Recounting will Small Concepts, Learning with Free Examples

W. Zhang, C.-C. Tan, S. A. Zhu, T. Yao, L. Pang and C.-W. Ngo
NIST TRECVID Workshop (TRECVID'12), 2012

VIREO @ TRECVID 2011: Instance Search, Semantic Indexing, Multimedia Event Detection and Known-Item Search

C.-W. Ngo, S. A. Zhu, W. Zhang, C.-C. Tan, T. Yao, L. Pang and H.-K. Tan
NIST TRECVID Workshop (TRECVID'11), 2011