Contour-Assisted Visual Inference: Systems, Algorithms, and Applications


Investigators:
Haibin Ling, PhD
Dept. of Computer & Information Sciences
Temple University
hbling AT temple.edu
(tel)1-215-204-6973 | (fax)1-215-204-5082

Jingyi Yu, PhD
Dept. of Computer & Information Sciences
University of Delaware
yu AT eecis.udel.edu
(tel)1-302-831-0345

Supported by:
     
NSF project link
      

Summary:
This project develops a new camera toward acquisition and application occlusion contour (OC) images. Unlike regular photographs, an OC image removes the effects of illumination, texture, and appearance while preserving important depth edges and silhouette. Being well known to play important roles in many computer vision tasks, OC has been limited in real world applications due to the lack of effective acquisition systems and the lack of systematic approaches to utilize OCs. This project, on the sensor front, designs a new Occlusion Contour Camera (OC-Cam) based on the multi-flash camera technique. The camera first takes successive photos of a scene with flash lights at different locations and then calculates the occlusion contours by analyzing the patterns in the shadow. On the application fronts, this project systematically develops OC-assisted visual inference algorithms. At the core is a novel and effective metric for measuring the similarity between OCs. Based on this metric, the project explores a wide range of OC applications including visual recognition and retrieval, visual tracking, visual summarization and privacy preserving surveillance.
The project has deep impact on broad areas of computer vision, artificial intelligence, criminal justices, and disaster controls, both in research and education. Due to the importance of OCs in human vision, the study generates testbed to the study of visual psychology. In addition, the proposed OC-Cam serves as conceptual inspiration for constructing the next-generation surveillance systems and can fundamentally change the way people think about images and photographs. The captured OC datasets and relevant tools are made available to other researchers, to provide a platform for validating new contour-based computer vision algorithms.

Objectives:
The aims of the project include (1) developing an OC acquisition camera, (2) investigating OC-Assisted Visual Inference, including visual recognition, visual tracking, visual summarization, and visual privacy protection, and (3) deploying OC databases for the research community.

Selected Publications (links to data and code provided whenever applicable)

Symmetry-Aware Graph Matching
T. Wang, H. Ling, C. Lang, and H. Yang
Pattern Recognition (PR), in press
Salient Object Detection via Structured Matrix Decomposition
H. Peng, B. Li, H. Ling, W. Hu, W. Xiong, and S. Maybank
IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 2016.
code | data | benchmark |
Light Mixture Intrinsic Image Decomposition Based on a Single RGB-D Image
Guanyu Xing, Yanli Liu, Wanfa Zhang, and Haibin Ling
The Visual Computer, 32(6-8): 1013-1023, 2016
PDF
Tensor Power Iteration for Multi-Graph Matching
X. Shi, H. Ling, W. Hu, J. Xing, and Y. Zhang
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016.
PDF |
Path Following with Adaptive Path Estimation for Graph Matching
T. Wang and H. Ling
Proc. of AAAI Conference on Artificial Intelligence (AAAI), 2016
PDF
Encoding Color Information for Visual Tracking:Algorithms and Benchmark
P. Liang, E. Blasch, and H. Ling
IEEE Trans. on Image Processing (T-IP), 24(12):5630--5644, 2015.
PDF | Download Benchmark
Predicting Image Memorability by Multi-view Adaptive Regression
H. Peng, K. Li, B. Li, H. Ling, W. Xiong, and W. Hu
ACM Multimedia Conference (ACM MM), 2015.
PDF
Cross-Age Face Verification by Coordinating with Cross-Face Age Verification
L. Du and H. Ling
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2015.
PDF | Source code
Dynamic Scene Classification Using Redundant Spatial Pooling
L. Du and H. Ling
IEEE Trans. on Cybernetic, in press
PDF
Covert Photo Classification by Fusing Image Features and Visual Attributes
H. Lang and H. Ling
IEEE Trans. on Image Processing (T-IP), 24(10):2996--3008, 2015
dataset
Robust Nighttime Vehicle Detection by Tracking and Grouping Headlights
Q. Zou, H. Ling, S. Luo, Y. Huang, and M. Tian
IEEE Trans. on Intelligent Transportation Systems (T-ITS), in press
GARP-Face: Balancing Privacy Protection and Utility Preservation in Face De-identification
L. Du, M. Yi, E. Blasch, and H. Ling
Proc. of IEEE Int'l Joint Conf. on Biometrics (IJCB), Clearwater, 2014.
PDF |
Exploiting Competition Relationship for Robust Visual Recognition
L. Du and H. Ling
Proc. of AAAI Conference on Artificial Intelligence (AAAI), 2014
PDF | Code
Transfer Learning Based Visual Tracking with Gaussian Process Regression
J. Gao, H. Ling, W. Hu, and J. Xing
In Proc. of European Conference on Computer Vision (ECCV), Part III, LNC 8691, pp. 188-203, 2014.
PDF | Supplementary | Code |
Bin Ratio-Based Histogram Distances and Their Application to Image Classification
W. Hu, N. Xie, R. Hu, H. Ling, Q. Chen, S. Yan, and S. Maybank
IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 36(12): 2338-2352, 2014.
PDF |
Multi-target Tracking with Motion Context in Tenor Power Iteration
X. Shi, H. Ling, W. Hu, C. Yuan, and J. Xing
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Columbus OH, 2014
PDF
Curvilinear Structure Tracking by Low Rank Tensor Approximation with Model Propagation
E. Cheng, Y. Pang, Y. Zhu, J. Yu, and H. Ling
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Columbus OH, 2014
PDF
Saliency Detection on Light Fields
N. Li, J. Ye, Y. Ji, H. Ling, and J. Yu
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Columbus OH, 2014
PDF | dataset
Visual Tracking via Online Non-negative Matrix Factorization
Y. Wu, B. Shen, and H. Ling
IEEE Trans. on Circuits and Systems for Video Technology (T-CSVT), 24(3): 374-383, 2014.
PDF
Modeling Geometric-Temporal Context with Directional Pyramid Co-occurrence for Action Recognition
C. Yuan, X. Li, W. Hu, H. Ling, and S. Maybank
IEEE Trans. on Image Processing (T-IP), 23(2):658--672, 2014.
Salient Region Detection by UFO: Uniqueness, Focusness and Objectness
P. Jiang, H. Ling, J. Yu, and J. Peng
IEEE International Conference on Computer Vision (ICCV), 2013.
PDF | Code and results
Finding the Best from the Second Bests - Inhibiting Subjective Bias in Evaluation of Visual Tracking Algorithms
Y. Pang and H. Ling
IEEE International Conference on Computer Vision (ICCV), 2013.
PDF
Line-Assisted Light Field Triangulation and Stereo Matching
Z. Yu, X. Guo, H. Ling, A. Lumsdaine, and J. Yu
IEEE International Conference on Computer Vision (ICCV), 2013.
PDF
Scale and Object Aware Image Thumbnailing
J. Sun and H. Ling
International Journal of Computer Vision (IJCV), 2013, in press.
PDF | Code and data
Efficient Minimum Error Bounded Particle Resampling L1 Tracker with Occlusion Detection
X. Mei, H. Ling, Y. Wu, E. Blasch, and L. Bai
IEEE Trans. on Image Processing (T-IP), 22(7): 2661-2675, 2013
PDF
Wavelet Domain Multi-fractal Analysis for Static and Dynamic Texture Classification
H. Ji, X. Yang, H. Ling, and Y. Xu
IEEE Trans. on Image Processing (T-IP), 22(1):286-299, 2013
PDF
Multi-target Tracking by Rank-1 Tensor Approximation
X. Shi, H. Ling, J. Xing, and W. Hu
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2013.
PDF
3D R Transform on Spatio-Temporal Interest Points for Action Recognition
C. Yuan, X. Li, W. Hu, H. Ling, and S. Maybank
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2013.
PDF
Foreground and Scene Structure Preserved Visual Privacy Protection using Depth Information
S. Elezovikj, H. Ling, and X. Chen
IEEE Int'l Conf. on Multimedia and Expo (ICME), 2013.
PDF
Using Maximum Consistency Context for Multiple Target Association in Wide Area Traffic Scenes
X. Shi, P. Li, W. Hu, E. Blasch, and H. Ling
Proc. Int'l Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2013
PDF
Kernel-based Motion-blurred Target Tracking
Y. Wu, J. Hu, F. Li, E. Cheng, J. Yu, and H. Ling.
Int'l Symposium on Visual Computing (ISVC), 2011.
Blurred Target Tracking by Blur-driven Tracker
Y. Wu, H. Ling, J. Yu, F. Li, X. Mei, and E. Cheng
IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, 2011.
PDF (~11M) | TUblur sequences and annotation (~430M) |
Category Classification Using Occluding Contours
J. Sun, C. Thorpe, N. Xie, J. Yu, and H. Ling
5th Int'l Symposium on Visual Computing (ISVC), 2010.
PDF | DT-OC5 dataset