Exploring Hidden Coherent Feature Groups and Temporal Semantics for Multimedia Big Data Analysis

Authors

Yimin Yang, School of Computing and Information SciencesFollow

Document Type

Dissertation

Degree

Doctor of Philosophy (PhD)

Major/Program

Computer Science

First Advisor's Name

Shu-Ching Chen

First Advisor's Committee Title

Committee chair

Second Advisor's Name

Jainendra K. Navlakha

Second Advisor's Committee Title

Committee member

Third Advisor's Name

Xudong He

Third Advisor's Committee Title

Committee member

Fourth Advisor's Name

Keqi Zhang

Fourth Advisor's Committee Title

Committee member

Fifth Advisor's Name

Mei-Ling Shyu

Fifth Advisor's Committee Title

Committee member

Keywords

Multimedia data mining, big data, multiple correspondence analysis, multimedia semantic retrieval, temporal analysis, feature analysis, disaster management

Date of Defense

8-31-2015

Abstract

Thanks to the advanced technologies and social networks that allow the data to be widely shared among the Internet, there is an explosion of pervasive multimedia data, generating high demands of multimedia services and applications in various areas for people to easily access and manage multimedia data. Towards such demands, multimedia big data analysis has become an emerging hot topic in both industry and academia, which ranges from basic infrastructure, management, search, and mining to security, privacy, and applications. Within the scope of this dissertation, a multimedia big data analysis framework is proposed for semantic information management and retrieval with a focus on rare event detection in videos. The proposed framework is able to explore hidden semantic feature groups in multimedia data and incorporate temporal semantics, especially for video event detection. First, a hierarchical semantic data representation is presented to alleviate the semantic gap issue, and the Hidden Coherent Feature Group (HCFG) analysis method is proposed to capture the correlation between features and separate the original feature set into semantic groups, seamlessly integrating multimedia data in multiple modalities. Next, an Importance Factor based Temporal Multiple Correspondence Analysis (i.e., IF-TMCA) approach is presented for effective event detection. Specifically, the HCFG algorithm is integrated with the Hierarchical Information Gain Analysis (HIGA) method to generate the Importance Factor (IF) for producing the initial detection results. Then, the TMCA algorithm is proposed to efficiently incorporate temporal semantics for re-ranking and improving the final performance. At last, a sampling-based ensemble learning mechanism is applied to further accommodate the imbalanced datasets. In addition to the multimedia semantic representation and class imbalance problems, lack of organization is another critical issue for multimedia big data analysis. In this framework, an affinity propagation-based summarization method is also proposed to transform the unorganized data into a better structure with clean and well-organized information. The whole framework has been thoroughly evaluated across multiple domains, such as soccer goal event detection and disaster information management.

Identifier

FIDC000146

Recommended Citation

Yang, Yimin, "Exploring Hidden Coherent Feature Groups and Temporal Semantics for Multimedia Big Data Analysis" (2015). FIU Electronic Theses and Dissertations. 2254.
https://digitalcommons.fiu.edu/etd/2254