Machine Learning-based Multimodal Data Compression

Jacob Forsell () and Yuyang Jin ()

Abstract

The field of learned image compression has been experiencing rapid development and increased research engagement. In this thesis, we aim to contribute to the field by extending a state-of-the-art learned image compression architecture, called LIC-TCM, by incorporating a depth map as a second complementary modality to further enhance image compression. Additionally, we explore the inverse approach, where we primarily compress a depth map (which can be represented as an image) using LIC-TCM and incorporate the corresponding image frame as a secondary complementary modality. Based on these explorations, we propose three unique multimodal compression architectures. Our experimental results demonstrate overall improvements in compression performance and indicate a positive direction for future research.