MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model
CoRR(2024)
摘要
With the evolution of storage and communication protocols, ultra-low bitrate
image compression has become a highly demanding topic. However, existing
compression algorithms must sacrifice either consistency with the ground truth
or perceptual quality at ultra-low bitrate. In recent years, the rapid
development of the Large Multimodal Model (LMM) has made it possible to balance
these two goals. To solve this problem, this paper proposes a method called
Multimodal Image Semantic Compression (MISC), which consists of an LMM encoder
for extracting the semantic information of the image, a map encoder to locate
the region corresponding to the semantic, an image encoder generates an
extremely compressed bitstream, and a decoder reconstructs the image based on
the above information. Experimental results show that our proposed MISC is
suitable for compressing both traditional Natural Sense Images (NSIs) and
emerging AI-Generated Images (AIGIs) content. It can achieve optimal
consistency and perception results while saving 50
potential applications in the next generation of storage and communication. The
code will be released on https://github.com/lcysyzxdxc/MISC.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要