Motif Mining: Finding and Summarizing Remixed Image Content

Document Type

Conference Proceeding

Publication Date

1-2023

Publication Title

Proceeding of the IEEE Winter Conference on Applications of Computer Vision

Pages

1319-1328

Publisher Name

Institute of Electrical and Electronics Engineers

Publisher Location

Waikoloa, HI

Abstract

On the Internet, images are no longer static; they have become dynamic content. Thanks to the availability of smartphones with cameras and easy-to-use editing software, images can be remixed (i.e., redacted, edited, and re-combined with other content) on-the-fly, allowing a world-wide audience to repeat the process many times. From digital art to memes, the evolution of images through time is now an important topic of study for digital humanists, social scientists, and media forensics specialists. However, because typical data sets in computer vision are composed of static content, there has been limited development of automated algorithms for analyzing remixed content. In this paper, we propose the idea of Motif Mining: the process of finding and summarizing remixed image content in large collections of unlabeled and unsorted data. For the first time, this idea is formalized and a reference implementation grounded in that formalism is introduced. We conduct experiments on three meme-style data sets, including a newly collected set associated with the Russo-Ukrainian conflict. The proposed motif mining approach is able to identify related remixed content that, when compared to similar approaches, more closely aligns with the preferences and expectations of human observers.

Identifier

978-1-6654-9346-8

Comments

Author Posting © ©️2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The definitive version was published in the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Proceedings, pp. 1319-1328 2023. https://doi.org/10.1109/WACV56688.2023.00137

Creative Commons License

Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.

Link to Full Text

Share

COinS