"Pruning One More Token is Enough: Leveraging Latency-Workload Non-Line" by Nick Eliopoulos, Purvish Jajal et al.
 

Document Type

Conference Proceeding

Publication Date

4-8-2025

Publication Title

2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

Pages

7153 - 7162

Publisher Name

IEEE

Abstract

This paper investigates how to efficiently deploy vision transformers on edge devices for small workloads. Recent methods reduce the latency of transformer neural networks by removing or merging tokens with small accuracy degradation. However these methods are not designed with edge device deployment in mind: they do not leverage information about the latency-workload trends to improve efficiency. We address this shortcoming in our work. First we identify factors that affect ViT latency-workload relationships. Second we determine token pruning schedule by leveraging non-linear latency-workload relationships. Third we demonstrate a training-free token pruning method utilizing this schedule. We show other methods may increase latency by 2-30% while we reduce latency by 9-26%. For similar latency (within 5.2% or 7ms) across devices we achieve 78.6%-84.5% ImageNet1K classification accuracy while the state-of-the-art Token Merging achieves 45.8%-85.4%.

Comments

Author Posting © IEEE, 2025. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The definitive version of this work was published at https://doi.org/10.1109/WACV61041.2025.00695.

Available for download on Thursday, April 08, 2027

Plum Print visual indicator of research metrics
PlumX Metrics
  • Usage
    • Downloads: 5066
    • Abstract Views: 29
  • Captures
    • Readers: 7
see details

Share

COinS