TRACE: Transformer Residual Activation Codebook Encoding for Communication-efficient Distributed Edge Inference

TRACE is a framework for distributed Transformer inference that uses vector quantization at residual layers to eliminate downstream computation bottlenecks and…