Research
Quantization of Large Models with Weight Outliers and Sparsity
Researchers introduce a new post-training quantization method for large models that decomposes weights into a quantized matrix, a low-rank component, and a…
Research
#quantization#large language models#post-training quantization#model compression#sparse matrix#Llama 3.2 3B#Meta Platforms#Llama 3.1 8B#Inference