Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit

LCLMs compress LLM context before decode — 8.8x faster at 16x compression, beating every KV cache method tested. Open-sourced by NYU and Columbia.

AV2 promises 30% better compression than AV1 and costs nothing to use

The Alliance for Open Media (AOMedia) recently released version 1.0.0 of the AV2 specification and reference code. The ...