The code base for project Layer-Condensed KV Cache, a new variant of transformer decoders in which queries of all layers are paired with keys and values of just the top layer. It reduces the memory ...

Abstract: This paper explores how cache-optimized C and Rust source code impact the performance of a critical system under radiation. Several implementations of the matrix multiplication algorithm ...