KVarN: Native vLLM KV-cache quantization back end by Huawei

Article URL: https://github.com/huawei-csl/KVarN

Comments URL: https://news.ycombinator.com/item?id=48399974

Points: 4

# Comments: 0