Our crawler
App TXT reads only public files: /app-ads.txt on developer websites and /sellers.json of ad systems. It never crawls other pages and never submits forms.
User agents
AppTXT-Autodetect/1.0 (+https://app-ads.app/bot) AppTXT-Sellers/1.0 (+https://app-ads.app/bot)
- AppTXT-Autodetect reads https://<domain>/app-ads.txt (the www. address and plain HTTP as fallbacks).
- AppTXT-Sellers reads https://<ad system domain>/sellers.json.
- If a site refuses the first request (a firewall blocking unknown bots), the app-ads.txt read is retried once with a regular browser user agent.
How often
- app-ads.txt — once a night per checked domain, a few retries during the day if the read fails, and when someone checks the domain on app-ads.app.
- sellers.json — once a week per ad network (the very large ones less often).
- One file per request, a few domains in parallel; requests time out after 15–30 seconds.
Allowing the crawler
app-ads.txt is meant to be public: ad systems and buyers must be able to read it. If your firewall or CDN blocks bots, allow the user agents above for /app-ads.txt (and /sellers.json for ad systems). A blocked read shows on your domain page as “blocks automated access” — it isn’t counted as lost revenue, but we can’t check your file.
Contact
Questions or a request to stop reading your domain: [email protected]. See also the methodology.
Related guides
More
- app-ads.txt generator & hosting
- Top App Publishers
- Top Publishers by Errors
- Top Publishers by Missing Lines
- Top Publishers by Encoding Problem
- Top Ad Optimised Publishers
- Top Publishers by Access Errors
- Top Publishers by Broken app-ads.txt Format
- Recent App-Ads.txt Validations
- Publisher Monetisation Intelligence
- Ad networks in app-ads.txt
- Top Advertising Platforms
- Methodology