How App TXT reads app-ads.txt files

App TXT started as an app-ads.txt generator; the domain pages, rankings and Intelligence grew out of the same parser. This page describes that parser and the nightly robot around it — in the order a file goes through it.

Which domains we look at

A domain gets a page the first time anyone checks it on app-ads.app, or when we add it to our own list of app publishers. Ad systems get pages of their own on /networks — the ones we track, plus any ad system that at least three checked sites name in their files. Ad systems are never ranked next to publishers, even when they publish an app-ads.txt themselves.

The nightly read

Every domain page is refreshed once a night by our crawler (what it sends and how to allow it). It asks for https://<domain>/app-ads.txt, falls back to the www. address and to plain HTTP, and only follows redirects that still end on an app-ads.txt. A read that fails is tried again later the same day, and anyone can force a fresh read with the Re-check button on the page.

Each night leaves a mark for the day: loaded, down — the file couldn’t be fetched — or blocked — the site answered our crawler with 401, 403, 429 or a bot challenge. Down days are what cost a publisher money: for that share of the month buyers couldn’t verify its sellers. Blocked days are shown on their own and never counted as a loss, since ad systems use crawlers of their own.

A file, line by line

The parser decodes the file first (UTF-16 and other encodings included — and says so, because many ad system crawlers only read UTF-8), then reads every line against the IAB app-ads.txt format. A short file, with what the parser says about each line written above it:

# valid — Google AdMob, the publisher’s own account
google.com, pub-1234567890123456, DIRECT, f08c47fec0942fa0
# valid — a partner (reseller) line
pubmatic.com, 156931, RESELLER, 5d62403b186f2ace
# invalid — the fields must be separated by commas
unity3d.com 1234567 DIRECT
# duplicate of the first line
google.com, pub-1234567890123456, DIRECT, f08c47fec0942fa0
# a variable — the company that owns the apps
OWNERDOMAIN=studio.com

A response that isn’t a text file at all — an HTML page, an XML document, a redirect somewhere else, or nothing that parses as a record — is reported as a broken format rather than as a pile of invalid lines. Two quieter hints are added when they apply: no OWNERDOMAIN, and a file that is only reachable without working HTTPS.

Counting missing lines

This is the part that comes from the generator. For every ad network we keep a template: its own DIRECT line plus the RESELLER lines of the exchanges it works with, refreshed daily from the network. A network counts as used when the file has the publisher’s own DIRECT account with it. The template lines of the used networks that the file doesn’t contain are its missing lines — the number shown on the domain page and in the missing-lines ranking. Networks only present through someone else’s partner lines don’t add to it.

From numbers to a verdict

Duplicates are counted and shown, but on their own they don’t change the verdict: buyers read a repeated line once.

sellers.json, briefly

Once a week the robot downloads the sellers.json of each tracked ad network and reads it as a stream. It keeps only a summary — how many sellers, publishers, intermediaries and confidential entries — and the entries of the seller IDs that already appear in some checked app-ads.txt; the file itself is thrown away. Very large files, over 12,000 sellers (Google’s among them), are refreshed by hand. What we look for: an account the network doesn’t list, a DIRECT line for an account the network calls an intermediary, an ad system domain that no longer resolves, and a network without a sellers.json. The seller’s name and domain are also what names the owner on an Intelligence account page.

How the rankings move

The size, errors, missing-lines and encoding rankings are rebuilt every six hours from the latest reads; access errors, broken format and the ad-optimised list follow the nightly read. A site whose file disappears drops out of every ranking and comes back on the first read that finds the file again.

Shared accounts

Publisher Monetisation Intelligence links two sites when both files carry the same DIRECT account of a mediation network (AdMob, AppLovin, Unity, ironSource, Meta and a few more). We note the day the shared line appears and the day one side drops it. An ID found on more than 150 sites is a network-wide ID, not a link, and is left out. Files that name the same OWNERDOMAIN are labelled as one owner, the same MANAGERDOMAIN as one manager. A shared line tells you two files contain the same account — whether that means a publishing deal, an agency or a test is for you to judge.

Reusing our numbers

FAQ

How often is a domain re-checked?

Every night, with a retry later in the day if the read failed. The Re-check button on a domain page reads the file again at once.

My page shows a mistake — what do I do?

Fix the file and press Re-check: the page updates immediately. If the parser itself got something wrong, use “Wrong or missing line?” on the domain page or write to [email protected]. For ownership questions, adding OWNERDOMAIN to your app-ads.txt settles most of them.

Why is my site marked as blocking bots?

Our crawler got 401, 403, 429 or a challenge page instead of the file. It doesn’t count as lost revenue, but check that ad system crawlers aren’t blocked the same way — the crawler page lists what to allow.

Why aren’t ad networks in the publisher rankings?

Their own app-ads.txt describes their inventory, not an app publisher’s, so comparing them with studios would be misleading. They have their own pages and the Top Advertising Platforms list instead.

Do you store sellers.json files?

No. We keep a summary of each file and the entries of seller IDs that appear in checked app-ads.txt files, and download it again on the next weekly read.

Is the data free to use?

Yes, under CC BY 4.0 with credit to App TXT (app-ads.app). Intelligence links aren’t part of it.

Related guides

All app-ads.txt guides

More