Choose an App Store experiment when keyword scores and downloads disagree

A promising keyword score, a higher observed position and a weak download total are not three versions of the same measurement. Before changing the title again, decide which uncertainty you are trying to resolve: relevance, discoverability, the product explanation or an incomplete acquisition record.

The useful output is one justified next investigation, not a list of fashionable ASO tactics. A candidate phrase may deserve research without proving demand. A screenshot may deserve review without proving that it caused the download gap. Keep those distinctions visible before investing more effort.

Label the evidence before interpreting it

Record a third-party keyword score with its provider, definition, observation date and market. If the scoring method is unavailable, say so. Do not convert a proprietary value into a count of people searching, and do not assume that two tools use equivalent scales.

Record an observed rank separately: exact query, storefront, language, device or surface, observation time and whether the result was actually inspected. A single position is a contextual observation, not a traffic forecast. An unexplained screenshot of a rank chart is not enough to establish what prospective users saw.

Finally, use the actual acquisition report with its named metrics, filters and reporting period. These records can inform one another, but they cannot be added into a single popularity score without a defensible method.

A developer’s December 2025 forum question illustrates the confusion: the founder asks how to interpret tool volume and difficulty values after low initial traction. A later reply promotes its author’s own audit product. That vendor contribution is not another independent buyer question or an endorsed diagnosis.

Reject a relevance mismatch before chasing the score

Apple’s search guidance describes both metadata relevance and user behavior as factors. It asks developers to use accurate, specific terms and excludes irrelevant terms and competing app names. This does not validate a third-party score or guarantee a rank for a chosen phrase.

Consider the fictional scanner app PageFold. A tool assigns an attractive score to a phrase about a capability PageFold does not provide. The correct editorial decision is to reject that phrase, not to imply the capability in the subtitle and hope downloads justify it.

A second phrase accurately describes a supported document task. It becomes a candidate for investigation. The team still needs to understand whether its intended audience uses that wording and whether the current store explanation matches the task. “Accurate” and “popular” are different properties; neither should be invented to make the shortlist look complete.

Preserve the rejected alias and the reason. A later researcher should not reintroduce the same unsupported phrase because its tool score looks appealing.

Reconcile the report before naming the bottleneck

Apple’s metric definitions define displayed conversion using total downloads and pre-orders divided by unique-device impressions. A pre-order is not counted again when downloaded. Total downloads include first-time downloads and redownloads. Product-page views are a different measure, and usage reporting relies on analytics-sharing participation.

Do not divide downloads by product-page views and label the result Apple’s displayed conversion rate. You may define another analysis, but name its numerator, denominator and limitations explicitly. Keep acquisition, later useful work and payment separate.

Apple also notes in its search-performance guidance that Apple Ads activity appears in source-filtered search reporting. “Search” therefore cannot automatically be relabeled “organic.” Record paid activity and source definitions before attributing a change to metadata.

For PageFold, compare records from the same storefront and period, with the applicable download composition visible. If the necessary impressions or source details are missing, the next action is to resolve that evidence gap. It is not yet defensible to blame the screenshots, price or keyword selection.

Even complete descriptive records do not establish cause. An app with exposure and few downloads may have several competing explanations. Turn those into testable questions instead of presenting a confident diagnosis from a small chart.

Distinguish a metadata observation from a creative experiment

Apple’s product page optimization documentation describes randomized treatments for icons, screenshots and app previews. It does not describe title or keyword rewrites as randomized treatments in that system. A before-and-after title change is a different study with different limitations.

Keep a dated metadata history: what changed, when it became available and what else changed around it. If title, screenshots, price and external promotion all moved together, do not attribute a later rank or download change to one element. A tidy retrospective story is not a controlled comparison.

The next proposal may concern creative rather than metadata. For example, an intended user might misunderstand which document task PageFold supports. A reviewed screenshot hypothesis could make that task clearer while preserving factual accuracy. Preparing such a treatment is not permission to publish or start a test.

Worked example: higher rank after several changes

Imagine the fictional walking-route app LoopWalk. Its team changes the title and screenshots during the same period, then observes a better position for one query. Downloads remain sparse. This fictional case provides no evidence that either change caused the position or that the query carries substantial demand.

The team first preserves the exact changes and report context. It then identifies a narrower explanation question: do intended walkers understand that the app offers a short selected route, rather than requiring them to inspect a crowded map of every nearby place?

An authorized task-comprehension review could reveal what people believe the page offers. That would be qualitative evidence about interpretation, not a conversion estimate. If a supported randomized creative comparison later becomes appropriate, its hypothesis should concern one clear factual presentation difference. Do not simultaneously relabel the experiment as a test of the new title.

A separate founder discussion reports multiple title and screenshot changes alongside higher rank but limited downloads. Commenters offer explanations, but the actual analytics and image evidence have not been independently inspected here. The discussion supports the existence of the question, not the cause of that app’s result. Its retrieved date signals conflict, so this article does not present it as a verified current-week trend.

Respect the possibility of an inconclusive test

The product-page-optimization guide uses existing performance to estimate a test’s duration and shows when more data is needed. More treatments can take longer to distinguish. Its published examples also show that a plausible creative hypothesis can lose. Those examples are not reusable uplift forecasts.

Do not invent a sample threshold, expected improvement or test duration for a low-traffic app. Review the applicable current account information when authorized. If there is insufficient evidence for a meaningful comparison, a narrower comprehension review or a held decision can be more honest than launching a test and declaring a winner from noise.

Do not buy traffic simply to make an unfinished research plan look statistically complete. Spending, audience selection and experiment execution are separate decisions. Neither a high tool score nor this worksheet authorizes them.

End with one next action and a reason

Use the blank ASO decision worksheet to select one outcome: reject an inaccurate phrase, recover a missing acquisition record, investigate a specific explanation problem, or prepare an eligible creative hypothesis. Name what evidence could change that decision.

For a metadata observation, retain the date and concurrent changes. For a creative proposal, retain the treatment scope and what would remain unknown even if it performed differently. For an unresolved report, keep the gap visible rather than filling it with modeled demand.

Later distinguish prepared, reviewed, submitted, served, experimentally observed, downloaded, useful-task completed and paid outcomes. No listing, campaign or experiment has been changed here. The immediate improvement is a clearer decision about what to learn next, not a claim that the app now ranks or converts better.