Evaluating Statistical Claims Pattern - Signal Vs Noise

Separating the key info about a study's design from distracting details

These questions give you a study where the sample was drawn from a broad population, but the data is presented for a subset of that sample. A bar graph or table shows results for only part of the group — and the SAT wants to see if you'll mistake the subset for the population.

The Core Insight

Generalizability is determined by how the sample was selected, not by what data is shown. If the sample was randomly drawn from all adult residents, the results generalize to all adult residents — even if the table only shows data for non-smokers, or AP students, or discount-code users.

Example — Graph Shows a Subset

A city health department selected $800$ adult residents at random from a city-wide registry. A bar graph displays average resting heart rates for the $550$ participants who were confirmed non-smokers. To which population can the conclusions be most appropriately generalized?

A) All adult residents of the city
B) The 550 non-smokers in the sample
C) The 800 adult residents in the sample
D) All adult non-smokers in the city

The sample was drawn from all adult residents of the city.
The bar graph showing 550 non-smokers is irrelevant to generalizability.
Answer: A

Why the Subset Is Noise

The study didn't select 550 non-smokers. It selected 800 adults, and 550 of them happened to be non-smokers. The subset data is part of the analysis, but it doesn't change who the sample represents.

Think of it this way: if you randomly survey 800 people from a city and then sort the results by hair color, you can still generalize to the whole city — even though your table only shows data for blondes.

Example — Table Shows a Subset

An agricultural scientist selected $120$ corn farms at random from all corn farms in a state. The table displays information about the 85 farms that used drip irrigation. Which is the largest population to which results can be generalized?

A) The 85 farms with drip irrigation
B) The 120 farms selected
C) All corn farms in the state
D) All corn farms with above-average yield

Drawn from: all corn farms in the state. The table's 85-farm subset is noise.
Answer: C

How to Spot This Pattern

These questions always have two numbers: 1. A larger number (the sample size, e.g., 800, 120, 250) 2. A smaller number (the subset shown in the graph/table, e.g., 550, 85, 180)

If you see both numbers, ask: "From which group was the random sample actually drawn?" That's your answer — not the subset in the data display.

What to Do on Test Day

• Ignore the graph or table for the generalizability question. It's designed to pull your attention toward the subset.
• Look for the sentence with "randomly selected" or "selected at random." The population named there is the answer.
• The answer with the bigger number is usually wrong (it's the sample). The answer with the smaller number is definitely wrong (it's the subset). The correct answer describes the full population.
• Key principle: Generalizability is about the sampling method, not the data presentation.