What the margin of error is
A survey result is an estimate of something you cannot see. The margin of error is how far the estimate could sit from the truth and still be the kind of thing this sample would produce. Four percent of respondents said yes, plus or minus three points, means the data is compatible with anything from one percent to seven.
The formula is the standard one for a proportion: the z for your confidence level, times the square root of p times one minus p over the sample size. Everything else on this page follows from it.
Two things about it get forgotten in the room where the result is presented. The margin belongs to each question separately, so a survey with twenty questions has twenty margins. And the margin is widest at fifty percent, which is why a calculator that asks what share you expect and gets no useful answer should assume half, as this one does.
Why a bigger group barely needs a bigger sample
This is the single most expensive misunderstanding in survey work, and it is the reason for the table above.
Precision comes from the number of answers, almost never from the number of people who could have given them. Four hundred answers describe a group of a hundred thousand about as well as they describe a group of ten thousand. The exception is small groups, and it goes the other way: when the sample is a large share of the whole group, there is less left to be uncertain about, and the answers you need go down.
That exception is the finite population correction, and most calculators leave it out because they assume you are sampling a country. Research inside a company is not sampling a country. It is sampling the four hundred people who administer one tool, and leaving the correction out is how a team ends up chasing 384 responses from a population of 500 when 218 would have done.
Why the result you have needs an interval
The number that comes back from a survey is a point. It gets written on a slide as a point, argued about as a point, and compared with last quarter as a point. It is not a point, and the third block above exists to say so.
The interval most people compute is the textbook one, p plus or minus z times the standard error. It is in every statistics course and every spreadsheet, and at the sample sizes product teams work with it is wrong in a visible way: twenty-seven of thirty is ninety percent, and the textbook interval runs from seventy-nine percent to a hundred and one. There is no such thing as a hundred and one percent, and an interval that produces one is not describing the uncertainty correctly anywhere near the edges.
The adjusted interval fixes it by adding roughly two successes and two failures before doing the arithmetic. That sounds like a fudge and it is not: it is Agresti and Coull’s correction, and the reason to use it is that its real coverage is close to the ninety-five percent on the label, while the textbook interval’s is often far below. The tool shows both, because the gap between them is the argument.
What this does not cover
Sampling error is the only error this page models, and it is rarely the largest one.
Who answers is usually a bigger problem than how many. A panel that skews toward engaged users will give you a precise estimate of what engaged users think, and the margin of error says nothing at all about that.
Question wording moves answers further than sample size does. So does the order the questions come in, and whether the person asking has an obvious preference.
And a sample size calculated for one question does not cover the cross-tabulation somebody will ask for in the meeting. Four hundred answers split by three segments and two regions is thirty to seventy answers per cell, and the margins on those cells are two or three times the one on the page.