ACADEMIC WRITING GUIDE

How to Handle Missing Survey Data in a Dissertation (With Worked Examples)

By Acadelyra Editorial TeamPublished

You export your questionnaire responses and notice that some students skipped the study-hours question, others left a rating scale unfinished, and a few submitted nearly empty forms. Should you delete those responses? Replace blanks with averages? Analyse only the complete questionnaires?

There is no responsible universal answer. Missing responses can change your sample size, the precision of estimates, and sometimes the conclusions you draw. The right approach depends on what is missing, why it may be missing, which research question you are answering, and what assumptions your analysis makes.

This guide explains how to examine missing survey data, compare defensible approaches and write a transparent methods-and-results account. The worked datasets and numbers are hypothetical teaching examples, not findings from real participants. Institutional requirements, the design of your study, and qualified statistical advice should guide consequential modelling decisions.

What counts as missing survey data?

A missing value is information needed for an analysis that was not observed or cannot responsibly be used. Examples include a skipped answer, a dropout between survey waves, an unavailable administrative grade, or a response invalidated by a verified data-entry error.

Not every blank cell is genuinely missing. A participant who correctly skipped a question because it was not applicable is different from someone who was asked the question but declined to answer. Likewise, a valid response of 0 is not missing merely because another student's answer is blank.

Distinguish:

Distinguishing missing responses from valid skips and zero values
SituationExampleAppropriate first action
Item nonresponseParticipant skips the weekly study-hours questionFlag as missing for that item; investigate pattern
Unit nonresponseInvited student provides no usable questionnaireRecord separately in recruitment/response accounting
Planned skipStudent without employment is not asked hours workedMark not applicable rather than inventing zero
Invalid entryA score of 110 on a verified 0–100 scaleCheck original record; correct only with evidence
Survey attritionParticipant completes baseline but not follow-upReport wave-specific missingness and study implications

For upstream screening of invalid records, see how to clean research data before analysis. This guide instead concentrates on the analysis decisions that follow. For recruitment-level denominators, see how to calculate and report a survey response rate.

Step 1: Preserve the raw responses and create a missing-data log

Keep an unchanged, access-controlled copy of the original export. Work on a clearly named analysis copy, with identifiers protected according to the approved research-data plan. Document the survey version, eligibility criteria, skip logic, variable definitions, missing-value codes, and corrections.

For each key variable, record the number of eligible respondents who should have been asked, number observed, number missing and the percentage missing. Preserve distinctions such as declined, not applicable and not asked where the system permits.

Missing percentage = (number missing ÷ number eligible and expected to answer) × 100.

The denominator must match the question. A mandatory question shown only to students using online classes should not use every survey respondent as its denominator.

Worked example 1: Calculate missingness and the analytic sample

Suppose a hypothetical survey contains 200 eligible submissions. The study asks whether weekly independent-study hours are associated with assessment performance. The questionnaire also records perceived feedback usefulness.

Per-variable missingness among 200 eligible submissions
VariableExpected responsesObservedMissingMissing percentage
Weekly study hours2001802010.0%
Assessment score (with permission)200190105.0%
Feedback usefulness2001703015.0%

Calculations: study hours = 20 ÷ 200 × 100 = 10.0%; score = 10 ÷ 200 × 100 = 5.0%; feedback = 30 ÷ 200 × 100 = 15.0%.

But these percentages do not tell us how many people have complete data across all three fields. Missing entries may occur in the same people or different people.

Consider this incorrect initial pattern table. Its mutually exclusive counts will be checked against the variable summary before any analysis:

Incorrect initial pattern table: feedback count disagrees with the variable summary
PatternParticipants
All three variables observed155
Study hours missing only15
Assessment score missing only5
Feedback usefulness missing only20
Both study hours and assessment score missing5
Total200

Check the column totals: study hours missing = 15 + 5 = 20; assessment scores missing = 5 + 5 = 10; feedback usefulness missing = 20, not 30. This reveals an inconsistency between the first summary and the pattern table. Before analysis, the researcher must reconcile the data—not choose whichever number looks convenient.

Here is the corrected pattern table matching the stated variable totals:

Corrected and verified missingness patterns: 200 eligible submissions
Verified patternParticipants
All three observed145
Study hours missing only15
Assessment score missing only5
Feedback usefulness missing only30
Both study hours and assessment score missing5
Total200

Now all totals agree: study hours missing 20, assessment scores missing 10, feedback usefulness missing 30. The complete-case sample for a model including all three variables is 145/200 = 72.5%. Therefore 55/200 = 27.5% would be excluded by listwise deletion for that specific model—even though no individual variable exceeds 15% missing.

For a simpler model using only study hours and assessment score, an initial claim of 170 eligible complete pairs would be incorrect. The 30 respondents missing only feedback have both required fields; therefore 145 + 30 = 175. Check directly by counting everyone missing study hours or score: 20 + 10 − 5 overlapping = 25; 200 − 25 = 175. The apparent 170 would be an error. This is why researchers should calculate model-specific analytic N from the actual pattern, rather than subtracting percentages mechanically.

What this example teaches: tabulate patterns, check arithmetic across tables, and report analytic sample sizes for each analysis. Do not assume that the number of complete questionnaires is the same as the number usable for every research question.

Step 2: Examine whether missingness is patterned

Compare respondents with and without missing values on variables that are actually observed—for example, programme, survey wave, recruitment route or baseline measures. Review whether skipped responses cluster near the end of a long questionnaire, appear after a sensitive question, or occur disproportionately in a particular group.

A missingness indicator (1 = missing, 0 = observed) can help with descriptive comparisons. These checks generate evidence about possible reasons for missingness; they do not reveal the unknown answers.

A small sample may offer limited ability to detect patterns, and an insignificant test of association does not prove the absence of bias. Use subject knowledge and survey-process evidence alongside statistics.

Step 3: Understand MCAR, MAR and MNAR—without pretending you can prove which applies

Missing-data mechanisms describe assumptions about why values are missing.

Missing completely at random (MCAR): the chance a value is missing does not depend on observed or unobserved values in the relevant setting. Example: a genuinely random technical interruption loses selected responses regardless of participant characteristics or answers.

Missing at random (MAR): after conditioning on suitably observed information, the chance of missingness does not additionally depend on the unseen value. Example: older students skip a study-hours question more often, but the relevant difference is explainable by observed age and other modelled variables. MAR does not mean blanks occur randomly with no pattern.

Missing not at random (MNAR): missingness still depends on the unobserved information even after accounting for observed variables. Example: respondents with exceptionally low academic confidence are less willing to answer a confidence question, even among respondents with otherwise similar measured characteristics.

These are simplified conceptual examples, not diagnoses of an actual dataset. With several partly observed variables, missingness can have more complex patterns. Crucially, MAR versus MNAR cannot generally be established conclusively from the observed data alone. Researchers therefore justify plausible assumptions and examine what happens when those assumptions change.

The methodological article Missing data: A statistical framework for practice emphasises linking assumptions, analysis and sensitivity checks rather than treating labels as substitutes for reasoning.

Step 4: Choose a method that matches your estimand and assumptions

An estimand is the particular quantity you want to estimate—such as the average examination score among eligible respondents or an adjusted association between study hours and performance. You need to know what answer you want before choosing a missing-data method. Our dissertation data analysis planning guide helps align each research question, variable and intended analysis.

Missing-data approaches and their assumptions
ApproachWhat it doesKey caution
Complete-case analysisAnalyses records with every variable required by the model observedCan lose substantial information; may introduce bias depending on missingness and analysis
Available-case summariesUses available observations for each separate statisticDenominators change; combining estimates can be difficult
Single-value fillingReplaces missing entries with one fixed value, e.g., the sample meanOften distorts variation, relationships and uncertainty; generally poor default
Multiple imputation (MI)Generates multiple plausible completed datasets, analyses each, then pools estimatesDepends on a suitable imputation model and assumptions; not a magic fix
Likelihood-based approachesEstimate model parameters using observed data under relevant model assumptionsRequires an appropriate model, design and missingness assumptions
Sensitivity analysisRepeats analyses under meaningful alternative assumptionsTests robustness; does not identify the missing truth

Do not automatically delete every incomplete questionnaire. But do not automatically impute everything either. The proportions, pattern, variable types, study size, plausible causes of missingness and target model matter more than a simplistic universal cutoff. For the wider workflow, see how to analyse quantitative data.

UCLA's multiple-imputation teaching guide contrasts complete-case approaches with imputation and explains how analytic samples can shrink. These principles apply beyond any one software package, although specific commands differ.

Why replacing blanks with the mean can mislead

Consider the observed scores 2, 4 and 6, with one additional score missing. The observed mean is 4. Filling the missing score with 4 produces 2, 4, 6, 4, still with mean 4. But it adds no variability for the filled-in case and pretends the unknown response is known to be average. In real analyses, this can understate uncertainty and distort relationships between variables. Do not treat that fill-in as genuine evidence.

What multiple imputation actually does

Under appropriate assumptions and with a compatible model, multiple imputation generates several plausible values rather than a single apparently certain replacement. The analyst fits the intended model to each completed dataset and pools coefficient estimates and their within- and between-imputation uncertainty using the applicable combining rules. A complete implementation must respect variable types, dependencies, interactions, clustering where relevant, and the final analysis model.

It is not responsible to promise that MI always removes bias or is mandatory whenever one question is skipped. It cannot independently establish MAR or repair severe problems with representativeness.

Worked example 2: How missingness can change a reported mean

Suppose 10 hypothetical eligible participants were intended to provide a confidence score from 0 to 10. Eight observed scores are 4, 5, 6, 6, 7, 7, 8, 9; two participants did not answer.

The observed total is 52, so the available-case mean is 52 ÷ 8 = 6.5. Two of ten scores are missing (20%).

A simplistic interpretation would say that the average score of all ten students was 6.5. That claim is not supported: the mean of the eight observed responses is 6.5, but the two missing scores are unknown.

Because the valid scale runs from 0 to 10, the possible full-sample means range from (52 + 0 + 0) ÷ 10 = 5.2 to (52 + 10 + 10) ÷ 10 = 7.2. These are mathematical bounds under the stated scale, not probabilities or confidence intervals. They demonstrate what the unobserved values could change, not which values are most plausible.

Hypothetical confidence-score bounds and illustrative scenarios
IllustrationValues assumed for two missing scoresFull-sample mean
Lower bound0 and 05.2
Midpoint illustration5 and 56.2
Upper bound10 and 107.2
Observed cases onlyMissing cases excluded6.5 (n = 8)

The midpoint scenario is not a recommended imputation method. Its sole purpose is to make uncertainty visible. For a dissertation, explain whether the target is the observed-respondent mean or an estimate for all eligible participants, and identify the assumptions needed for the latter.

Step 5: Evaluate whether the analysis is robust

A sensitivity analysis asks whether a substantive conclusion changes when a plausible alternative missing-data assumption or analysis method is used.

For example, a researcher might compare a pre-specified complete-case model with an appropriately specified MI analysis, while explaining the assumptions of both. If missingness may depend on unseen scores, a justified MNAR sensitivity scenario may shift imputed values or examine informative bounds. Do not select alternative analyses solely because they improve a p-value.

Suppose a hypothetical adjusted association is estimated at 0.30 units in a complete-case model and 0.18 units in a suitable MI model. Those figures alone are not proof that one estimate is correct. The researcher should compare precision, analytical samples, model specifications and reasons why methods differ. If the substantive interpretation changes, that uncertainty belongs in the discussion.

Sensitivity checks are especially important when substantive claims depend on uncertain missingness assumptions. Research on assumptions and analysis planning warns that missing-data mechanisms for multiple variables can be more complex than simple labels suggest.

Step 6: Report the decisions so readers can trace them

A good dissertation does not hide missingness behind the phrase “incomplete responses were removed.” A reader should know:

  1. What data were expected and which variables were partly observed.
  2. The number and percentage missing, using correct denominators.
  3. Missingness patterns relevant to the analytical model.
  4. Which records each analysis used and why.
  5. The method selected and its central assumptions.
  6. Any imputation or likelihood model details needed for reproducibility.
  7. Sensitivity checks and material differences in results.
  8. Limitations arising from data that were never observed.

For further guidance on describing estimates and uncertainty, consult how to report statistical results in a dissertation.

Worked example 3: Complete dissertation-style methods and results passages

Hypothetical scenario: A cross-sectional survey of 200 eligible undergraduates examines study hours and assessment scores, with feedback usefulness considered as an additional predictor. The missingness counts and verified patterns are those in Worked Example 1. The study is observational and cannot establish causation.

Methods paragraph — planned or approved analysis:

Questionnaire records will be screened using the survey codebook, including checks for eligibility, skip logic and designated missing-value codes. For each analysis variable, the number and percentage of missing responses will be reported. Patterns of missingness will be examined descriptively against available participant characteristics. The primary association model will use a prespecified complete-case analysis, with its effective sample size disclosed. This choice will be discussed in light of potential loss of precision and selection bias. If justified by the amount and pattern of missingness and an adequately specified model, a separate multiple-imputation sensitivity analysis will be considered. The study will not claim to establish MCAR or MAR conclusively from observed responses.

Results paragraph — completed-study illustration, using only verified hypothetical counts:

Of 200 eligible survey submissions, 180 contained study-hours data (90.0%), 190 contained assessment scores (95.0%) and 170 contained feedback-usefulness responses (85.0%). A complete-case model requiring all three measures included 145 participants (72.5% of eligible submissions). The study-hours/assessment-score model, which did not require feedback usefulness, included 175 participants (87.5%). These differing analytic sample sizes arose from the overlap of missing responses across variables, not from an additional eligibility rule. Estimates from the statistical models are not reported in this teaching example because no model has been fitted to actual data.

Limitations paragraph:

Missing responses may have affected both the precision and the representativeness of the observed associations. Descriptive checks using available variables cannot rule out differences related to unobserved values. Any inferential conclusions should therefore be interpreted in relation to the analytic sample, plausible missingness mechanisms and the results of justified sensitivity analyses. As a cross-sectional observational study, the research cannot establish that study hours caused differences in assessment performance.

The sample counts above are internally reconciled. The example intentionally does not invent a regression coefficient, significance test, imputed estimate or statistical power calculation.

Common mistakes when handling missing survey responses

Turning every blank into zero. A legitimate zero, a planned skip and an unanswered item mean different things.

Using “5% missing” as an automatic rule. No single percentage can determine the best method without reference to the design, mechanism, variable and research question.

Assuming all incomplete respondents are unusable. A respondent may contribute to one analysis but not another, depending on its required variables.

Reporting the recruited N as every model's N. Identify the actual analytic denominator rather than borrowing the invitation or submission total.

Treating a nonsignificant missingness test as proof of MCAR. Limited evidence of a pattern is not proof that no meaningful pattern exists.

Imputing without preserving uncertainty. Filling blanks with a single mean can falsely suggest precision.

Calling imputation recovered data. Model-generated values depend on assumptions and cannot replace information that was never measured.

Suppressing differences between sensitivity analyses. If alternative defensible choices change the conclusion, that is important evidence about robustness.

Inventing completed methods in a proposal. Use future tense for planned procedures and accurate past tense for work genuinely performed.

Missing-data checklist for dissertation students

  • [ ] Preserve a secure original dataset and document all analysis versions.
  • [ ] Distinguish missing, not applicable, declined and valid zero values.
  • [ ] Calculate per-variable missing counts with correct denominators.
  • [ ] Create and reconcile missingness-pattern tables.
  • [ ] Identify the analytic N separately for each question or model.
  • [ ] Examine possible patterns using recorded variables and survey context.
  • [ ] Justify the chosen method and its assumptions; do not rely on a universal cutoff.
  • [ ] Conduct proportionate sensitivity checks where conclusions depend on uncertainty.
  • [ ] Report exclusions, modelling choices and limitations honestly.
  • [ ] Follow supervisor, institutional, ethical and relevant disciplinary requirements.

Frequently asked questions

Should I delete participants who skipped one survey question?

Not automatically. Determine whether the question is needed for the specific analysis, whether the skip was valid, and how exclusion would affect the sample and possible bias. Document the decision.

Can I replace missing questionnaire answers with the average?

Mean filling may seem easy but commonly distorts variation and ignores uncertainty. It is rarely a defensible default for inferential analyses. Seek a method suited to your design and research question.

Is a dataset with 10% missing data unusable?

No universal percentage decides usability. Ten percent missing in a peripheral variable is different from ten percent missing in a primary outcome concentrated among particular participants. Examine patterns, assumptions and consequences.

Can a test prove my data are missing at random?

No. Statistical checks can identify some patterns in observed information, but MAR and MNAR assumptions generally cannot be distinguished definitively from observed data alone.

Do I always need multiple imputation?

No. Some designs and estimands support appropriate complete-case or likelihood-based approaches; others may benefit from MI. The method must be justified rather than chosen automatically.

Where should I discuss missing data in my dissertation?

Usually in the methodology or analysis subsection, results where sample sizes and estimates are reported, and limitations where missingness materially affects interpretation. Follow your university's structure.

Final thoughts

Missing survey data are not merely untidy spreadsheet cells. They raise questions about who is represented, what can be estimated and how confident you can be in your conclusions. Start by describing what is absent; then make a reasoned, documented decision about how to analyse what remains.

A transparent analysis does not promise perfect data. It shows readers exactly how incomplete evidence was handled and where uncertainty remains.

If you need help improving your dissertation's methods explanation, data-reporting clarity or alignment between research questions and analysis, explore Acadelyra's academic-support services. Your research decisions, underlying data and academic responsibilities remain your own.

Further methodological reading