B. Outlining the hypothesis and research question

From informal claim to formal question

The first step of II.A asked for a central claim stated informally, in whatever language was sufficient to fix a target for later drafting. This section formalizes that claim into two related but distinct objects: a research question, which states what the article sets out to answer, and a hypothesis, which states a candidate answer to it in advance of the evidence. Formalizing the claim this way is not a matter of polish. A research question and hypothesis stated precisely enough can be checked directly against the Method, Experiments, and Results that follow, which is exactly the kind of check the informal claim from II.A was too loosely stated to support.

Research question and hypothesis are not the same object

The research question is what the article investigates; the hypothesis, where one is offered, is a specific, falsifiable prediction about what that investigation will find. Every article needs the former. Not every article needs, or should force, the latter. A research question can be posed and honestly investigated without a predicted answer — “does model behavior on task X depend on factor Y” is a complete research question on its own — and forcing a hypothesis onto a genuinely open question tends to produce one of two failures: a hypothesis vague enough to be unfalsifiable, which contributes nothing beyond the question it restates, or a hypothesis quietly written after the results were already known, which reads as a prediction but was not one. This second failure has a name in the methodological literature, hypothesizing after the results are known, and it is worth naming here because it is difficult to detect in one’s own writing — the hypothesis reads perfectly naturally, since it was written to fit the results, and only the discourse-preparation step of II.A, done honestly and before the Results are drafted, catches it.

Matching the question to the structural variant

Section I.B distinguished three common structural variants, and the research question takes a different shape in each.

In the system/method paper, the hypothesis is usually implicit and comparative: that a specific design choice improves measured performance relative to the strongest available baseline. It is rarely stated as a formal hypothesis in the prose, but it is exactly what the ablation study described in I.B is designed to test, and outlining it explicitly during preparation — even if it is later folded into the Introduction’s contribution statement rather than given its own sentence — keeps the ablation study honest about what it is meant to isolate.

In the empirical/analysis paper, the research question is foregrounded rather than implicit, and the hypothesis, when present, is often a prior assumption held by the field that the article sets out to test rather than a prediction the authors are confident in. Section I.B’s worked example, the HANS diagnostic study, is built on exactly this shape: the research question is whether NLI models perform genuine entailment reasoning, and the tested hypothesis is the field’s implicit assumption that high accuracy on standard benchmarks reflects that reasoning — a hypothesis the article is specifically designed to be capable of refuting.

In the resource/dataset paper, the governing question is usually not predictive at all: not “what will happen” but “is this resource adequate for its intended purpose.” This is still a real, checkable question — adequacy can be operationalized through inter-annotator agreement, coverage statistics, and baseline results, as described in I.B — but treating it as a hypothesis in the predictive sense forces an artificial framing onto a question that is properly one of quality and fitness for use.

Properties of a well-formed research question or hypothesis

Four properties distinguish a research question or hypothesis ready to anchor an article from one that still needs work.

  • Falsifiable. There must exist a result — a number, a pattern, a comparison — that would count as evidence against the hypothesis, or against a particular answer to the research question. A question or hypothesis compatible with any possible result is not yet doing any work.
  • Singular. A research question that bundles several independent questions under one heading (“does X improve performance, and why, and does it generalize”) should be separated into its components, each checkable against its own piece of evidence, rather than left to be answered all at once by a single set of results.
  • Appropriately scoped. The question should be broad enough to be worth answering and narrow enough that the Method and Experiments actually planned can answer it; a question scoped to “how should NLP models handle ambiguity” is unlikely to be answerable by any single article’s experiments, however good they are.
  • Motivated by a stated gap. The question should be traceable to a specific absence in existing work — the same gap that Related Work’s bounding function (I.C) will later make explicit — rather than appearing without explanation, since a question’s importance is established by what it is missing an answer to, not asserted on its own.

Four properties of a well-formed research question or hypothesis: falsifiable, singular, appropriately scoped, and motivated by a stated gap.

A worked example

Compare a well-formed version of the HANS research question against a flawed one that fails each property in turn. Well-formed: “Do NLI models rely on syntactic heuristics — lexical overlap, subsequence, and constituent structure — rather than entailment reasoning, such that their accuracy collapses on cases engineered to decouple those heuristics from correct labels?” This is falsifiable (a heuristic-targeting test set that models pass would count against it), singular (one mechanism, one predicted failure pattern), appropriately scoped (answerable by constructing and evaluating on one diagnostic set), and motivated by a stated gap (prior work had already raised concerns about dataset artifacts without isolating specific heuristics this precisely). Flawed: “Do NLI models understand language well, and why, and could this be improved?” This version is unfalsifiable (no single result would settle “understand language well”), bundled (three questions — performance, explanation, improvement — masquerading as one), unscoped (no experiment answers “understand language” in general), and unmotivated (no specific absence in existing work is named). The flawed version could still motivate a research program, but it is not yet a question a single article’s Experiments section could be built to answer.

Where the hypothesis and research question function in the article

The hypothesis or research question outlined at this stage is not itself a section of the prototype from I.A; it is the seed from which several claim-bearing components in I.C’s taxonomy are grown. It becomes the center of the Introduction’s preview of the contribution, it disciplines the Experiments section by fixing in advance what result would and would not support the claim, and it gives the Conclusion something specific to close against — the same correspondence between Introduction and Conclusion whose absence I.C flagged as a diagnostic of an unprepared discourse. Outlining it precisely now, before any of those sections is drafted, is what makes that later correspondence possible rather than accidental.


This site uses Just the Docs, a documentation theme for Jekyll.