F. Methods and experimental protocol writing

Two distinct jobs under one heading

Section I.C separated the Method’s function from the Experiments section’s function even though I.A’s macro-structure lists them as adjacent components, and this section’s own heading — Methods and experimental protocol writing — preserves that same pairing without collapsing the distinction. The Method has a reconstructive function: it must let a competent reader understand, and in principle reproduce, what was done. The Experiments section has a narrower, separate function: it fixes the empirical protocol — data, baselines, metrics — independently of the outcome that protocol will produce, which is what makes the subsequent Results interpretable and falsifiable. Writing them well means respecting this distinction on the page, not just in principle: a reader should be able to tell, from where a sentence sits, whether it is describing what the system does or fixing the terms under which it will be tested.

Writing the Method: organize by reconstruction, not by discovery

Section II.A argued that the order of exposition should be decoupled from the order of discovery, and nowhere does that argument matter more directly than in the Method. A method was very rarely built in the order it is best explained; components were added, removed, and revised as problems surfaced during the actual work. The Method section should instead be organized in the order a reader would need to encounter its pieces to reconstruct the whole — the component-by-component convention I.B described for the system/method paper, moving through architecture, objective, and procedure in the sequence that makes each piece intelligible before the next depends on it, regardless of the order in which those pieces were actually developed or discovered.

A second, practical question the Method section must answer for every piece of detail is where it belongs: in the main text, because it is necessary to understand the core design, or in an Appendix, because it is configuration detail needed only for exact reproduction. Section I.C described the Appendix’s function as offloading supporting material without making it inaccessible, and the Method section is where that routing decision is actually made, sentence by sentence, while drafting.

Finally, the Method’s reconstructive function is easy to compromise by letting evaluative language creep in — describing a design choice as superior or elegant rather than simply describing what it is and how it works. Section I.C flagged this as persuasion smuggled into a component whose job is reconstruction; the practical fix is to route any comparative or evaluative claim about a design choice to Related Work, where it can be argued as a differentiation sentence (II.E), or to the Results, where it can be supported by evidence, rather than asserting it in the Method itself.

Writing the experimental protocol: fix it before the outcome

The Experiments section exists to state, before any result is reported, exactly what will count as support for or against the hypothesis from II.B. Four elements need to be fixed there. The data: what was used, how it was split, and what preprocessing was applied, stated precisely enough that another researcher could assemble the same inputs. The baselines: which existing approaches the present work is compared against, and why those specific ones represent the strongest available comparison — ideally the same clusters of prior work organized by dimension in Related Work (II.E), now instantiated as concrete, named systems. The metrics: what is actually being measured, chosen because it operationalizes the construct the hypothesis in II.B is about, not because it is the easiest number to report. And the implementation details needed to reproduce the numbers that follow — compute budget, number of runs, and how variance across runs was handled, an increasingly expected element given how much single-run results in NLP are now known to vary.

Matching Method and Experiments to the structural variant

The three structural variants from I.B shape this pair of components as directly as they shape Related Work. In the system/method paper, the Method is subdivided by architectural component and the Experiments section fixes datasets, baselines, and metrics against which every subsequent ablation is judged. In the empirical/analysis paper, the corresponding section is often a brief Setup rather than a full Method, since the models or data under study already exist and are not themselves being proposed — but its brevity is not license for imprecision, since a Setup section that is not claim-bearing in I.C’s sense still needs to be fully reconstructible; under-specifying it because it “isn’t the contribution” is a common and avoidable error. In the resource/dataset paper, the Method is replaced entirely by a description of data collection and annotation procedure, written in roughly the chronological order the resource was actually built, including the quality-control steps that will later be substantiated by the statistics reported alongside it.

Checking the protocol against the hypothesis

The Experiments section is only doing its job if it is checkable against the hypothesis or research question fixed in II.B: does the chosen metric actually measure what the hypothesis makes a claim about, or does it measure something merely correlated with it that happened to be convenient to report? A protocol executed carefully but aimed at the wrong construct does not support the claim, however clean the resulting numbers look, and this mismatch is often invisible from within the Experiments section alone — it is only visible by holding the section up against the hypothesis it is meant to test.

Common failure modes

Beyond persuasion creep in the Method and imprecision in an under-specified Setup, three further failures recur. Baselines can be included in a weakened or undertuned form, inflating the apparent size of the improvement in a way that damages credibility as soon as a knowledgeable reader notices — the experimental analogue of the dismissive tone flagged in Related Work. A protocol can be adjusted after preliminary results are seen — a metric swapped, a baseline dropped, a data split redrawn — which quietly undermines the falsifiability the Experiments section exists to establish, in the same way that stating a hypothesis after the results are known undermines the hypothesis (II.B). And a Method or Experiments section can omit a detail that is not glamorous but is necessary for reproduction — a random seed, a specific preprocessing step, a hyperparameter search range — on the assumption that it is too minor to matter, when its absence is exactly what a reader attempting to reproduce the work will notice first.

A practical check

A useful test for both components together: could someone competent in the subfield, given only the Method and Experiments sections, rebuild the system and run the same protocol without needing to ask a clarifying question? Any point where the honest answer is no marks either a missing detail or a piece of reasoning that has been asserted rather than shown — and, in either case, a place to revise before the Results section is drafted, not after.