Chapter 12 Statistical writeups

12.1 General principles of statistical reporting

1. Replicability (provide enough information for someone else to be able to do what you did)
2. Interpretability
   1. When possible, make it so the numbers indicate something meaningful, and make sure the reader knows what those are.
   2. Link to a good visualization that parallels your statistical design
3. Readability (have it structured in a clear way)
4. Justification for your design choices
5. Redundancy
   1. Avoid when unnecessary
   2. Embrace when useful (summarization, takeaways)

12.2 Suggested formula

Description of analysis

I find it useful to make it clear to the reader how you will do your analysis, prior to actually presenting the results.

1. Relate your analysis to your original research question/hypotheses.
   • What EXACTLY is your response variable? (how are you quantifying it?)
   • What are your fixed predictor variables? For each,
     • What is the unit of measurement?
     • Have they been scaled/modified? Why?
     • For categorical factors:
       • What are the levels?
       • What is the coding scheme?
       • What is the reference level for each?
2. Describe the analysis principles and model(s) that you will be using
   • What kind of model(s) will be using, and why?
   • Which software and packages are you using?
   • How are you determining significance?
   • Which fixed effects will be included in your model? Will you include interactions?
   • If using mixed-effects models, which grouping variables/random effects are you including?

Presentation of results

1. Descriptive statistics
   • Present graphs of your results: think about the clearest way to present these in terms of getting your point across to the reader about the answer to your research question.
     • Make sure figures are clear, with all necessary information clearly labeled on the axes and/or in captions.
     • Make sure any text in the figure (axis labels, etc.) is big enough to be visible (it should be about as big as the main text!)
     • It is good to have graphs that are easily relatable to your statistical models
   • Consider the following:
     • Summary statistics (appropriate measures of central tendency and/or variance) corresponding to these graphs in tables.
     • Confidence intervals, either in the graphs or in the summary tables.
   • Give a prose summary of the results, describing what these graphs are showing. You will not be talking about significance at this point, just pointing the reader to overall patterns.
2. Inferential statistics
   • Present the output of your statistical models
     • Remind the reader of the model structure
     • Present the relevant numbers
       • beta-coefficients, standard error, t- or z-scores, p-values
       • I think it’s clearest to present the full output of the model in a table, but you can also put the numbers within the prose.
   • Give a prose description of the statistical results
     • Be systematic: Describe and interpret each relevant part of the output in a logical way
     • Walk the reader through an interpretation of the estimates (tell them what it means in relation to your actual research question)
     • Give a concrete measure of the magnitude of the effect
       • In regression models, this will usually be the estimate/beta-coefficient
       • For logistic regression, you will need to convert log odds to probability
3. Takeaway
   • After going through the results, end with a summary of the important takeaways, relating them to your research questions.

12.3 General principles of statistical design

There are many degrees of researcher freedom in how to build a model, and little consensus on the “right” way to do so. However, there are a few things that are uncontroversially good to do:

• Design high-powered studies with precise predictions (when possible); be very cautious about giving a general interpretation for exploratory/post-hoc tests.
• Replicate.
• Make data and analyses public when possible.

Winter (2019) provides a great discussion in his final chapter.

References

Winter, Bodo. 2019. Statistics for Linguists: An Introduction Using R. Routledge.