Site icon R-bloggers

Power analysis for default Bayesian t-tests

[This article was first published on The 20% Statistician, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
One important benefit of Bayesian statistics is that you can provide relative support for the null hypothesis. When the null hypothesis is true, p-values will forever randomly wander between 0 and 1, but a Bayes factor has consistency (Rouder, Speckman, Sun, Morey, & Iverson, 2009), which means that as the sample size increases, the Bayes Factor will tell you which of two hypotheses has received more support.

Bayes Factors can express relative evidence for the null hypothesis (H0) compared to the alternative hypothesis (H1), referred to as a BF01, or relative evidence for H1 compared to H0 (a BF10). A BF01 > 3 is sometimes referred to as substantial evidence for H0 (see Wagenmakers, Wetzels, Borsboom, & Van Der Maas, 2011), but what they really mean is substantial evidence for H0 compared to H1.



Table 1 from Wagenmakers et al., 2011, #fixedthatforyou

Since a Bayes Factor provides relative support for one over another hypothesis, the choice for an alternative hypothesis is important. In Neyman-Pearson Frequentist approaches, researchers have to specify the alternative hypothesis to be able to calculate the power of their test. For example, let’s assume a researcher no idea what to expect, and decided to use the average effect size in psychology of d = 0.43 as the expected effect when the alternative hypothesis is true. It then becomes easy to calculate that you need 115 participants in each group of your experiment, if you plan to perform a two-sided independent t-test with an alpha of 0.05, and you want to have 90% power.

When calculating the Bayes Factor, the alternative is specified through the r-scale. This is a scale for the Cauchy prior, which is chosen in such a way that the researcher expects there is a 50% chance of observing an absolute effect larger than the scale value chosen. As Rouder and colleagues say: “A well-known and attractive alternative to placing priors on the mean µ is to place them on effect size, where effect size is denoted by δ and given as δ = µ/σ. Thus, the researcher above would choose an r-scale of 0.43.

The default Bayesian t-test originally used a r-scale of 1 (a very large effect), but the updated default test uses a r-scale of 0.707. This means that whenever you perform a study where you calculate the default Bayes Factor, and find a BF10 < 1/3, you have observed support for the null-hypothesis, relative to an alternative hypothesis of an effect of d = 0.707.

Let’s assume the default Bayesian t-test answers a question you are interested in, and you want to know whether your data is stronger support for a d = 0 compared to a d = 0.707. You know that to have 90% power to observe a d = 0.707 in an independent t-test, you need 44 participants in each condition, and you have heard people recommend to use at least 50 participants in each condition, so you set out to collect 50 participants in each condition. You are also completely fine with a null-result: You have submitted your study to a journal that accepts pre-registered reports, and the reviewers there agreed the data will be interesting whether they support the null hypothesis or the alternative hypothesis. What is the probability you will observe support for H0 relative to H1 (see this paper by Felix Schonbrodt and colleagues for similar Frequentist calculations of Bayes Factors)?

There are no Bayesian power calculators (yet, as far as I know). Bayes Factors express the relative evidence in your data, and you don’t need a threshold to interpret a Bayes Factor. In principle, any BF10 < 1 provides stronger support for the null hypothesis than for the alternative hypothesis. But no one will be convinced if you argue for the null hypothesis based on a BF10= 0.98, and it’s practical to have some sort of agreed upon threshold where we all agree data can be interpreted as ‘support’ – hence tables with Bayes Factor thresholds as the one reported in Wagenmakers et al, 2011 (see above). So what are the chances of actually observing relative support for the null hypothesis, compare to an alternative hypothesis of d = 0.707, with 50 participants in each condition, when you want a BF10 < 1/3?

Using the R code below, you can easily calculate this probability yourself. It also provides the probability of finding support for the alternative hypothesis. This means you can run the simulations twice, once setting the true effect size to 0, once setting the true effect size to 0.707, and see whether your sample size is large enough to get what you want from the data you plan to collect.Note you need quite some simulations to be accurate, and this takes minutes)

Running these simulations, we see there’s about a 69% probability of finding support for the null-hypothesis with a BF10< 1/3 if the true effect size is 0. In the histogram of Bayes Factors below, this is the distribution to the left of the left vertical red line. Approximately 1.6% of the Bayes Factors give misleading evidence (a BF > 3, to the right of the right vertical red line) and around 30% of the Bayes Factors are inconclusive evidence.



The second set of simulations reveals there’s approximately a 85% chance of finding a BF10 > 3 when the true effect size is 0.707. Based on these data, and your explicit interest in providing support for the null-hypothesis, you might want to collect some more participants.

Let’s look at some other scenarios. Imagine that in the previous situation, the true effect size was d = 0.43. It turns out that 13% of our experiments will find relative support for the null-hypothesis, 38% of the tests will find relative support for the alternative hypothesis, and 48% of the tests will be inconclusive, with a Bayes Factor between 1/3 and 3. Luckily, with Bayesian statistics you can continue collecting data, but since no one has unlimited resources, it’s important to know whether it’s feasible to collect sufficient data to answer your question in the first place. For example, in the default Bayesian t-test, don’t bother trying to find ‘strong evidence’ (BF < 1/10) for the null if you are planning to collect less than 200 participants. When the null is really true, and you plan to collect data from 100 to 200 participants in each condition, you’ll never find the evidence you are looking for. With 300 participants in each condition, you’ll start to get around 33% probability of finding what you want, indicating a threshold of 10 might be unreasonable for single studies, and more suitable for meta-analyses.  

The simulations also show that the probability of an incorrect conclusion (observing support for H0 when H1 is true, or support for H1 when H0 is true) is very low. Indeed, it’s so low, that we might want to reconsider used a Bayes Factor of 3 as a threshold.

For example, if we set a threshold as low as 1.2, instead of 3, the simulations show that, when the null-hypothesis is true, approximately 5% of the Bayes Factors we observe will provide relative support for the alternative hypothesis, and over 90% of the Bayes Factors will provide relative support for the null hypothesis, with 50 participants in each condition (the remainder, 5% fall in the small inconclusive evidence area). This may not be ‘substantial evidence’ but you won’t be wrong very often, in the long run, when you decide to accept the null hypothesis over the alternative hypothesis of d = 0.707 when you collect 50 participants in each condition.

You may not like the use of a cut-off value to indicate ‘relative support’ as proposed by some Bayesians (Dienes, 2016; Wagenmakers et al., 2011). I think these cut-offs are most important when planning an experiment. It makes sense to aim for support most people will find convincing. When the data are in hand, you can interpret it any way you like. If you agree, then important questions are which cut-off values are acceptable under which circumstances? Can we develop formulas for Bayesian power analysis instead of having to rely on simulations?

I learned something by playing around with these simulations, and I expect others moving to Bayes Factors will have similar questions as I tried to examine here. Power analysis is  Frequentist concept, and it might not be close to the heart of Bayesians, but I expect it meets a future need of researchers switching to Bayesian statistics, especially when they want to design studies that provide support for the null.

< !--[if gte mso 9]>< xml> < o:OfficeDocumentSettings> < o:AllowPNG/> < ![endif]-->
< !--[if gte mso 9]>< xml> < w:WordDocument> < w:View>Normal < w:Zoom>0 < w:TrackMoves/> < w:TrackFormatting/> < w:DoNotShowPropertyChanges/> < w:PunctuationKerning/> < w:ValidateAgainstSchemas/> < w:SaveIfXMLInvalid>false < w:IgnoreMixedContent>false < w:AlwaysShowPlaceholderText>false < w:DoNotPromoteQF/> < w:LidThemeOther>EN-US < w:LidThemeAsian>X-NONE < w:LidThemeComplexScript>X-NONE < w:Compatibility> < w:BreakWrappedTables/> < w:SnapToGridInCell/> < w:WrapTextWithPunct/> < w:UseAsianBreakRules/> < w:DontGrowAutofit/> < w:SplitPgBreakAndParaMark/> < w:EnableOpenTypeKerning/> < w:DontFlipMirrorIndents/> < w:OverrideTableStyleHps/> < m:mathPr> < m:mathFont m:val="Cambria Math"/> < m:brkBin m:val="before"/> < m:brkBinSub m:val="--"/> < m:smallFrac m:val="off"/> < m:dispDef/> < m:lMargin m:val="0"/> < m:rMargin m:val="0"/> < m:defJc m:val="centerGroup"/> < m:wrapIndent m:val="1440"/> < m:intLim m:val="subSup"/> < m:naryLim m:val="undOvr"/> < ![endif]-->< !--[if gte mso 9]>< xml> < w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="false" DefSemiHidden="false" DefQFormat="false" DefPriority="99" LatentStyleCount="371"> < w:LsdException Locked="false" Priority="0" QFormat="true" Name="Normal"/> < w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 1"/> < w:LsdException Locked="false" Priority="9" SemiHidden="true" UnhideWhenUsed="true" QFormat="true" Name="heading 2"/> < w:LsdException Locked="false" Priority="9" SemiHidden="true" UnhideWhenUsed="true" QFormat="true" Name="heading 3"/> < w:LsdException Locked="false" Priority="9" SemiHidden="true" UnhideWhenUsed="true" QFormat="true" Name="heading 4"/> < w:LsdException Locked="false" Priority="9" SemiHidden="true" UnhideWhenUsed="true" QFormat="true" Name="heading 5"/> < w:LsdException Locked="false" Priority="9" SemiHidden="true" UnhideWhenUsed="true" QFormat="true" Name="heading 6"/> < w:LsdException Locked="false" Priority="9" SemiHidden="true" UnhideWhenUsed="true" QFormat="true" Name="heading 7"/> < w:LsdException Locked="false" Priority="9" SemiHidden="true" UnhideWhenUsed="true" QFormat="true" Name="heading 8"/> < w:LsdException Locked="false" Priority="9" SemiHidden="true" UnhideWhenUsed="true" QFormat="true" Name="heading 9"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="index 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="index 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="index 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="index 4"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="index 5"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="index 6"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="index 7"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="index 8"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="index 9"/> < w:LsdException Locked="false" Priority="39" SemiHidden="true" UnhideWhenUsed="true" Name="toc 1"/> < w:LsdException Locked="false" Priority="39" SemiHidden="true" UnhideWhenUsed="true" Name="toc 2"/> < w:LsdException Locked="false" Priority="39" SemiHidden="true" UnhideWhenUsed="true" Name="toc 3"/> < w:LsdException Locked="false" Priority="39" SemiHidden="true" UnhideWhenUsed="true" Name="toc 4"/> < w:LsdException Locked="false" Priority="39" SemiHidden="true" UnhideWhenUsed="true" Name="toc 5"/> < w:LsdException Locked="false" Priority="39" SemiHidden="true" UnhideWhenUsed="true" Name="toc 6"/> < w:LsdException Locked="false" Priority="39" SemiHidden="true" UnhideWhenUsed="true" Name="toc 7"/> < w:LsdException Locked="false" Priority="39" SemiHidden="true" UnhideWhenUsed="true" Name="toc 8"/> < w:LsdException Locked="false" Priority="39" SemiHidden="true" UnhideWhenUsed="true" Name="toc 9"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Normal Indent"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="footnote text"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="annotation text"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="header"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="footer"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="index heading"/> < w:LsdException Locked="false" Priority="35" SemiHidden="true" UnhideWhenUsed="true" QFormat="true" Name="caption"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="table of figures"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="envelope address"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="envelope return"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="footnote reference"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="annotation reference"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="line number"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="page number"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="endnote reference"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="endnote text"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="table of authorities"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="macro"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="toa heading"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Bullet"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Number"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List 4"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List 5"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Bullet 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Bullet 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Bullet 4"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Bullet 5"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Number 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Number 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Number 4"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Number 5"/> < w:LsdException Locked="false" Priority="10" QFormat="true" Name="Title"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Closing"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Signature"/> < w:LsdException Locked="false" Priority="1" SemiHidden="true" UnhideWhenUsed="true" Name="Default Paragraph Font"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Body Text"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Body Text Indent"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Continue"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Continue 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Continue 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Continue 4"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="List Continue 5"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Message Header"/> < w:LsdException Locked="false" Priority="11" QFormat="true" Name="Subtitle"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Salutation"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Date"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Body Text First Indent"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Body Text First Indent 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Note Heading"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Body Text 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Body Text 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Body Text Indent 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Body Text Indent 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Block Text"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Hyperlink"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="FollowedHyperlink"/> < w:LsdException Locked="false" Priority="22" QFormat="true" Name="Strong"/> < w:LsdException Locked="false" Priority="20" QFormat="true" Name="Emphasis"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Document Map"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Plain Text"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="E-mail Signature"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Top of Form"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Bottom of Form"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Normal (Web)"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Acronym"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Address"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Cite"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Code"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Definition"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Keyboard"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Preformatted"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Sample"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Typewriter"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="HTML Variable"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Normal Table"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="annotation subject"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="No List"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Outline List 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Outline List 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Outline List 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Simple 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Simple 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Simple 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Classic 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Classic 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Classic 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Classic 4"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Colorful 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Colorful 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Colorful 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Columns 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Columns 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Columns 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Columns 4"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Columns 5"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Grid 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Grid 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Grid 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Grid 4"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Grid 5"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Grid 6"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Grid 7"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Grid 8"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table List 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table List 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table List 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table List 4"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table List 5"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table List 6"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table List 7"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table List 8"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table 3D effects 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table 3D effects 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table 3D effects 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Contemporary"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Elegant"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Professional"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Subtle 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Subtle 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Web 1"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Web 2"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Web 3"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Balloon Text"/> < w:LsdException Locked="false" Priority="39" Name="Table Grid"/> < w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true" Name="Table Theme"/> < w:LsdException Locked="false" SemiHidden="true" Name="Placeholder Text"/> < w:LsdException Locked="false" Priority="1" QFormat="true" Name="No Spacing"/> < w:LsdException Locked="false" Priority="60" Name="Light Shading"/> < w:LsdException Locked="false" Priority="61" Name="Light List"/> < w:LsdException Locked="false" Priority="62" Name="Light Grid"/> < w:LsdException Locked="false" Priority="63" Name="Medium Shading 1"/> < w:LsdException Locked="false" Priority="64" Name="Medium Shading 2"/> < w:LsdException Locked="false" Priority="65" Name="Medium List 1"/> < w:LsdException Locked="false" Priority="66" Name="Medium List 2"/> < w:LsdException Locked="false" Priority="67" Name="Medium Grid 1"/> < w:LsdException Locked="false" Priority="68" Name="Medium Grid 2"/> < w:LsdException Locked="false" Priority="69" Name="Medium Grid 3"/> < w:LsdException Locked="false" Priority="70" Name="Dark List"/> < w:LsdException Locked="false" Priority="71" Name="Colorful Shading"/> < w:LsdException Locked="false" Priority="72" Name="Colorful List"/> < w:LsdException Locked="false" Priority="73" Name="Colorful Grid"/> < w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 1"/> < w:LsdException Locked="false" Priority="61" Name="Light List Accent 1"/> < w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 1"/> < w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 1"/> < w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 1"/> < w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 1"/> < w:LsdException Locked="false" SemiHidden="true" Name="Revision"/> < w:LsdException Locked="false" Priority="34" QFormat="true" Name="List Paragraph"/> < w:LsdException Locked="false" Priority="29" QFormat="true" Name="Quote"/> < w:LsdException Locked="false" Priority="30" QFormat="true" Name="Intense Quote"/> < w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 1"/> < w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 1"/> < w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 1"/> < w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 1"/> < w:LsdException Locked="false" Priority="70" Name="Dark List Accent 1"/> < w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 1"/> < w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 1"/> < w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 1"/> < w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 2"/> < w:LsdException Locked="false" Priority="61" Name="Light List Accent 2"/> < w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 2"/> < w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 2"/> < w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 2"/> < w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 2"/> < w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 2"/> < w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 2"/> < w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 2"/> < w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 2"/> < w:LsdException Locked="false" Priority="70" Name="Dark List Accent 2"/> < w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 2"/> < w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 2"/> < w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 2"/> < w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 3"/> < w:LsdException Locked="false" Priority="61" Name="Light List Accent 3"/> < w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 3"/> < w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 3"/> < w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 3"/> < w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 3"/> < w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 3"/> < w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 3"/> < w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 3"/> < w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 3"/> < w:LsdException Locked="false" Priority="70" Name="Dark List Accent 3"/> < w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 3"/> < w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 3"/> < w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 3"/> < w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 4"/> < w:LsdException Locked="false" Priority="61" Name="Light List Accent 4"/> < w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 4"/> < w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 4"/> < w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 4"/> < w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 4"/> < w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 4"/> < w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 4"/> < w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 4"/> < w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 4"/> < w:LsdException Locked="false" Priority="70" Name="Dark List Accent 4"/> < w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 4"/> < w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 4"/> < w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 4"/> < w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 5"/> < w:LsdException Locked="false" Priority="61" Name="Light List Accent 5"/> < w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 5"/> < w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 5"/> < w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 5"/> < w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 5"/> < w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 5"/> < w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 5"/> < w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 5"/> < w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 5"/> < w:LsdException Locked="false" Priority="70" Name="Dark List Accent 5"/> < w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 5"/> < w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 5"/> < w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 5"/> < w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 6"/> < w:LsdException Locked="false" Priority="61" Name="Light List Accent 6"/> < w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 6"/> < w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 6"/> < w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 6"/> < w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 6"/> < w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 6"/> < w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 6"/> < w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 6"/> < w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 6"/> < w:LsdException Locked="false" Priority="70" Name="Dark List Accent 6"/> < w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 6"/> < w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 6"/> < w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 6"/> < w:LsdException Locked="false" Priority="19" QFormat="true" Name="Subtle Emphasis"/> < w:LsdException Locked="false" Priority="21" QFormat="true" Name="Intense Emphasis"/> < w:LsdException Locked="false" Priority="31" QFormat="true" Name="Subtle Reference"/> < w:LsdException Locked="false" Priority="32" QFormat="true" Name="Intense Reference"/> < w:LsdException Locked="false" Priority="33" QFormat="true" Name="Book Title"/> < w:LsdException Locked="false" Priority="37" SemiHidden="true" UnhideWhenUsed="true" Name="Bibliography"/> < w:LsdException Locked="false" Priority="39" SemiHidden="true" UnhideWhenUsed="true" QFormat="true" Name="TOC Heading"/> < w:LsdException Locked="false" Priority="41" Name="Plain Table 1"/> < w:LsdException Locked="false" Priority="42" Name="Plain Table 2"/> < w:LsdException Locked="false" Priority="43" Name="Plain Table 3"/> < w:LsdException Locked="false" Priority="44" Name="Plain Table 4"/> < w:LsdException Locked="false" Priority="45" Name="Plain Table 5"/> < w:LsdException Locked="false" Priority="40" Name="Grid Table Light"/> < w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light"/> < w:LsdException Locked="false" Priority="47" Name="Grid Table 2"/> < w:LsdException Locked="false" Priority="48" Name="Grid Table 3"/> < w:LsdException Locked="false" Priority="49" Name="Grid Table 4"/> < w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark"/> < w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful"/> < w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful"/> < w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light Accent 1"/> < w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 1"/> < w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 1"/> < w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 1"/> < w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 1"/> < w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful Accent 1"/> < w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful Accent 1"/> < w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light Accent 2"/> < w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 2"/> < w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 2"/> < w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 2"/> < w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 2"/> < w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful Accent 2"/> < w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful Accent 2"/> < w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light Accent 3"/> < w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 3"/> < w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 3"/> < w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 3"/> < w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 3"/> < w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful Accent 3"/> < w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful Accent 3"/> < w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light Accent 4"/> < w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 4"/> < w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 4"/> < w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 4"/> < w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 4"/> < w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful Accent 4"/> < w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful Accent 4"/> < w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light Accent 5"/> < w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 5"/> < w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 5"/> < w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 5"/> < w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 5"/> < w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful Accent 5"/> < w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful Accent 5"/> < w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light Accent 6"/> < w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 6"/> < w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 6"/> < w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 6"/> < w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 6"/> < w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful Accent 6"/> < w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful Accent 6"/> < w:LsdException Locked="false" Priority="46" Name="List Table 1 Light"/> < w:LsdException Locked="false" Priority="47" Name="List Table 2"/> < w:LsdException Locked="false" Priority="48" Name="List Table 3"/> < w:LsdException Locked="false" Priority="49" Name="List Table 4"/> < w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark"/> < w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful"/> < w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful"/> < w:LsdException Locked="false" Priority="46" Name="List Table 1 Light Accent 1"/> < w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 1"/> < w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 1"/> < w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 1"/> < w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 1"/> < w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful Accent 1"/> < w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful Accent 1"/> < w:LsdException Locked="false" Priority="46" Name="List Table 1 Light Accent 2"/> < w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 2"/> < w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 2"/> < w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 2"/> < w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 2"/> < w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful Accent 2"/> < w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful Accent 2"/> < w:LsdException Locked="false" Priority="46" Name="List Table 1 Light Accent 3"/> < w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 3"/> < w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 3"/> < w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 3"/> < w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 3"/> < w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful Accent 3"/> < w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful Accent 3"/> < w:LsdException Locked="false" Priority="46" Name="List Table 1 Light Accent 4"/> < w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 4"/> < w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 4"/> < w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 4"/> < w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 4"/> < w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful Accent 4"/> < w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful Accent 4"/> < w:LsdException Locked="false" Priority="46" Name="List Table 1 Light Accent 5"/> < w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 5"/> < w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 5"/> < w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 5"/> < w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 5"/> < w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful Accent 5"/> < w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful Accent 5"/> < w:LsdException Locked="false" Priority="46" Name="List Table 1 Light Accent 6"/> < w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 6"/> < w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 6"/> < w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 6"/> < w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 6"/> < w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful Accent 6"/> < w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful Accent 6"/> < ![endif]-->< !--[if gte mso 10]>< ![endif]-->
Dienes, Z. (2016). How Bayes factors change scientific practice. Journal of Mathematical Psychology. http://doi.org/10.1016/j.jmp.2015.10.003

Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237. http://doi.org/10.3758/PBR.16.2.225

Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & Van Der Maas, H. L. (2011). Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011). Retrieved from http://psycnet.apa.org/journals/psp/100/3/426/
 


To leave a comment for the author, please follow the link and comment on their blog: The 20% Statistician.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.