A couple weeks back I advised the folks over at Brulosophy to switch to an upper tailed binomial proportions test for determining significant results in their exbeeriments. I also created a p-value calculator for them, which you can now use for your own exbeeriments!
In case you were wondering, the statistical analysis associated with the triangle test compares the proportion of test participants whom have correctly identified an odd beer out, to the proportion of tasters that would be expected to correctly identify the odd beer purely due to random chance. The greater the proportion of correct participants, the more evidence there is against the “random chance” null hypothesis. As we are only interested if the different beers can be correctly distinguished, this will be an “upper tailed” test. That is to say, the potential result of the odd beer out being correctly identified less than we’d expect under random chance isn’t of particular interest to us (and is not particularly likely either).
In plain terms, our null hypothesis is that the true proportion of population that can correctly identify the odd beer out is 1/3. Our alternative hypothesis is that the true proportion of the population that can correctly identify the odd beer out is greater than 1/3.
The exact p-value can be calculated using the binomial distribution. Specifically, the p-value is found as the probability of having observed at least as many correct tasters, if the population proportion is infact equal to 1/3.
An approximate p-value can be calculated assuming that under the null hypothesis, the estimated proportion will follow a normal distribution with mean equal to 1/3. The approximate method may be used when the sample size is equal or greater than 25.