Significant(Yui,alone: Signifikanz,British: significance) IsProbability theory-statisticsIn terms ofprobabilityTypicallyaccidentalIt's hard to think of it, and it seems to be meaningful. "^[1]Point to that.

P value

Null hypothesisStatistics that are more extreme (contrary to hypothesis) than those actually calculated from the data are observed underprobabilityIs called the P value. Considering various problems associated with the use of P-values社会 心理学Systemjournal,(English edition）(BASP) is the null hypothesis significance test and the likestatisticsAnnounced that target processing will be prohibited^[2].

Significance level

Significance levelHow accurate is α (0 <α <1)?Null hypothesisIs a constant that indicates whether to reject.Significance level αHypothesis testIt is,At the time ofIs rejected.At this time, "statistics are significant at the α level".Significance level α is a hypothesisProbability of being rejected by hypothesis test even though is correct (First-class errorProbability of committing).The Japanese Industrial Standards define it as "the upper limit of the probability of type I error".^[3] .

Significance levelGenerally, 0.05 (5%) is used as the value of, but the method of taking it differs depending on the academic discipline, survey, and research subject.Social scienceIn some cases, 0.1 (10%) is used, and strictness is required.natural ScienceIn some cases, 0.01 (1%) is used.In addition, when displaying data, multiple significance levels may be used at the same time by setting the significance level. For example, * if it is significant at 0.05 level, **, *** for 0.01 level and 0.001 level, respectively. indicate.

Just because it is significant does not mean that it is not a coincidence, it just means that it is unlikely to be a coincidence.Therefore, for example, if the significance level is 5%, there is a high possibility that "it is just a coincidence, but it is mistakenly judged to be" meaningful "".

vice versa,In the case of, it can only be said that there is no significant difference.^[4].

Criticism of significance level

In the early 2010s科学Is "Replication crisisResearchers, funding agencies, and publishers are worried that scholarly literature may be covered in unreliable results. In 2017, 72 prominent researchers published a treatise that the low statistical standards of evidence in claiming new discoveries contributed to the reproducibility crisis.Significance level preferred by scientists to assess the statistical significance of new discoveriesThe value of is from 0.050.005Should be lowered tostatisticsLandlords insist.

On the other hand,Illinois Institute of Technology OfComputer scienceShlomo Argamon said, "As long as there are many ways to experiment, no matter how small the significance levelEven if the value of is used, it is very likely that one experimental method will be significant by chance. "Actually small significance levelIf you use the value ofStorage problemBecomes more prominent and many papers cannot be published^[5].

In 2016, the American Statistical Association issued a statement on this issue^[6]However, "No matter how small the effect, if the sample size is large or the measurement accuracy is sufficiently high, it can be a small P value." "The P value is a description of the hypothesis and the data based on the assumptions behind the calculation. Organize basic issues with P-values, such as "not a description of the underlying assumptions themselves," and emphasize that "P-values ​​alone are not a good indicator of evidence for statistical models or hypotheses." is doing.

In 2019, more than 800 scientists said,Nature, And if the P value is greater than the significance level, it can only be said that there is no significant difference, but there are documents that mistakenly infer that there is no significant difference = no effect of drugs, etc. Stopped using "statistical significance" because it was found in 791% of the 51 literatureConfidence intervalShould be used in other words as the term compatible section.^[4].

Multiple comparison

When the same type of test is repeated to determine the overall significance (), If the significance level for one test is α, the probability of obtaining a significant result even once for k similar trials. Depends on the independence of k trials.For example, when k trials are independent, Will be.However, Since the upper limit of is kα, if the significance level for one test is set to α / k, the test with a significance level of at most α can be performed for k similar trials.thisBonferroni's methodThat is.However, in this method, the significance level decreases as the value of k increases, and the practicality becomes poor.Therefore, a method with higher power has been proposed.In the olden days, the LSD method was preferred because it was easy to calculate along with the Bonferroni method.Nowadays,HSD methodAnd the Ryan method are the most common.Also,Scheffe's methodAnd WSD method are also coming to be seen.They are,Analysis of varianceIt is also used for subtests when the main effect of factors of 3 levels or more is significant.

Significant difference

The null hypothesis is "twoParameterIn the form of "there is no difference", the null hypothesis is rejected and "between the two parameters"Significant differenceThere is a conclusion.

Confidence interval and hypothesis test

Consider the case where the statistic X is an estimator of a parameter θ.At this time, the range that X can take so that the null hypothesis is not rejected at the significance level α is the confidence level.Is equal to the confidence interval of θ for.

For example, if we consider the sample mean X as an estimator of the population mean θ, the null hypothesis:The range of X that is not rejected at the 5% significance level isConsistent with the 95% confidence interval for.

footnote

References

• Yasuo Nishioka "Mathematics Tutorial Easy Talking Probability Statistics"Ohmsha, 2013.ISBN 9784274214073.
• Koji Fushimi""Kawade Shobo, 1942.ISBN 9784874720127.
• Mathematical Society of Japan"Mathematical Dictionary"Iwanami Shoten, 2007.ISBN 9784000803090.
• -1: 1999 statistics - the termとsymbol − Part 1:probabilityAnd general statistical terms, Japanese Standards Association, http://kikakurui.com/z8/Z8101-1-1999-01.html

Related item

• Reproducibility
  • Replication crisis
• probability
  • Probability theory
• statistics
  • Estimated statistics
  • Hypothesis test
  • Confidence interval