Statistical Significance and the PHC Curve

This book explains the importance of using the probability that the hypothesis is correct (PHC), an intuitive measure that anyone can understand, as an alternative to the p-value. In order to overcome the “reproducibility crisis” caused by the misuse of significance tests, this book provides a detailed explanation of the mechanism of p-hacking using significance tests, and concretely shows the merits of PHC as an alternative to p-values.

In March 2019, two impactful papers on statistics were published. One paper, “Moving to a World Beyond ‘p <0.05’”, was featured in the scholarly journal The American Statistician, overseen by the American Statistical Association. The title of the first chapter is “Don’t Say ‘Statistically Significant’”, and it uses the imperative form to clearly forbid the use of significance testing. Another paper, “Retire statistical significance”, was published in the prestigious scientific journal Nature. This commentary was endorsed by more than 800 scientists, advocating for the statement, “We agree, and call for the entire concept of statistical significance to be abandoned.”

Consider a study comparing the duration of hospital stays between treatments A and B. Previously, research conclusions were typically stated as: “There was a statistically significant difference at the 5% level in the average duration of hospital stays.” This phrasing is quite abstract. Instead, we present the following conclusion as an example: (1) The average duration of hospital stays for Group A is at least half a day shorter than for Group B. (2) 71% of patients in Group A have shorter hospital stays than the average for Group B. (3) Group A has an average hospital stay that is, on average, no more than 94% of that of Group B. Then, the probability that the expression is correct is shown. That is the PHC curve.

Hideki Toyoda as a psychometrician and an expert in structural equation modeling, the author has calculated the brand value of Brand Japan since the first survey. Brand Japan is a brand value evaluation project sponsored by Nikkei BP Consulting, which has been implemented since 2001. The project visualizes the evaluations of over 50,000 people for 1,500 brands of companies, products, and services. It is the largest brand value evaluation survey in Japan and is useful for verifying the results of practical brand strategies, public relations activities, and marketing, as well as setting future goals.

The author is also an expert in item response theory, and has been in charge of the planning, analysis, and evaluation of the Nikkei TEST since 2008, when the first public test was held.The Nikkei Test is a test organized by the Nikkei newspaper that measures the “breadth and amount of knowledge necessary for business, as well as the ability to think.” The test uses questions created from real-world economics and is represented by five evaluation axes. There are four types of Nikkei Tests, which can be chosen based on the purpose and form of the test. These include a nationwide simultaneous test (online test), a corporate/group test, a test center test, and a Nikkei Test training drill. The test is a valuable tool for self-analysis and evaluation, as well as for accelerating skill development.