Objective Bayesian Hypothesis Testing
The article discusses Objective Bayesian Hypothesis Testing, critiques P-values, introduces expected encompassing intrinsic Bayes factors (EEIBFs) as a reliable alternative, and emphasizes the need for better statistical education.
Read original article
The article discusses the concept of Objective Bayesian Hypothesis Testing, emphasizing the importance of deriving posterior probabilities for hypotheses using default Bayes factors. It highlights the historical context of hypothesis testing, referencing the clinical trials conducted by Cushny and Peebles at Kalamazoo Psychiatric Hospital, which aimed to evaluate the effectiveness of different soporifics. The text critiques the common reliance on P-values in hypothesis testing, pointing out the frequent misunderstanding that P-values indicate the probability of the null hypothesis being true. It cites examples of misinterpretation, including the "P value fallacy," and discusses the limitations of traditional significance testing. The article introduces the concept of expected encompassing intrinsic Bayes factors (EEIBFs) as a more reliable method for hypothesis testing, providing a framework for calculating posterior probabilities under objective priors. An example using the data from the hyoscine trial illustrates how to apply EEIBFs to compare the effectiveness of two drugs. The article concludes by discussing the foundational role of Bayes factors and objective priors in Bayesian hypothesis testing, contrasting them with traditional methods and emphasizing the need for better statistical education on these topics.
- Objective Bayesian Hypothesis Testing offers a framework for deriving posterior probabilities using Bayes factors.
- The article critiques the common use of P-values, highlighting frequent misunderstandings about their interpretation.
- Expected encompassing intrinsic Bayes factors (EEIBFs) are proposed as a more reliable alternative for hypothesis testing.
- The historical context of hypothesis testing is illustrated through the clinical trials of Cushny and Peebles.
- The importance of proper statistical education on Bayesian methods is emphasized.
Related
Six things to keep in mind while reading biology ML papers
The article outlines considerations for reading biology machine learning papers, cautioning against blindly accepting results, emphasizing critical evaluation, understanding limitations, and recognizing biases. It promotes a nuanced and informed reading approach.
Is Everything BS?
Rory Sutherland emphasizes combining behavioral science and creativity for effective problem-solving. He advocates for a balanced approach, highlighting the significance of psychological insights alongside traditional methods to address various challenges successfully.
Gaussian Processes
Bayesian Optimization explained with applications in Hyperparameter Optimization and Neural Architecture Search. Uses Gaussian Processes to balance exploration and exploitation, adapting iteratively for dynamic input space exploration. Versatile optimization technique with evolving applications.
Ranking Fields by p-Value Suspiciousness
The article addresses p-hacking in research, focusing on suspicious p-value clustering across fields. Economics shows more credible results. Replication crisis is universal, emphasizing the call for research integrity and transparency.
A/B testing mistakes I learned the hard way
Lior Neu-ner highlights key A/B testing mistakes, emphasizing the need for a clear hypothesis, proper result segmentation, careful user selection, and monitoring counter metrics to avoid misleading conclusions.
4 commentsAs it was very difficult for someone like me without higher stats or math education, I can highly recommend the following additional sources:
- https://www.redjournal.org/article/S0360-3016(21)03256-9/ful...
- https://amplitude.com/blog/frequentist-vs-bayesian-statistic...
- https://indico.cern.ch/event/568904/contributions/2651065/at...
Related
Six things to keep in mind while reading biology ML papers
The article outlines considerations for reading biology machine learning papers, cautioning against blindly accepting results, emphasizing critical evaluation, understanding limitations, and recognizing biases. It promotes a nuanced and informed reading approach.
Is Everything BS?
Rory Sutherland emphasizes combining behavioral science and creativity for effective problem-solving. He advocates for a balanced approach, highlighting the significance of psychological insights alongside traditional methods to address various challenges successfully.
Gaussian Processes
Bayesian Optimization explained with applications in Hyperparameter Optimization and Neural Architecture Search. Uses Gaussian Processes to balance exploration and exploitation, adapting iteratively for dynamic input space exploration. Versatile optimization technique with evolving applications.
Ranking Fields by p-Value Suspiciousness
The article addresses p-hacking in research, focusing on suspicious p-value clustering across fields. Economics shows more credible results. Replication crisis is universal, emphasizing the call for research integrity and transparency.
A/B testing mistakes I learned the hard way
Lior Neu-ner highlights key A/B testing mistakes, emphasizing the need for a clear hypothesis, proper result segmentation, careful user selection, and monitoring counter metrics to avoid misleading conclusions.