Hypothesis Test for a Binomial Proportion (Beta)
Imagine as defendant’s counsel in a securities class action lawsuit, you are considering a motion for dismissal with (the completely fictional) Judge Jones. Data for this judge suggests that he is unreceptive to motions for dismissal having granted only 6 of the last 30, for a dismissal proportion of 20%. Of all securities class action cases nationwide, you are given to understand that approximately 32% terminate via a motion to dismiss, significantly more than the dismissal rate for the judge in your case.
As counsel you consider whether to take the data on Judge Jones into consideration. Given the small sample, you wonder whether chance could explain the low level of dismissals for your judge. You decided to run a hypothesis test to see if the Judge’s dismissal proportion is significantly lower the nationwide average for all judges.
Enter the following inputs above as shown here:
Click ‘Calculate’ to produce the hypothesis test charts and tables.
Notice we are using a 5% required significance level for this particular test, but keep in mind that the resulting ‘actual’ significance level may be lower because the distribution is discrete (see note below).
Now look at the results as illustrated by the histogram in Chart 1. This is a graphical representation of a hypothesis test and it works like this: The vertical bars in Chart 1 represent the theoretical probability of finding each possible number of dismissals (0-30) in the given sample size based on the assumption that Judge Jones’s propensity to grant dismissal motions is the same as the national average. This starting assumption, that the Judge is no different from the national average is called the ‘null hypothesis’. Notice that the probability of a high or low number of dismissals diminishes as it diverges from the national average rate of 32%.
The region of the chart shaded yellow is called the ‘Rejection Region’. The probability of finding a number of dismissals in this region is less than or equal to the significance level of the test. If the observed number of dismissals was located in this region, it would be considered sufficiently extreme that we could reject the null hypothesis. In this case, we would be able to conclude that the judge’s propensity to grant dismissal motions was significantly lower than the national average.
However, although the rate at which this judge grants dismissal motions is lower than the national average, it is not so low as to fall into the Rejection Region. After allowing for the small sample size, and the variability which could occur by chance, the observation for Judge Jones could be considered fairly commonplace. We cannot reject the null hypothesis – we cannot conclude that this judge is not the same as the national average when it comes to granting motions to dismiss.
Hypothesis testing methods like this can also be used to assess the extent to which a law firm’s win rate may or may not be statistically different from its peer group.
Method of Calculation
This hypothesis test tool uses a binomial model for a proportion. For the purposes of illustration, the method presented is based on a ‘Critical Values’ approach to the interpretation of hypothesis tests as opposed to the p-value method. Users may find it helpful to familiarize themselves with both approaches.
Essentially this application computes the discrete probability of finding X number of positives in the given sample size assuming the null hypothesis to be true. The charts and tables then highlight the relevant ‘Rejection Region’ (or ‘Critical Region’) based on the given significance level for the test.
A benefit of the Critical Values method is that interpretation of the test is simple and visual. If the observed number of positives represented by the red vertical line is inside the yellow “Rejection Region”, the test is significant at the required level of significance
Notice that, because the binomial proportion distribution is discrete (as opposed to continuous), the actual level of significance for the test will not always be equal to the required (input) level. The two measures are reported at the bottom of each analysis. Users should always check the difference and make sure they are comfortable with the effective level of significance being employed by the test.
- This application is in Beta release until further notice.
- A binomial proportion hypothesis test such as the one used here is suitable only for data reflecting binary outcomes like win/lose, grant/deny, guilty/not guilty. When the data is continuous such as the level of damages awards, other continuous distributions such as the normal distribution or the t-distribution should be considered. We will be adding these other methods to our Law-Stats tools shortly. In the meantime, if you require customized legal data analytics, send us a message using the Contact Form or send an email to email@example.com.
- Note that the application is limited to a maximum sample size of n=169. This is a limit of the chart resolution. We will be updating this application shortly to accommodate larger sample sizes.
- The provision of these data analytic tools should not be construed as endorsing the data-driven analysis of law. There are many reasons why the quantitative analysis of legal data may be invalid in certain circumstances. For a more detailed discussion of these concerns, see When Big Legal Data Isn’t Big Enough: Limitations in Legal Data Analytics.
- In general, all quantitative analytic methods, whether they be data-driven or model-based, have their strengths and their shortcomings. In our view, empirical and theoretical approaches are not universally helpful — much depends on the situation being analyzed. In the case of legal data analytics, if the data sample is sufficiently free from sample bias and the sample is big enough so that confidence intervals are relatively narrow and p-values are low, there can be an analytic advantage to incorporating the analysis of data into decision making.
- Notice that the Red Line marking the observed number of positives prints to the right hand side of the ‘bin’ in which it is located.
- You can ‘Right Click’ on the charts to copy/paste the chart images for use in reports and presentations provided that the SettlementAnalytics.com label is not removed.
- If you have suggestions as to how we can improve our applications, we would welcome your feedback. Please use the Contact Form or send an email to firstname.lastname@example.org.
For a useful introduction to hypothesis testing for a binomial proportion see, Crawshaw, Janet, and Joan Chambers. A Concise Course in Advanced Level Statistics: With Worked Examples. 4th ed. N.p.: Nelson Thornes, 2001, pp. 483-492.
Click on the tabs above to access other Law-Stats™ tools. Statistical testing applications are available for a confidence interval for a binomial proportion and a confidence interval for the difference between two binomial proportions.
Copyright © SettlementAnalytics 2016