Confidence Interval for a Binomial Proportion (Beta)
|Upper Confidence Bound|
|Lower Confidence Bound|
|Confidence Interval (% points)|
As General Counsel of a technology company, you are considering allocating a patent dispute case to a new law firm (“Law firm LLP”) because they seem to have a high success rate in patent disputes. Law Firm have won 16 out of 20 court cases on behalf of patentee companies in patent infringement disputes. What does this sample of data really have to say about Law Firm? This confidence interval application can help answer this question.
Enter the inputs above as shown here:
Notice we have added an assumed national average win rate of 70% for patentee counsel by inputting “70” in the appropriate field. And we have added a custom confidence level of 85% by inputting “85”.
Enter the above inputs and click “Calculate” to produce a table of confidence intervals and two charts.
The results illustrate how the sample of historical data describing Law Firm’s win rate is not at all definitive. When we consider the sampling error in the small amount of data for Law Firm, the uncertainty is enormous. With a 95% confidence level, Law Firm’s population win rate could be as high as 94% but as low as 56%. An uncertain range of 38 percentage points! And notice if we wanted to be 99% confident of describing Law Firm’s win rate, the uncertainty would rise to 47 percentage points. In short, although we know the fact of Law Firm’s historical win rate, it doesn’t say very much about what it is we care about: their future win rate. The uncertainty implied in the sample of data is so great that Law Firm’s performance cannot be said to be distinguished from the national average.
Look now at Chart 2. This chart calculates a range of 95% confidence intervals assuming different sample sizes for Law Firm. Notice that the uncertain range in such a measurement would not exclude the national average win rate until the sample size was over 100. The exact numbers will vary depending on your inputs. The point is to illustrate the type of data volumes you will need before measurements of legal performance become meaningfully distinguished from national averages or other relevant peer groups.
Similar confidence interval analyses can be used to quantify the uncertainty in judicial decision making and other legal metrics.
Method of Calculation
There are several different methods for computing confidence intervals and each have their advantages and disadvantages. However, because of the mostly binary nature of trial outcomes, judicial decisions and legal metrics in general, we have provided an application using a confidence interval for a binomial proportion. The advantage of using a binomial model as opposed to a normal distribution is that the binomial model can reflect the skew or asymmetry of the confidence interval as the proportion or win rate approaches high or low extremes.
There are then several different methods of computing binomial proportion confidence intervals. In this application we have used the Clopper-Pearson method (sometimes called the ‘exact method’), which also has advantages and disadvantages. One criticism of the Clopper-Pearson method is that it provides a conservative measure of the interval – it is generally wider than with other methods.
- This application is in Beta release until further notice.
- Binomial proportion confidence intervals such as the one used here are suitable only for data reflecting binary outcomes like win/lose, grant/deny, guilty/not guilty. When the data is continuous such as the level of damages awards, other methods of computing the confidence interval should be used. We will be adding these other methods to our Law-Stats tools shortly. In the meantime, if you require customized legal data analytics, send us a message using the Contact Form or send an email to email@example.com.
- The provision of these data analytic tools should not be construed as endorsing the data-driven analysis of law. There are many reasons why the quantitative analysis of legal data may be invalid in certain circumstances. For a more detailed discussion of these concerns, see When Big Legal Data Isn’t Big Enough: Limitations in Legal Data Analytics.
- In general, all quantitative analytic methods, whether they be data-driven or model-based, have their strengths and their shortcomings. In our view, empirical and theoretical approaches are not universally helpful — much depends on the situation being analyzed. In the case of legal data analytics, if the data sample is sufficiently free from sample bias and the sample is big enough so that confidence intervals are relatively narrow and p-values are low, there can be an analytic advantage to incorporating the analysis of data into decision making.
- Notice that the Red Line marking the Null Level in Chart 2 is printing slightly to right of its proper location. We are working on this issue.
- You can ‘Right Click’ on the charts to copy/paste the chart images for use in reports and presentations provided that the SettlementAnalytics.com label is not removed.
- If you have suggestions as to how we can improve our applications, we would welcome your feedback. Please use the Contact Form or send an email to firstname.lastname@example.org.
Clopper, C., Pearson, E. S., “The use of confidence or fiducial limits illustrated in the case of the binomial” (1934) Biometrika 26, 404–413.
Click on the tabs above to access other Law-Stats™ tools. Statistical testing applications are available for hypothesis testing of a binomial proportion and confidence interval for the difference between two binomial proportions.
Copyright © SettlementAnalytics 2016