Thursday, October 31, 2013

Multiple Comparisons, Margins of Error, and the Affordable Care Act Census Data

In my last post I looked at changes in the census rates in 51 states (50 plus DC) since parts of the act have come into effect.  I found 14 with a significant change from 2010 to 2011 (the most recent years available since the ACA was passed).  Each individual state was classified as changed if the difference in their rates were outside of the 95% probability margin of error (MOE) for both years.  That means that we are 95% certain that the actual rate is between an upper and lower limit.

For example Texas had a MOE of +/- 0.2%, that means it's estimated rate for 2010 of 26.3% is between 26.5% and 26.1% with 95% probability while it's estimated rate for 2011 of 25.7%  is between 25.9% and 25.5% with 95% probability.  Because the intervals for both years do not overlap, we can be confident that the change in the rate is real across the years.


Contrary to his claims, the results suggest that Ted Cruz's Texas so far has had a real but small decrease in the uninsured rate since the ACA or Obamacare has been enacted. In the graph above, California and Vermont have had significant decreases while Missouri was the only one that increased.  Massachusetts and Pennsylvania stayed the same.  The other states are summarized in my previous post.  

Statistician critics may argue that repeating 51 comparisons inflates the chance that at least one state has been significantly different by pure chance.  The 95% confidence interval means that there is a 5% chance or 0.05 probability that each individual comparison is significant by pure chance.  Repeated 51 times means that the expected number of chance differences is 51(0.05)=2.55.  Because there were 14 significant differences which is well above the expected number of chance differences.     I can be confident that almost all of changes in the rates are real.

Looking at the county level rates for Pennsylvania there were zero significant changes either positive or negative out of the 67 counties.  Counties with small populations have very large MOEs however.

**Related Posts**

The Affordable Care Act (ACA) Having Little Effect on PA's Uninsured Rate So Far (Repost with PUSH)