This is the first in a series of blogs on this topic. This one is a long blog but you need to read it very carefully. You need to understand exactly how the information provided by the MOH to Dr Hutchison is so misleading and the implications of that for the whole debate.
In December 2010, the Ministry of Health (MOH) replied to other submitters, and my written and oral presentation of March and June 2010, in which I stated that there had never been a scientifically valid vaccinated unvaccinated study. On page 42 of their 102 page submission, the MOH stated: A recently published study showed that vaccinating children on time (according to the immunisation schedule) during infancy had no effect on neuropsychological outcomes 7 to 10 years later, compared to children who had were vaccinated late or not at all (Smith and Woods 2010). The study used publicly available data on 1,047 children from the United States’ Vaccine Safety Datalink project, and analysed 42 neuropsychological outcomes. For some outcomes, vaccination on time was associated with better performance; no statistically significant differences favoured late or no vaccination.
The next paragraph said:
With respect to specific diseases, there are claims that atopy (the asthma, eczema and hay fever triad) is more common in children who are immunised than those who are not. There is conflicting evidence from a small number of studies on this topic. Many of the studies use sampling methods with a high probability of bias, such as parent recall in groups of parents who choose not to immunise their children. One of the largest and best conducted studies was a longitudinal cohort study of 167,240 children followed up prospectively (DeStefano 2002). In this group there was no evidence of a relationship between asthma and immunisation status.
This multiple asthma porkie will be dealt with in another blog, but note the highlighted portion above, about sampling methods with a high probability of bias
So let’s look at this Smith 2010 study which the MOH uses to counter my allegations.
If you just want a ten-second soundbite, here it is.
The Smith 2010 study is not valid science, because there were too many exclusions, and too few never vaccinated children. Furthermore, this study can't prove that vaccinated children were healthier than unvaccinated children, because as the authors admit, there were too few never vaccinated children to do the required analyses.
If that's enough to convince you, then read no further.
But if you really want to understand exactly why this study cannot be seriously misrepresented the way the Ministry of Health did to the Parliamentary Select Committee, you will have to wade through the rest of this blog to understand EXACTLY why the Ministry of Health is wrong.
Here are extracts from a previous blog on this study:
Note the underlining where the study authors say;
"This cohort did not have enough children who were fully unvaccinated in the first year of life to form robust estimates of neuropsychological outcomes as compared with children with other patterns of receipt. This is an inherent limitation of any VSD-based study given the generally high immunization rates of children within the member health maintenance organizations."
But even more odd was this next sentence: "We did not attempt to control statistically for potential differences between completely unvaccinated children and those with later receipt." Yet the Ministry of Health stated to the select committe in December 2010, that there were: “... no statistically significant differences favoured late or no vaccination.” There were 9 (yes - NINE) never vaccinated children ('fully unvaccinated' is a stupid expression), compared with 1038 children, who were fully (early or late) vaccinated.
So just how “scientific” was this study? It wasn’t for many reasons, two of which are below:
1) The Smith 2010 study omitted the Boniferri adjustment, mentioned in a previous blog.
A community board contributor who appears to be in charge of undergraduate honors students at University, stated in her first post on a community board thread:
“I read the study quickly last night.
Two words: Bonferroni adjustment.”
Later in the same thread she explained:
“Basically, it means that when you run a lot of tests (like the 42+ they ran), you have to adjust the p-value to account for that. The p-value is typically set at .05, which means that for something (a difference between the 2 groups) to be accepted as statistically significant, the probability of that difference being due to chance is less than 5 in 100. Well, if you run 42+ tests, SOMEthing will come up significant, just because you've run that many tests. (and, with a p-value of .05, 5 tests out of 100 would come up significant, just due to chance). So, a Bonferroni adjustment, you divide the p-value by the number of tests you run.
.05/42 = .0011
None of their results would be significant if they had done a Bonferroni adjustment.
A good peer review should have stopped that paper right there. We make our undergraduate honors candidates use Bonferroni adjustments when they run more than 4 or 5 tests!
Although one might argue that it would still be publishable withOUT finding any differences between the lesser vaxed group and the most vaxed group, as this is the type of finding that the CDC would use as "evidence" that vaccines are "safe."
As the study stands now, the 2 groups - lesser-vaxed and more vaxed - were different on SES and mother's education (favoring the more vaxed group), and the differences on the neuropsy tests that favored the vaxed group all but disappeared when they statistically accounted for SES and income. The ONE difference that is left is the one they should not have accepted had they done a Bonferroni adjustment.
HOWEVER, the fact that the lesser-vaxed group had lower SES and lower mother's education means that it is NOT a good comparison group to the lesser-vaxed children of today.”
However, the lack of good solid "scientific evidence" doesn’t stop at just ignoring the Bonferroni adjustment.
2) The fundamental problem is gross selection bias in the study group itself, and extrapolating that “bias” to the great unselected, unwashed masses of children in the world, as did the MOH....
So, who were the children studied?
First up, the Smith 2010 study was simply a rework of the same CDC based population data from Thompson, Price, Goodson et al (2007).
Thompson, Price, Goodson et al (2007) (pdf uploaded onto blog) said
“We enrolled 1047 children between the ages of 7 and 10 years and administered standardized tests assessing 42 neuropsychological outcomes. (We did not assess autism-spectrum disorders.)”
Smith and Woods (2010) (pdf uploaded onto blog) said:
“Publicly available data, including age at vaccination, from a previous VaccineSafety Datalink study of thimerosal exposure and 42 neuropsychological outcomes were analyzed.
In summary, these include assessments of speech and language, verbal memory, achievement, fine motor coordination, visuospatial ability, attention and executive-functioning tasks, behavior regulation, tics, and general intellectual functioning. (Page 1135.)
Easy. That cuts down the research work.
So how was this same study population, which was used in both studies, selected?
Wouldn’t you want the Ministry of health, to know whether or not both the Thompson - and Smith study cohorts - had “sampling methods with a high probability of bias”?
The selection process for the group of children used in both the THOMPSON 2007and SMITH 2010 study cohort is contained in a technical pdf on the CDC website:(pdf uploaded onto blog)
On Page 1 Chapter one, (which is page 9 and 10 of the 179 page CDC pdf), we read this relating to the first THOMPSON 2007 study:
“The primary weakness of the current study is that exposure levels were not determined in a randomized controlled trial (RCT) design. Although the study measured and controlled for a wide range of potential confounders, it is impossible to know with certainty whether the threat of selection bias has been eliminated. Selection bias will have affected the results if one or more unmeasured factors have causal effects on both the amount of exposure that children receive, and outcome measures. Given this important limitation of the design of the study, results can only be judged as informative, not conclusive. The study was intended to be an important contribution to a growing literature regarding the possible effects of ethylmercury, and was not intended to be a definitive concluding statement of whether the ethylmercury in thimerosal-containing vaccine sand immune globulins does or does not cause harm.”
Okay, now move on to page 18, (page 26 in the pdf), where you see HOW these 1047 children eventually used in both studies were chosen:
The sample frame included 27, 240 children, okay?
Of those 27, 240 children 7,982 were selected, okay?
Out of those 7,982 children 3,648 had records scrutinised, to see if they met the criteria to be included in the Thompson 2007 study. Okay?
This is what the Thompson 2007 says about these children:
"Of 3648 children selected for recruitment, 1107 (30.3%) were tested. Among children who were not tested, 512 did not meet one or more of the eligibility criteria, 1026 could not be located, and 44 had scheduling difficulties; in addition, the mothers of 959 children declined to participate. Most of the mothers (68%) who declined to participate in the study and provided reasons for nonparticipation cited a lack of time; 13% reported distrust of or ambivalence toward research. Of the 1107 children who were tested, 60 were excluded from the final analysis for the following reasons: missing vaccination records, 1 child; missing prenatal records, 5; missing data regarding weight, 7; and discovery of an exclusionary medical condition during record abstraction, 47. Thus, 1047 children were included in the final analyses. The exposure distribution of the final sample was similar to the exposure distribution of the initial 3648 children selected for recruitment in the study.
Our study had several limitations. A majority of the selected families declined to participate or could not be located, and we were able to enroll only 30% of the subjects included for recruitment. Therefore, our findings may have been influenced by selection bias. In addition, we were not able to control for interventions, such as speech therapy, that may have ameliorated the potential negative effects of thimerosal exposure and could have biased the results toward the null hypothesis. Given that parents were not trained to assess tics, the parental ratings of tics may have been less reliable than the ratings by trained evaluators. We did not assess exposure to thimerosal beyond 214 days. Finally, the information available for some potential confounding factors, such as family income, which may have resulted in unmeasured residual confounding, was imprecise. Our study did not examine the possible association between autism and exposure to mercury from vaccines and immune globulins."
Page 18 (26 in the CDC technical pdf) lists these 60 excluded mentioned above, as:
Excluded (n=60)
1 – Year 1 care not in HMO;
24 low birth weight, (exclusionary Medical condition.)
7 no weight data,
5 No prenatal exposure data,
23 Exclusionary medical condition (total 47 exclusionary medical condition.)
= 60 excluded.)
(The red above is added to match with the Thompson 2007 exclusionary criteria)
So.... 512 were eliminated because they didn’t meet the INCLUSION criteria. After they got to 1,107 children, 60 more were excluded with 47 of those removed, because they had a medical condition listed in the EXCLUSION criteria.
It also says in the first appendix, that they found
