by Jack Springston–
It sometimes amazes me how people go about doing testing for indoor air quality surveys and then interpreting the data that they collect. Often, individuals will approach a problem with their own preconceived notions on what is causing the issue, and then go about doing testing to prove that they are correct. For instance, say you have a building where some of the occupants are complaining about allergies, watery eyes and upper respiratory irritation. An IAQ consultant then comes to the building, find a couple of water stained ceiling tiles with suspect microbial growth and concludes that it’s obviously a mold problem. He then goes about collecting a few spore trap samples, along with maybe a tape lift sample or two from the ceiling tiles. The tape lift sample results come back showing that there is mold growth present (no duh) and the air sample results indicate that Aspergillus/Penicillium counts for the two samples that he collected inside are slightly elevated compared to the single outdoor sample that he took as he left the building. Based on those results, the IAQ consultant proclaims that, yes indeed, what we have here is a mold growth problem and you need to remove the ceiling tiles and HEPA vacuum the entire area.
Folks, let me tell you right now that this is just plain wrong. When it comes to hypothesis testing, as Beatrice in the Esurance commercial says, “That’s not how it works. That’s not how any of this works.” By centering on a single concern, and performing sampling that only focuses on trying to prove that that particular issue is the culprit, you have completely ignored all the other potential contributing factors. In the example above, which is based on an actual problem that I was involved in years ago, the real culprit was pet dander that was being brought in on a new employee’s clothing – not a couple of water stained ceiling tiles. It seems that the woman was a real cat lover (she had something like a dozen or so at home!) and several of her co-workers were quite allergic to cats.
So, how do we go about doing valid hypothesis testing for indoor air quality surveys? Well, in order to do so, we first need to understand the concept of the null hypothesis and what types of errors we may encounter when doing our testing.
Testing and the Null Hypothesis
In either statistics or observational data, the null hypothesis generally refers to a default position that there is no relationship or correlation between two different observed or measured singularities. Typically, the null hypothesis is assumed to be true until, and unless, evidence indicates otherwise. Testing is then performed, not to prove that the null hypothesis is correct, but rather to try and disprove – or nullify (hence the name ‘null’) – the hypothesis. Depending upon our data, we can potentially reject or disprove the assumption, but we do not ever say that the null hypothesis is accepted or proven, regardless of what the data may seem to show.
If you are having difficulty wrapping your head around the whole concept, one way that you can think about it is our criminal justice system. When a defendant is brought to trial for a crime that they have allegedly committed, the null hypothesis is that they are innocent. The prosecution then presents evidence (data), to try and reject the null hypothesis and show that the defendant is guilty. However, if the evidence presented by the prosecution is not sufficient enough to sway the jury into rejecting the null hypothesis, then the verdict is that the defendant is “not guilty,” not that they are “innocent.” That is a big difference.
For instance, let’s say that you wanted to test and see if there was a correlation between the number of people in a space and the carbon dioxide levels. Now, just about everyone knows that people exhale carbon dioxide and the more people that you put into an enclosed space, the greater the concentrations of carbon dioxide within that space will be. But suppose that you are really bad at designing experiments and failed to bother checking the ventilation rate within the space, which could clearly skew your results. Also, you really couldn’t find that many volunteers who wanted to sit around in the room while you did sampling, the only available space that you could find to do your testing was an airport hangar, and the sampling method that you chose to use had an error rate of +20 percent. Because of all these confounding issues, none of which you were cognizant of, you found no difference in carbon dioxide levels when there were no people within the hangar versus when you had five people present. Does that prove that there is no correlation between the number of people in a space and the carbon dioxide levels? No, all it proves is that you really need to consider changing careers.
Applying Null Hypothesis Testing to IAQ Surveys
Any complaint-driven IAQ investigation begins with an initial statement of definition of the problem. For many investigations, this statement may be as simple as “The occupants in office X are complaining about the air quality.” In order to scientifically test the assertion that there is a problem with the air quality, the investigator continues the process of inquiry through adopting the null hypothesis (H0) and the collection of data. For example, the null hypothesis may be “There are no discrete air quality problems in office X.” Next, one or more possible explanations for the cause of the air quality complaints are stated in the form of alternative hypotheses (Ha1, Ha2, …Han), such as “Volatile organic compounds (VOCs) are the cause of the air quality complaints at office X” or “Lack of ventilation is the cause of the air quality complaints at office X.” Often times, the building occupants may suggest possible causes for their complaints during the initial stages of an investigation, which can greatly assist in the development of alternate hypotheses.
As previously noted, the goal of the ensuing investigations and testing is to try and disprove the null hypothesis. So, using the example given above, let us suppose that you believe that VOCs are the root cause for IAQ complaints in office X. You then go and design a sampling strategy that includes collecting air samples for specific VOCs using evacuated stainless steel canisters in both the complaint and non-complaint area(s), as well as outside the building. Remember, though, the purpose of the testing is not to prove that VOCs are the cause of the complaints, but rather to reject the null hypothesis that there are no discrete air quality problems in office X. If you do testing, and do not find any unusual concentrations of VOCs in the office, that does not prove that the null hypothesis is correct. All that proves is that you need to dig deeper and develop and test new hypotheses.
On the other hand, if you do testing and find elevated levels of VOCs, then you can, with some degree of confidence, probably reject the null hypothesis and conclude that yes, there is a problem with the air quality in office X. However, that testing does not necessarily prove your hypothesis that VOCs are the root cause of the problem. All it proves is that you have unusual levels of VOCs. Quite often, IAQ problems can be caused by multiple different issues, and not just one single contaminant or problem. You may very likely resolve the VOC issue, only to find out that people are still complaining about the air quality, at which point it is time to develop, and test, a new hypothesis.
Types of Error
When doing hypothesis testing and deciding whether or not to reject the null hypothesis, there are two basic types of errors that can occur. The first type of error, or Type I error, is also known as a “false positive” with respect to the non-null hypothesis. In other words, this is an error where the true null hypothesis is rejected and the alternate hypothesis is incorrectly accepted. For instance, the investigator incorrectly concludes that mold is the problem when, in reality, it is animal dander, or a defendant is convicted for murder when, in fact, it was a one-armed man who did it.
The second type of error, Type II error, is often referred to as a “false negative.” In this instance, the testing data fails to reject a false null hypothesis. For example, office workers are complaining about headaches, fatigue and malaise that comes and goes. An IAQ investigator does some testing for general air quality parameters, all of which are within acceptable limits, and concludes that there is no problem with the air. What the investigator failed to catch, however, was that the outdoor air intakes for the HVAC units are located by the loading dock and that delivery trucks occasionally idle next to the intakes. If the investigator had done data logging for carbon monoxide over an extended period of time, rather than just collecting some real time grab samples, he likely would have noticed an occasional spike in carbon monoxide levels in the offices where the complaints have been occurring and been able to correlate these spikes to the delivery trucks.
|Test says you don’t have it||Test says you do have it|
don’t have it
|TRUE NEGATIVE||TYPE I ERROR|
|You really |
do have it
|TYPE II ERROR|
The best way to try and avoid such errors is to collect lots of data. The more data you collect, the less likely it is that you will have either a false negative or a false positive result. However, as a professional, you need to avoid using the shotgun approach to trying to solve IAQ problems. Testing for everything and anything is unnecessarily burdensome and can be prohibitively expensive. If you are unsure about what you should be testing for, or do not know how to properly interpret the results from tests that you have taken, then the likelihood that you will draw a wrong conclusion, and make either a Type I or Type II error, can rise significantly.
So, the next time that you get a call from someone saying that they have an IAQ problem that they need solved, remember to use the scientific method of investigation and do not allow yourself to become fixated on one, single hypothesis when determining causation.
Jack Springston, CIH, CSP, FAIHA, is a Senior Project Manager at TRC Companies and Past-Chair of AIHA’s IEQ Committee. Jack is an industrial hygiene consulting professional with over 28 years of experience in recognizing, evaluating, measuring and controlling employees’ exposures to health hazards in their workplace. He has been a Certified Industrial Hygienist since 1993, and is one of only approximately 60 CIH’s who also currently have a sub-specialty certificate in Indoor Environmental Quality, and a Certified Safety Professional since 1996. Jack has participated in and overseen hundreds of indoor environmental quality studies, both investigative and proactive, in over 50,000,000 square feet of building space.