University of Iowa Study Finds Foobot’s Precision is “Excellent” for Measuring Particulates
Foobot is one of several indoor air quality monitors to reach store shelves in recent years. But in the science and HVAC worlds, devices have been measuring air quality for decades. Being professional-grade, these older devices are incredibly accurate and reliable… but expensive. Decent ones start around $5,000 while many others cost multiples of that.
Home monitors like Foobot don’t need scientific precision to do their job well. Being off by a few micrograms here and there won’t change a recommendation to clean the air, or stop someone from finding a pollution source. That’s why it’s fine to make compromises in the name of affordability. But you can only compromise so much; readings that are too inaccurate or inconsistent are useless because you can’t trust them.
Thanks to an independent study published in the Journal of Aerosol Science, we know that this isn’t the case with Foobot. In a controlled lab environment, they tested Foobot along with two competing air quality monitors in detecting three kinds of aerosols and measured their performance against a $6,000 photometer by Thermo Scientific. In short, they concluded that Foobot’s accuracy was “excellent,” beating both the competitors in precision and consistency.
Before we go deeper into what they found, let’s talk about how a consumer-grade indoor air quality monitor, like Foobot, is different from the expensive ones.
How are consumer IAQ monitors different?
Foobot and monitors like it use much cheaper sensors than their professional-grade brethren. In the past, these sensors varied far too much to be used reliably. But with the advent of the Internet of Things, it’s now possible for a low cost, onboard computer to process the signal from these sensors in a way that accounts for their shortcomings; think of how your brain hears words through mild phone static or an accent. This makes the device far more accurate than the sensors alone.
This also means that the accuracy of the device depends on the quality of the software that’s processing the signals. Just like living abroad can make it easier to understand a country’s accent, algorithms can be tuned and calibrated to paint a clearer picture from the same sensor data. Thus, air quality monitors using similar technology can vary greatly in quality.
The experiment
The lab at University of Iowa tested the devices against three aerosols:
- Salt,
- Arizona road dust,
- Welding fume
They had a Plexiglas chamber divided in half. In one half, they slowly released one of the aerosols and mixed it with clean air. Then, the mixture would pass to the other half, where they put Foobot, the other two consumer monitors and the $6,000 photometer.
What device will have the most accurate and consistent measurement? What’s the difference between…
This let them test each aerosol separately, and at seven or eight different concentrations each (50 micrograms of salt per cubic meter, 100 micrograms of salt, 500 of salt, 50 micrograms of welding fume, etc.) for 15 minutes. Since they already knew the concentrations they were feeding into the chamber, they could see exactly how accurate each device measured the different chemicals and concentrations.
The Results
Here are the results of the salt aerosol test:
The straight diagonal line through the middle represents perfect accuracy: the “real” concentration being identical to the measured one. It’s called the “1:1 line.” You can see that the $6,000 photometer (the “pDR”) lines up almost exactly with the 1:1 line; it gives its money’s worth.
The green line just below it is Foobot’s readings. The fact that it’s lower than the 1:1 line means there’s a “bias,” or a difference in all the measurements in a specific way (here, that it’s lower than reality). However, the fact that it’s such a straight line means it’s highly consistent. If you removed the bias, Foobot’s line would be nearly identical to the photometer’s.
Competitor 1 not only has a greater bias, but curves in different directions depending on the concentration. Competitor 2 is far worse; past 300 micrograms per cubic meter, it barely registers a change even when the concentration doubles.
Foobot did even better on the road dust test. Its readings are even more accurate than the photometer at concentrations under 100 micrograms, and roughly the same as the photometer above that. The competitors did better than with the salt, but still varied significantly as shown below.
Copyright © 2017 Elsevier Ltd All rights reserved. Source: Journal of Aerosol Science
The welding fumes were the hardest, as shown below. All the monitors, including the photometer, had enormous biases. Foobot was much less consistent than in the other trials. However, each measurement went up and down by roughly the same amount and in the same direction as the photometer. This means that, while Foobot’s readings of welding fumes are less accurate, it’s still roughly on par with devices that charge 30 times as much.
Copyright © 2017 Elsevier Ltd All rights reserved. Source: Journal of Aerosol Science
The researchers concluded that “For all aerosol types, PM2.5 measured with the Foobot and [the photometer] were highly linear with that from reference instruments, whereas PM2.5 from [Competitor 1 and 2] were not.”
The consistency between readings is most important because it’s the change that indicates that something happened to the air quality in a room. Whether the concentration jumped from 200 to 600, or 180 to 540, doesn’t matter. The fact that it tripled is what’s telling.
For all aerosol types, PM2.5 measured with the Foobot and [the photometer] were highly linear with that from reference instruments, whereas PM2.5 from [Competitor 1 and 2] were not.
Scientific where it matters
If you’re a scientist measuring precise air conditions, you can see from its biases that Foobot is not a scientific instrument. It was never meant to be. But that’s okay. Every breath of air taken in every classroom, kitchen or cradle matters to someone’s health. We can’t possibly leave monitors in all those places when each one is the cost of a used car. We need monitors that are affordable but still accurate enough.
When Foobot says there’s a problem, there is one. When Foobot says you’re safe, you are. That much is trustworthy. It’s priced so that anyone can afford it, and it’s easy to use even if you’ve never worn a lab coat. Professional-grade equipment has an important place in advancing our understanding of pollution. But it’s Foobot that has a place in our daily lives.