Ever heard of AD-X2? There's no reason you should. It was useless. But that didn't stop people for calling for tests on it for over a decade, and kicking up a scandal that (unjustly but temporarily) claimed the job of the director of the National Bureau of Standards.
It was 1947, and World War II had been a drain on the nation's resources. Among those drained resources was lead, and among the things made with lead were car batteries. At a time when more and more people wanted cars, this proved expensive, so anything that would make car batteries last longer was valuable. There were a lot of worthless stuff masquerading as valuable in 1947, and the National Better Business Bureau took some samples of various kinds of battery-life-extension products over to the National Bureau of Standards to be tested. Each product proved useless, and its ingredients were noted as the sort of thing that doesn't lengthen battery life.
One product wasn't tested. AD-X2 was not patented, and so its ingredients didn't have to be announced. Its inventor, Jess Ritchie, was so confident in his product that he asked the National Bureau of Standards to test it. The National Bureau of Standards turned him down, in a decision that shows that doing the right thing often leads to regrettable results. The institute wasn't a product-testing service; it was a government agency designed to prevent fraud, so if Ritchie wanted his product tested, he would have to do it elsewhere.
Ritchie decided that that shouldn't be the case and begin to spin a story. Instead of a government agency testing popular products to prevent fraud, the NBS became a big government agency refusing to serve the small businessman - possibly because they were in the employ of Big Battery. (If batteries ran out faster, battery manufacturers would sell more of them, after all.) Ritchie gathered support until the Federal Trade Commission asks the NBS to test the product. The NBS did. Twice. AD-X2 was useless.
That left a lot of people, including some state senators, with a rather unpleasant choice. They could either admit that they had, with the best intentions, supported someone who was completely wrong, or they could clamor for more testing. They clamored for more testing. The NBS tested AD-X2 again in 1953. The product was still useless.
This is when things got complicated. Ritchie got AD-X2 tested by scientists at M.I.T. They found that it was still pretty much useless. They had tested it under a different set of standards, so their tests results were slightly different from the NBS results. The supporters of Ritchie and AD-X2 said that this showed the NBS was involved in something shady, and called for more testing, this time by the National Academy of Sciences. They also called for the resignation of Allen Astin, the director of the National Bureau of Standards. They got both. In short order they found out that AD-X2 was still useless. And the Academy of Science was incensed by the idea that Astin had to resign for doing honest, repeated tests of AD-X2. Astin was reinstated.
And AD-X2 was still championed by those who wanted further testing. The furor lasted well into the 1960s. When the product did die out, it wasn't because the matter was settled, but because people didn't see the need for it anymore.
There's no denying that sometimes products or concepts require extensive testing. AD-X2, however, is a classic example of the stalling technique that some people use when they dislike a certain concept but can't think of a reason why. There are always more questions that need to be answered, always a more comprehensive test that needs to be done. There's always a reason to call the testing agency into question. And, if even one test in a thousand indicates the desired results, there is no amount of 'further testing' that will wipe that one test from supporters' minds. Testing can be important, but it can be a way to put off further action, or mitigate personal embarrassment. AD-X2 ended up costing the public so much money in testing that it might have been cheaper to let the fraud slide. It would have been cheapest, though, to have accepted the results of the first test.
[Via Bogus Science, Measures for Progress, AD-X2 Battery Additive]
Entities 0 Name: NBS Count: 6 1 Name: Ritchie Count: 5 2 Name: National Bureau of Standards Count: 5 3 Name: Astin Count: 2 4 Name: Measures for Progress Count: 1 5 Name: Academy of Science Count: 1 6 Name: Allen Astin Count: 1 7 Name: M.I.T. Count: 1 8 Name: Via Bogus Science Count: 1 9 Name: Federal Trade Commission Count: 1 10 Name: National Better Business Bureau Count: 1 11 Name: National Academy of Sciences Count: 1 12 Name: Jess Ritchie Count: 1 Related 0 Url: http://ift.tt/1l9b4IR Title: 10 Things I Learned From Taking 100 Usability Tests Description: Usability testing is a technique to evaluate a product or service by testing it with users. The users work on tasks while observers take notes, listen, and learn. After taking more than 100 usability tests (I took them all on Usability Hub), I've noticed some interesting things about how people are using usability tests today.
Post a Comment for "This Classic Science"