Too much information

Humans like simple, straightforward information.
For example, when we are ill and do not know why, we likely would go to the local doctor for help. The doctor might order one or more tests, and we might patiently await results. Most of us like results like positive or negative, and do not like results like most likely or maybe. However, in medicine, whether human or veterinary, even a simple positive does not necessarily mean certainty; thus, we need to be careful in interpretation. We veterinarians regularly see clients use interpretations of test results in ways that might cause one to make the wrong decision because it is hard for us to understand why positive might not actually mean positive, for example.
Tests have some inherent characteristics. Sensitivity refers to the ability of a test to correctly find a positive result. For example, if you use on-farm culture to diagnose mastitis pathogens, the culture system might have 70% sensitivity to detect Staph aureus. That means the test would detect 70% of the truly positive samples. Specificity refers to the ability to correctly identify a negative result. So, for example, if the culture system has a specificity of 90%, it would correctly identify a negative result 90% of the time, but 10% of the time it would identify a true negative sample as Staph aureus positive. Tests also have something called positive and negative predictive values, which mean the actual sensitivity and specificity depend on the true prevalence in the sample population. This means, for example, that a test for Johne’s disease that has a specificity of 90% might not actually find 90% of the true positives every time. If the population of tested animals have a really low rate of true infection, the positive predicted value might be lower, and if nearly all of the animals actually have Johne’s, the positive predictive value might actually be substantially higher than 90%. This can be hard to get one’s head around, but it is important.
Let’s go back to the on-farm culture test. The Minnesota Easy Culture System II is probably the most widely used and best on-farm milk culture system. This test has two available systems, a Bi-plate and a Tri-plate. Perhaps we have a result that we think is Streptococcus uberis. We make the diagnosis and treat the cow according to the on-farm protocol. Is this appropriate? Unfortunately, it is not. According to a paper by Royster, et. al., (Journal of Dairy Science, 97:3648-3659), the Bi-Plate and Tri-Plate had high or intermediate sensitivity for the broad categories of no growth, gram positive, gram negative, Staph. aureus and Streptococcus spp., but both tests had poor sensitivity and specificity to identify more specific categories, such as Strep uberis. From the paper, “The results of this study suggest that using the Bi-Plate or Tri-Plate to identify specific bacterial species, other than Staph aureus, will frequently lead to inaccurate results and should not be attempted.” Thus, our test result may not actually be correct, and we should not make the diagnosis of Strep uberis even though the interpretation looks pretty clear.
There is another problem with test results. Let’s say you want to compare inseminators’ performance on your farm, and you decide to look at each breeder’s conception rates. You are pretty sure that each breeds the same type of cows, e.g. standing heats versus timed breeding, so you think that Joe’s conception rate of 52% is much better than Tony’s rate of 46%. You are likely correct in judging that the sensitivity and specificity as well as positive and negative predictive values of pregnancy diagnosis are very high, especially when combined with observations over time, such as a cow coming back into heat or not. However, the problem here is the certainty of knowing the results are really different might be very low. How can this be? The answer lies in the inability of a cow to be a little pregnant or a little open. Pregnancy is what we call a binomial outcome, meaning yes or no, or one or two. What if Joe and Tony each only inseminated 15 cows? There are not many possibilities of results with so few cows. In fact, one might need hundreds of breedings to actually tell the difference between a 46% and 52% conception rate. To understand this better, we use something called confidence intervals. Anyone who has looked at the BREDSUM command by inseminator in DairyComp305 should have noticed those little bars on the graph of results. The bars represent the “real” results, meaning the results we can be sure are correct with 95% confidence. So, for example, if Joe’s bars range from 44% to 60% and Tony’s range from 38% to 54%, the bars overlap, and we cannot say the results are actually different. We should make decisions based on these results with great caution.
These are just two examples of test results that may lead us to make the wrong decision. These results are really too much information and should be ignored rather than acted upon. For help interpreting any test results on your dairy, ask your veterinarian. They are experts in this area and can help.
Bennett is one of four dairy veterinarians at Northern Valley Dairy Production Medicine Center in Plainview, Minnesota. He also consults on dairy farms in other states. He and his wife, Pam, have four children. Jim can be reached at [email protected] with comments or questions.