Objective analysis of ELISA S/P results with mean-and-range charts
Research has demonstrated that pathogen test results objectively describe a population’s average antibody response as stable or unstable
By David H. Baum, Luis Giménez-Lirola, Jeffrey J. Zimmerman
Analysis and interpretation of pathogen surveillance data from livestock production sites can be greatly improved using methods that account for the variation historically associated with the production site, the sampling method, and the surveillance test platform. Mean-and-range charts [2] are designed for this purpose because they integrate the most recent set of test results with prior test results. In brief, mean-and-range charts place the current surveillance test results’ means, ranges, and dispersion (standard deviation) in the context of the same farm’s previous surveillance results’ means, ranges, and dispersion (standard deviation). Juxtaposing the current test results with the historical results accounts for variation inherent to the sampling-testing process and shows if - and - when the farm’s pathogen infection status (process) has changed. Herein we illustrate this approach using test data from a commercial farm’s surveillance for an undisclosed pathogen they believe was eliminated from the herd. These sampled farms’ results had been used by the farm to represent their gilts as pathogen-negative at sale.
We demonstrate that mean-and-range charts from pathogen test results objectively describe a population’s average antibody response as stable or unstable and that the charts are used to determine if the pathogen is detected or not detected in the population. Specifically, mean-and-range charts will objectively describe a pathogen’s status within populations following attempted pathogen elimination: whether the pathogen was eliminated and if it has remained eliminated. We do not suggest nor imply success or failure of method to eliminate the pathogen. Rather, the reader will learn how to interpret pathogen test results post-elimination as they arise from that population. Thus, we do not mention the name of the pathogen.
The farms were chosen for no other reason than they regularly submitted surveillance samples to the Iowa State University Veterinary Diagnostic Laboratory (ISU VDL). They are also within a multi-site swine production system. Each farm collected monthly serum samples for pathogen surveillance. Farm 1 and Farm 2 produced replacement gilts for commercial swine production. The customers that purchased replacement gilts did so on the assurance that a particular pathogen was not present in the gilt multiplication farm nor in gilts they purchased. Farm 3 was the gilt isolation facility for Farm 2. Thus, it housed replacement females that were purchased from another farm that was not known to be infected with the particular pathogen. All three farms became infected with the pathogen and all three farms implemented a pathogen elimination strategy.
Each farm’s pathogen surveillance serum samples were collected every four – to – six weeks and submitted to the ISU VDL for pathogen testing. Thus, there are some dates without an average or a range symbol on their respective chart. A surveillance serum sample set consisted of 10 – 81 individual sow samples. The 3-sigma limits are wider with fewer samples than when more samples were tested. These data were obtained from the ISU VDL Laboratory Information Management System (LIMS). The manufacturer’s instructions for pathogen antibody test kit define how optical densities (OD) of the samples and kit controls were used to calculate sample-to-positive ratio (S/P) for each sample’s result. S/P values are interpreted as positive when S/P > 0.4. An mean-and-range chart consists of an average graph and a range (R) graph. The x graph is a time series plot (x) of each surveillance sample set’s average S/P value, a central line calculated from all sample set averages, and 3 sigma limit boundaries (3 standard deviations) on each side of the central line. The R graph is a time series plot of each surveillance sample set’s range (the difference between the highest and the lowest S/P values in the sample set), a central line of all sample set ranges (average range), and 3 sigma limit boundaries on each side of the central line. The average range is used to estimate 3 sigma limits for both graphs.
Mean-and-range calculations of means, ranges, and limits were plotted according to methods described by Wheeler and Chambers[1]. The most recent S/P means (x) are interpreted in the context of the historical surveillance data, i.e., the S/P central line, the central line of ranges (R) and ±3 sigma limits indicating the out of bounds of the data [2]. The mean-and-range charts were examined for evidence of changes in result (S/P) variation [1,2], i.e., showed new patterns of variation in monthly testing data averages or ranges and/or contained data points located outside the 3-sigma boundaries. Mean-and-range chart interpretation is based upon the absence or presence of signals indicating lack of control [2]. In the context of pathogen surveillance, a chart with no signals is interpreted as “no change in pathogen status”; a chart with signals is interpreted as ”changing pathogen status”. Signals are defined by 4 rules of interpretation: 1) a data point falls outside a 3 sigma limit (Shewhart’s rule) or a specific pattern of distribution is observed (Western Electric zone tests [2]. 2) 2 of 3 successive values are on the same side of the central line and more than 2 sigma from the central line, 3) 4 of 5 successive values are on the same side of the central line and more than one sigma from the central line, or 4) 8 successive values located on the same side of the central line [2]. Signals on either the x or R chart indicate the need for limit and average recalculation. Recalculation is also in order when a known system change was made that would be expected to change the test results’ distribution. If no such evidence was observed, the sow herd's pathogen status was interpreted as stable and unchanged.
The central lines and limits of each sow farm’s first mean-and-range chart were calculated globally, i.e., for the entire data set. Because nearly all sample set S/P averages or R averages were outside the 3 sigma limits or they displayed patterns of distribution, the limits of the x and R charts were recalculated according to those signals’ appearance on the charts.
Figures 1 – 3 demonstrate this process from Farm 1’s dataset. Figure 1 shows monthly pathogen ELISA averages and ranges, their respective S/P or R central lines, and 3 sigma limits for the entire data set (global). Data points above or below the 3 sigma limits indicate that the pathogen antibody status of the sow farm was unstable. Table 1 is a summary of the Figure 1 global statistics.
Three pathogen ELISA average S/P patterns are observed in Figure 1. First, for the periods of Jan 2016 – May 2016 (Pattern 1), then Jun 2016 – Dec 2016 (Pattern 2), and Jan 2017 - Nov 2023 (Pattern 3). The statistics of these patterns are summarized in Table 2 and coincide with the original known pathogen – positive status, intentional herd exposure/re-exposure to pathogen, and depopulation followed by repopulation with pathogen -negative animals, respectively.
Figure 2 shows predictable (stable) distributions of ELISA result distributions in pattern 1. Pattern 2 contains 2 values below the lower out-of-bounds limit and 2 other values above the upper out-of-bounds limit. Pattern 3 has one out-of-limit monthly average and 4 out-of-limit ranges for January 2017 – February 2020. Thus, Pattern 3’s limits were recalculated for the periods of January 2017 – March 2018 and for April 2018 – February 2020. Pattern 3 of Figure 2 contained 3 distinct patterns of data distribution, the statistics of which are summarized in Table 3 and shown in Figure 3.
Figure 3 shows these additional patterns with their averages, ranges, and 3 sigma limits. Figure 3 includes the averages and ranges for these date ranges: January 2017 – March 2018, and for April 2018 - February 2020. One signal (R = 0.400) appears in the monthly range chart in December 2019. Table 2 summarizes the number of signals present in Figure 3. There were no signals in the x chart from Jan – 17 through Feb – 20, i.e., the herd was stably pathogen -negative. There was 1 signal in the R chart from Jan – 17 through Feb – 20.
Mean-and-range charts are said to be most efficient at detecting medium-to-large spike-type changes in values [3], although what is meant by “medium-to-large spike-type changes in values” is undefined by the authors. Regardless, the 4 rules of interpretation, i.e., any spike outside 3 sigma deviations plus the 3 other data distribution patterns previously described, are sufficient for interpreting mean-and-range charts [3]. In this report, we demonstrated how they can be used to characterize the pathogen status of a sow herd and how to account for non-specific reactions. That is, the pathogen ELISA S/P value mean-and-range charts can characterize the sow herd’s pathogen status from known-positive, through intentional pathogen exposure/infection, immediate post-depopulation, and later post-depopulation. These different pathogen statuses were defined by their distinct patterns of average S/P distributions, ranges, and 3 sigma limits.
The x portion of the mean-and-range charts (Figure 3) was stable during the known-positive, immediate post-depopulation, and later post-depopulation portions. The x portion of the mean-and-range chart of the intentional pathogen exposure/infection portion had unstable S/P averages. This is explained by the intentional exposure and re-exposure of the sow herd to pathogen in May-16 and in Sep-16. Each of these events was followed by instability in their portion of the x chart. This instability of the herd’s pathogen antibody confirmed that intentional exposure resulted in pathogen infection. Thus, this is an iatrogenic antibody kinetic curve and is evidence of pathogen re-infection. No such curve is observed in the x chart after the depopulation in January 2017. The later post-depopulation data did not display the instability of intentional exposure/infection, and antibody kinetic curve and a mean pathogen ELISA S/P of 0.123. Thus, the x shows no evidence of any pathogen infection in Farm 1 since its depopulation/repopulation.
The R portion of the mean-and-range charts was stable during the known-positive, intention exposure/re-exposure, and immediate post-depopulation portions of the herd’s pathogen surveillance. The post-depopulation section of the R chart showed six range values that were greater than the upper range limit. There are 6/81 range values that are outside of the upper range values. These are interpreted as non-specific results and not associated with any evidence of pathogen infection. Thus, the diagnostic specificity of the pathogen S/P for this herd can be estimated from the R chart. For the entire post-depopulation data was 0.926. As time rolls on, the diagnostic specificity remains 1.000 from June 2021 through November 2023; 29 ELISA-negative among 29 results from a herd expected to be pathogen-negative. Additional evidence of absence of infection is seen with the steadily declining pathogen ELISA S/P average ranges (Table 3).
Farm 2 Breed – to – Wean and its Gilt Isolation pathogen ELISA S/P charts are shown in Figures 4 and 5 after their regions were identified and limits/averages were recalculated. Farm 2 (Figure 4) pathogen S/P x had no evidence of pathogen infection prior to January 2022. There were 3 instances of nonspecific R values during the same 40-month period. An antibody kinetic curve begins in both the x and R chart on January 2022 and lasts through April of 2023. The last two data points of each are below their lower control limits. Additional months of sampling are needed to determine if the farm no longer infected with the pathogen.
Farm 2 Gilt isolation does not show evidence of pathogen infection in the x chart until April of 2022 when a pathogen antibody kinetic curve appears until November 2022. The x chart is unstable with 6 out – of – limit averages. The R chart is also unstable and averages around the positive cutoff (0.400) of the pathogen ELISA.
These mean-and-range charts characterize each of these farms by their pathogen ELISA S/P results. Farm 1 depopulated its infected population of animals and repopulated with expected-negative animals. In the course of the course of the next six-plus years, its pathogen negative status had been reasonably maintained. “Reasonably” is defined as no evidence of pathogen infection even with some out-of-limits ranges. Out – of – limit ranges can be problematic for the farm because its clients may have zero-tolerance for any animal that non-negative on the pathogen ELISA and have no interest in a reasoned discussion. Rather than sacrifice animals to prove a negative (a fool’s mission) and since these out – o – bounds ranges are tell the farm that something has entered it, the farm would do well to understand what other, non-infectious, activities or things might be associated with their appearances by producing non-specific reactors on the pathogen ELISA. Among which could be commingling, vaccination with gram-negative bacterins, vaccination with products containing potent adjuvants, off – test measurements of phenotypes, or changes in feed ingredients.
Farm 2’s pathogen ELISA S/P results might raise suspicions of the pathogen status of incoming gilts housed in their gilt isolation. Thus, Farm 2 should request the pathogen ELISA S/P results from the farm(s) that provide its replacement gilts.
Finally, mean-and-range charts are an objective method for assessing the pathogen status of a population. Unfortunately, they do not interpret themselves for the reader.
References
1. Wheeler, D.J.; Chambers, D.S. Shewhart's Control Charts. In Understanding Statistical Process Control, Second ed.; SPC Press, Inc: Knoxville, TN, 1992; pp. 37 - 54.
2. Wheeler, D.J. Detecting a Lack of Control. In Advanced Topics in Statistical Process Control. The Power of Shewhart Charts; SPC Press, Inc: Knoxville, TN, 1995; pp. 135-136.
3. Yahav, I.; Lotze, T.; Shmueli, G. Algorithm Combination for Improved Performancer in Biosurveillance: Univariate Monitoring. In Integrated Series in Information Systems 27, Sharda, R., Voß, S., Eds.Zeng, D., Chen, H., Castillo-Chavez, C., Lober, W.B., Thurmond, M., Eds.; Infecious Disease Informatics and Biosurveillance; Springer: New York Dordrecht Heidelberg London, 2011; Volume 27, pp. 173 - 189.
The authors are with Iowa State University Veterinary Diagnostic Laboratory, Ames, IA. Cover image from Getty Images.
