The Wisconsin Veterinary Diagnostic Lab - WVDL - involved in controversy over interpreting CWD test results and public disclosure.

An article on Wisconsin CWD test results by Associated Press reporter, Robert Imrie, has brought to light concerns over test procedures, test accuracy, and the reliability of test information released to hunters and the general public. The controversy surrounds 117 deer that initially tested positive on the new IDEXX rapid-screening test put into place by the DNR for the 2003 season. Because the older IHC test did not confirm those positive results, the protocol followed by the agency and its testing lab (WVDL) determined that the IDEXX test results were false. In fact, 74% of the IDEXX positives were eventually determined to be false - only 26% were confirmed by the older test. This low correlation between the two tests flies in the face of the results of the USDA trials of the test that showed almost 100% agreement between the two tests. Without a high correlation between the new IDEXX test and the older IHC test, the IDEXX would not have been certified for use.

Among the sources contacted in the Imrie investigation were some leaders in a DNR watchdog group called CALFARR (Citizens and Landowners for a Rational Response) who have long opposed the extreme response by the agency to the discovery of CWD in southern Wisconsin. The group has been conducting their own investigation into reporting failures and inconsistencies. On March 9th, they lodged a formal appeal with State Senate leader Erpenbach claiming a "hunters right to know" ALL of the results of the screening tests - not just those the DNR chose's to release. CALFARR believes that hunters who voluntarily submit samples for testing are entitled to full disclosure of test results - not a filtered version.

The IDEXX test only screens for CWD Suspects?
The DNR and its Lab - the Wisconsin Veterinary Diagnostic Lab - are arguing that the IDEXX test only produces a "suspect" result and thus it would be misleading or confusing to report such results to hunters.

What is odd about this position is that IDEXX itself makes quite strong representations of the accuracy of its CWD test. Its web site reports that the "proven IDEXX ELISA" CWD test is extremely accurate: " . . . 98.8% sensitivity and 100% specificity (sensitivity validated through IHC confirmation testing)."

IDEXX Laboratories claims on their website that every deer from data sets used in IDEXX test validation with an absorbance value over .21 has also tested positive using IHC testing. This represents a specificity of 100%. The confirmation testing data the company used to gain USDA certification are on file with the firm. Positive and negative controls are provided in each test. Positive controls demonstrate the presence of CWD and react to give IDEXX values >0.40, and negative controls demonstrate the absence of CWD and give IDEXX values less than 0.15.

IDEXX ELISA technology could offer far greater sensitivity and reliability than the older IHC methods. The older IHC method suffers from potential sampling error plus it involves human subjectivity in interpreting the results. In IHC, a paper thin slice is taken from the tissue for microscopic examination. There is always a possibility that the paper thin slice fails to capture deformed CWD prions that may be present in the larger homogenized tissue sample used for IDEXX testing. Several sections must be taken through various areas of the same tissue used for ELISA in order for IHC to be accurate. If the error doesn't occur at this sampling stage, it may occur in the reading of the slide. While science based, this reading is an art form that depends greatly upon the technician's experience and intellectual discipline. As in radiology, two Radiologists may disagree on the interpretation of an x-ray film.

ELISA technology on the other hand - uses a larger tissue sample and homogenizes it. Thus it involves a more representative sample of the larger piece of tissue. The equipment provides a digital readout allowing no room for subjectivity or individual differences in technician experience or competence. A reading of .41 will be what it is no matter who reads it.

Public records from the WVDL of IDEXX test result data showed 26 deer deemed "negative" for CWD by the Lab & the DNR had IDEXX results were more than twice the IDEXX CWD positive threshold! And 5 of these "negative" confirmations exceeded the IDEXX positive optical density threshold by a factor 5! (the positive threshold is an optical density reading of .21 . . . 5 of the rejected samples had densities exceeding 1.05).

So considering the ELISA design of the IDEXX test should reduce subjectivity and other sources of error, if a reading of 1.05 - 5 times the "suspect" threshold - turns out to be negative, wouldn't that cause a reasonable person to suspect something isn't working properly? Is the basic test is badly flawed in its construction or were gross errors were made by the technician? And if such a large discrepancy occurs - not just once - but five times, isn't something clearly amuck?

It would seem that unless the WVDL discovers and admits the possibility of technician error, it must consider that the IDEXX test itself may be unreliable. We know that the record of testing in Colorado using a similar ELISA test kit by an IDEXX competitor, Bio-Rad, has a near-spotless record of confirmatory testing (only 4 times in 47,000 cases did the Bio-Rad ELISA test and the IHC test fail to agree).

How Golden is the Gold Standard?
The DNR and the WVDL often refer to the older IHC test as the Gold Standard - a test whose accuracy and reliability is beyond question. Thus we were surprised when reading a Colorado State CWD Surveillance study to come upon the following statement:

"Most reference tests used in validation studies are imperfect and are incorrectly termed “gold standards”. Many authors have shown that the use of these imperfect reference tests for the calculation of sensitivity and specificity of new tests result in estimates that are biased (Staquet et al., 1981; Enoe et al., 2000). In the simple case, where the reference test is imperfectly sensitive but perfectly specific (e.g., bacterial, viral, or parasite isolation), the sensitivity and specificity of the new test will likely both be underestimated."

This same study report also explains the difficulty in properly interpreting IHC results:

" . . . The IHC is time consuming and requires specialized technical skills. Furthermore, IHC interpretation is subjective and can be influenced by prior knowledge of the case and associated results from the screening tests. Several new rapid screening tests are being validated to complement available techniques for the purpose of large scale screening for these diseases. Some of these tests have been validated and are currently being applied in disease control programs in several countries. Other tests are currently being validated for both screening and confirmation of these diseases.

All of these tests require the collection of appropriate clinical specimens, mainly the appropriate obex site of brain tissue, or the region proximal to the cortex in lymph nodes. Selection of the appropriate clinical specimens (tissue type, amount of tissue collected, etc) is an essential component if these tests are to have good diagnostic accuracy. Furthermore, it is known that the prion agent of these diseases tends to be unevenly distributed in the tissue specimens."

What is your testing objective?
Our internet research on this subject found an interesting a course outline on VETERINARY EPIDEMIOLOGY (Caution - long download time) that discussed, among other matters, how research objectives determine the type of procedures used. In employing a two test protocol, the lab may chose to run the tests parallel to one another or in series:

"Parallel interpretation
With parallel test interpretation, an animal is considered to have the disease, if one or more tests are positive. This means the animal is being asked to ‘prove’ that it is healthy.

Series interpretation
With serial test interpretation, the animal is considered to have the disease if all tests are positive. In other words, the animal is being asked to ‘prove’ that it has the condition."

From this it seems clear our Wisconsin officials are following a "series interpretation" strategy - thus the animal must test positive in both tests and "prove" it has the condition. It is not being asked to "prove" it is healthy. Thus our procedure means we can be very, very confident that every case we deem CWD positive is "in fact" CWD positive. We want to be quite sure we have no false positives.

On the other hand our procedure allows room for the possibility that some CWD negatives are "in fact" CWD positives. This is the direct result of our testing objective aiming to insure against false positives.

Thus we have a conflict of testing objectives. The DNR wants to be absolutely sure that when a deer is pronounced CWD positive, it is in fact CWD positive. Hunters - who request tests - want to know that when their deer is pronounced CWD negative, it is in fact CWD negative!

Another example of the needs of the Agency bureaucracy taking priority over the needs of hunters.

--Ross Reinhold, April 13, 2004 & April 29, 2004
roscoe@mhtc.net

Some information for this article was obtained from interviews with Mark Peck, Anthony Grabski, Ph.D. and Phil Muehrcke, Ph.D.

Internet References:

Colorado State CWD Surveillance study
http://www.cvmbs.colostate.edu/aphi/CWDRpt.pdf

Course outline on VETERINARY EPIDEMIOLOGY
http://www.vetmed.ufl.edu/classes/vem5503/spring2004/lectures/documents/Veterinary_Epidemiology_VEM5503_2004.doc

Wisconsin Veterinary Diagnostic Lab News Release
http://www.wvdl.wisc.edu/PDF/CWDNews3_16_04.asp

IDEXX Laboratory CWD Test Web Site
http://www.idexx.com/production/ruminant/ruminant7.cfm

Associated Press article by Robert Imrie on Wisconsin CWD testing
http://www.twincities.com/mld/pioneerpress/news/local/states/wisconsin/8155037.htm

Statement on IDEXX test results before 3/24/2004 DNR Board Meeting by Anthony Grabski, Ph.D.
http://www.caids-wi.org/ DNRBoard3-24-04.doc