A potential Alzheimer’s test

Last week there was media abuzz about a potential Alzheimer’s test. The work was led by Howard Federoff, a professor of neurology and executive vice president for health sciences at Georgetown University Medical Center and published in Nature Medicine.

I emailed the author over a week ago and requested a copy of the paper. I have not received a response. My complaint is once again if the authors are willing to give extensive media interviews then the paper should be readily available and not hidden behind a pay wall. The journal is charging $32 for a pdf version of the paper. I understand that journals need to make money. When the audience is the author’s peers, who are likely to subscribe to the journal or work at an institution that subscribes to the journal, then charging for copies makes sense. When the author chooses to make his audience the general public then there is an element of self promotion for both the author and the author’s institutions. In this case Georgetown University was more than happy to publicize the findings on their web site.

After watching the videos and reading a few of the reports that gave at least some description of the research I came away with a number of questions. Those questions may well be answered in the paper. However as I mentioned the author has not responded to my request for a copy of the paper. So I’ll ask my questions here.

For the research 525 people age 70 and over were recruited. Two to three years later an subset those people had developed Alzheimer’s. What the authors tell us is that 53 patients who developed Alzheimer’s were compared to 53 who had not developed the disease.

So question 1 is how the two sets of 53 patients were chosen? A related question is why limit the number without the disease to just 53. That seems to reduce the effectiveness of any analysis. There is nothing in the statistics that demands that the two groups be the same size.

They next searched through 4,000 biomakers and found ten that they linked to the disease. This appears to me to be extreme data mining. Going through 4,000 variables based on 53 patients it seems almost certain that several sets of ten biomarkers could be linked to the disease. Many of these would likely be just due to random chance. So my second question is how the researchers selected the final set of ten biomarkers that they are proposing to use for the test.

Part of the answer to this may be in the validation sample. The description of the validation sample leaves much to be desired. The sample size is only 40 participants. To be a proper validation sample those people must be selected from the original population without knowledge of either the test results or the occurrence of Alzheimer’s. With only 40 cases it would seem that the number of cases of Alzheimer’s that would occur would be much too small for that to serve as a useful validation sample.

We are also told that the test allows prediction with a 90 percent accuracy if a healthy person will develop Alzheimer’s. My next question is what this 90 percent means. My guess, and that is what it is, a guess, is that of 100 patients who show symptoms of the disease within three years will have tested positive. If that is the case what is the false positive rate. In other words of those with a positive test result what percent of those will develop the disease. This is different than asking what percent of those who will develop the disease test positive. This is important because if we are to start treatment for those who test positive we will be treating both those who will develop the disease and those who will not develop the disease. Both ethical and cost considerations come into play in implementing any treatment plan.

The authors offer the possibility of selecting subjects based on their test that then would become subject to drug tests to see if a treatment could prevent or delay the onset of Alzheimer’s. This is the second point at with the false positives become an important factor. In evaluating the effectiveness of the test consideration must be given to the fact that a certain number of subjects will not develop Alzheimer’s because they original screening test identified some subject who never would have developed the disease.

Keep in mind that developing a test that allows for treating the disease prior to the actual onset of the disease is a dream. It is something that works well with vaccines. That is why we take medicines to reduce blood pressure to prevent heart disease and other illnesses. Can it work with Alzheimer’s? That is the unknown.

Questions, questions, questions – if only the paper were readily available.