[gdpawel]

About a decade ago, scientists figured out how to transform genetic instructions into an electronic format. Gene profiling using a "microarray" - a chip of glass arrayed with thousands of gene fragments - was expected to revolutionize medicine by decoding the basis of disease.

"All human illness can be studied by microarray analysis, and the ultimate goal of this work is to develop effective treatments or cures for every human disease by 2050," wrote Mark Schena, an inventor of the technology.

But skepticism had set in. In an article in the Lancet, researchers reanalyzed the seven largest microarray studies on cancer prognosis. In five of the seven, this technology performed no better than flipping a coin. The two other studies barely beat horoscopes, according to John P. Ioannidis, a clinical epidemiologist with Tufts University School of Medicine, who wrote in an accompanying editorial.

To understand why, consider the fable about six blind men and an elephant. Each man feels a different part of the animal. One man argues that the creature is a snake, another a spear, another a wall, and so on. A little girl who can see the elephant says, "Each of you is right, but you are all wrong."

Depending on how researchers "feel" their molecular data - using computer analysis to massage, stroke and ignore certain parts - they may discover right answers that are all wrong.

David Ransohoff, a University of North Carolina epidemiologist, says results cannot be trusted unless they can be produced again and again: "Figuring out whether a result is real and not simply caused by chance is determined in part by validation - by reproducing the result in an independent set of samples." In other words, go feel another elephant.

But even that is not enough, Ransohoff and other experts say. The ultimate validation requires clinical studies in actual patients. A molecular diagnostic method must be as reliable as traditional tools such as imaging tests and surgical biopsy.

This analysis is tremendously manipulative, indirect, and often ambiguous. One problem is that trace proteins - the potential biomarkers - may be swamped by other proteins, despite techniques to concentrate the scarcest ones on the special chip that goes into the mass spectrometer.

Another problem is that the spectrometer's measurements - made after vaporizing the proteins and giving them a positive charge - are least reliable in the low range where biomarkers are presumed to exist.

Finally, the spectrometry results can be thrown off by countless variables, including machine miscalibration and handling of blood samples. All of which makes results difficult to reproduce, even in the same lab using the same blood samples.

Source: Lancet, February 7, 2002