How to solve irreproducibility problem – example set by collaboration

by Alexey Bersenev on March 18, 2014 · 0 comments

in consensus, cytometry, open science

Post to Twitter Send Gmail Post to LinkedIn

We have written a lot about importance of reproducibility. Inability to reproduce data, published by other labs is a frequent cause of scientific controversies, loss of credibility and paper retractions. Fortunately, we have some examples in the field, where controversies were tackled by collaborative efforts. Recently, I was happy to find one more example of irreproducibility solving effort.

Two US-based labs (Mina Bissell’s of Berkley U and Kornelia Polyak’s of Harvard U) were not able to reproduce each other findings, related to FACS-based identification of breast cancer stem cell markers (CD44/CD10/CD24) in human specimens. A big collaborative effort was set:

Rather than giving up or each publishing our data without the other laboratory, we decided to work together to solve these differences, even traveling from one laboratory to the other in order to perform experiments side by side on the same human breast tissue sample. This exercise confirmed our suspicions and resolved our problem.

Reproducibility collaboration turned into 2-year adventure, supported by separate grant. Labs identified that FACS-related issues were in tissue digestion methodology:

… this time-consuming and expensive exercise gave us the clue we had been waiting for: our methods for incubating the collagenase digests were distinctly different.

it was the vigorous agitation during collagenase treatment itself that lead to reduced CD44 antigen detection on the CD10+ cell population (CD44 levels were preserved on CD10− cells independently of the agitation procedure used).

A lot of lessons could be learned from this story, but the most important one – in reproducibility issues be friends, don’t fight! Contact each other, brainstorm, collaborate, set side-by-side experiments, consult, visit… It could be time consuming, but it worth trying.

The second, big lesson from this story – FACS analysis details could be very probable root case of irreproducibility (especially in stem cell research):

It is educational that CD44 staining used by countless laboratories in FACS analysis can so easily be altered by an apparently minor difference in methodology. The irreproducibility of CD44 in FACS analysis has become a legend. We expect sharing these experiences demonstrates that much can be learned through open collaboration and persistence.

I really would love to see more and more such efforts in stem cell research. Unfortunately, labs frequently fight for priority of discovery, for authority, for patents and grants. One of the current examples – VSEL controversy.

{ 0 comments… add one now }

Leave a Comment

Previous post:

Next post: