mleipold wrote:El-ad: those parameters are present in Helios data. Considering that the Helios instruments have been out for more than 2 years (and a number of CyTOF2 instruments have undergone the Helios upgrade), a large number of people should have data that could be looked at.
I've been working with Mount Sinai and as far as I know they don't have access to this data. Also, judging from public data sets, I couldn't find these channels. I might be doing something wrong though, please correct me if there are any public data sets that include these.
mleipold wrote:If you could give Bruce a wishlist for benchmarking, what would it be? ... Probably the main point from the Fluidigm UGM talk was basically being able to gate your LiveIntact Singlets away from Debris, Doublets, Dead Cells, and Beads in one go.
I would want to see an experiment where doublets can be easily identified through an orthogonal mean. That orthogonal mean would be the "gold standard" we are trying to identify. Two possible examples:
- Take one sample, split it to ten subsamples, barcode each, combine them, and acquire them using a mass cytometer. Cells which have two or more barcodes are doublets.
- Take one sample, split it to ten subsamples. In each subsample, measure CD45 using a different mass. Cells which have two or more CD45 channels are doublets.
Then, we could take several doublet identification methods, such as Bagwell's method, the standard event length versus DNA, backgating, clustering, what-have-you. Compare methods using precision/recall, F-score, AUC, or other metric of choice.
mleipold wrote:And what sort of biological results are you meaning?
Let's say I currently gate doublets using DNA versus event length. I'm really good at it, or I have a trained technician who does it, or my lab's expertise revolves around it. It removes most doublets and probably a small number of legit cells.
Now a magical method comes out that removes *all* doublets and *only* doublets from a sample (I'm exaggerating here for the argument's sake).
Why should I switch to the new magical method? Why should I invest time and effort in integrating it into my pipeline?
In my opinion, one way to show the utility of the new method is to provide an experiment where utilizing it allows the identification of biological insights that the current method failed to identify. A lower bar (which I would also accept) is to show some statistical relevance of magical method over existing method -- even something as simple as lower p values on a set of statistical tests.
Another option is to show that magical method is cheaper, faster, easier, or otherwise superior to the existing methods. If there was an easy way to run this doublet detection, great!
There are a lot of mass cytometry methods coming out right now. From my point of view, Bagwell's method warrants no special attention until there is some evidence that it provides a benefit over existing methods. I admit that the current methods are not very good, but still, there should be *something* coming from Bagwell to show that this warrants further attention.