FAQ  •  Register  •  Login

Keep the same viSNE plot when clustering...

Forum rules
Please be as geeky as possible. Reference, reference, reference.
Also, please note that this is a mixed bag of math-gurus and mathematically challenged, so choose your words wisely :-)



Posts: 50

Joined: Tue May 13, 2014 8:04 pm

Post Thu May 31, 2018 7:08 pm

Re: Keep the same viSNE plot when clustering...

Hi James,

You raise a good point about total cell number. When I recommended using "all" events for the clustering, I was assuming that Gael was working with a low number of events in the tSNE plot (roughly <1e6 cells, total). Cytobank's built-in viSNE applet makes it easy to subsample and normalize the number of cells across samples, and I believe it has a hard upper limit of 4e6 cells total.

Including "all" cells has two upsides:

(1) It gives the clustering algorithm (PhenoGraph) the same information that was available to the tSNE algorithm when it was run in Cytobank, so the clusters are more likely to make sense when overlaid in tSNE space
(2) The final tSNE plot will have the same density as what you see on Cytobank, except now you can see the cluster assignments (this was Gael's goal, from what I understood)

There's no hard and fast rule about using all events. You could certainly run PhenoGraph on a subset of the cells exported from Cytobank. Just be aware that when you re-upload those files, the tSNE plots will look sparser than they did before. Also, it's a little bit of an apples / oranges comparison because you're not giving PhenoGraph the same data that was provided to the tSNE algorithm. In an extreme example, suppose you subsampled to a very low number of cells (i.e. 100). This would not be enough information for PhenoGraph to distinguish outliers from rare but robust cell populations. In that case, the clustering would look terrible when they are overlaid on the tSNE plot.

The downside of all of the workflows we have discussed so far on this thread is that they require subsampling when dealing with large numbers of samples. This is a big compromise -- on the one hand, it makes it easy to visualize the clusters in tSNE space (i.e. you can view cell-level data) and to compare across samples (i.e. cluster names are preserved). On the other hand, you lose some resolution of rare cell subsets.

A different way of using PhenoGraph is to run it on each sample separately, and then assemble metaclusters from the clusters (see Levine et al. Cell 2015). Cytofkit's FlowSOM-meta implementation also works this way (see Van Gassen et al Cytometry A 2015). With this type of workflow, you can use many more cells per sample. I've done some recent analyses with 2e5 cells per sample x 80 samples = 1.6e7 cells total. However, the downside is that can't view the cell-level data in tSNE (1.6e7 cells would take forever to produce a tSNE plot). An alternative is to view the centroids of the first-round clusters in tSNE space, but that requires some custom coding -- cytofkit isn't built for that. I still use Cytofkit / PhenoGraph /tSNE for a quick look at sets of 1-20 samples (10,000 cells per sample), but I've moved on to a custom-built metaclustering pipeline for bigger datasets of 20+ samples.




Posts: 5

Joined: Mon Apr 30, 2018 9:37 pm

Post Fri Jun 01, 2018 10:29 pm

Re: Keep the same viSNE plot when clustering...

Hi everyone,

I tried everything and still haven't progressed a bit.

i will explained what I did and maybe someone will see what I'm doing wrong.... There is probably many things missing.
1- In Cytobank, did the initial gating and exported my data as ''events''. Got the FCS files and made sure everything was ok with Flowjo. I got the tSNE parameters and could easily work with my datas.
2- Loaded the FCS files into Cytofkit, set it up, ran Phenograph and got the usual folder with the FCS files and all those CSV and PDF files.
3- Phenograph settings (see picture):
- Merge method: all
- Transformation: cytofAsinh: Maybe I should choose none?
- Cluster method: Rphenograph
- Visualization: Null
4- If I load the FCS files from cytofkit into Flowjo, the tSNE parameters are there but no core parameters: I guess I should load R into flowjo to see them?
5- If I choose the tSNE parameters, everything is diffuse and there is no way this makes sense.
6- If I load the output into the shiny app, everything looks great but it's definitively not the initial viSNE from cytobank
7- Finally, I imported the FCS files into Cytobank (I did not find any other way so I imported them as a new experiment). Since they're not considered as a viSNE analysis, even if I chose the tSNE parameters I get something similar as in Flowjo.... garbage.

So Monday the Helios will get his weekly clean and then back to this :)

Thanks everyone and have a nice weekend!
Phenograph settings.png
FCS in Flowjo.png
FCS in Flowjo.png (11.67 KiB) Viewed 2795 times



Posts: 24

Joined: Sun Oct 19, 2014 10:15 pm

Post Sat Jun 02, 2018 10:31 am

Re: Keep the same viSNE plot when clustering...

Hi Gael,

Definitely looks weird. Based on the screenshot, and reading your description, it is possible that you have transformed (using arcsinh) the tSNE parameters when they were loaded into cytofkit (which you don't want to do). This would muck up the tSNE plots. In your description, you've identified that this may be a potential cause. It is a little complicated though: I'm not sure if the data that comes out of Cytobank is already transformed (or if it is just visualised using a transformed scale in Cytobank) -- but if it is already transformed, then you don't need to do it again in cytofkit. If it isn't transformed, then you would still need to transform the data in cytofkit, but NOT transform the tSNE parameters. I'm not sure if cytofkit gives you the ability to select which parameters are transformed when they are imported (I really like cytofkit, I just don't get to use it very often, as we are using a scripted analysis pipeline). Erin, perhaps you can comment on this?

There could be some other causes -- but from first glance I think that's most likely.

If you would like, I'd be happy to chat over Skype and walk you through some of the issues, and we can figure out a workflow that best suits your objectives. FYI, I live in Australia, so we are +14 hrs ahead of Montreal (or -10 hrs if you don't count which day it is). You can shoot me an email at thomas.ashhurst@sydney.edu.au if you are keen. If we sort it out, you should write up a little summary for the forum of what the issue was, and how you got around it.




Posts: 50

Joined: Tue May 13, 2014 8:04 pm

Post Sat Jun 02, 2018 11:01 pm

Re: Keep the same viSNE plot when clustering...

Hi Gael,

Sorry it didn't work on your first try. I think you are close. I tried it myself just now and one issue I ran into was that cytofkit didn't like the choice of merging "all" events. It gave me an error:
Extract expression data...
Error in if (nrow(x) < fixedNum) { :
missing value where TRUE/FALSE needed

So the solution is to choose "ciel" and set the maximum to an absurdly high number (9999999).

I've attached a PDF with a series of screenshots showing my workflow. I used Cytofkit v1.10.0 for this (from BioConductor). I was able to get the data uploaded to Cytobank and view the PhenoGraph clusters in the original tSNE1/tSNE2 axes from my previous vISNE run. Hopefully you can follow along and get the same results. Good luck!

180601 Howto Cytobank viSNE to Cytofkit PhenoGraph to Cytobank.pdf
Tutorial - viSNE to PhenoGraph to Cytobank
(1.32 MiB) Downloaded 288 times

Return to CyTOF data analysis

Who is online

Users browsing this forum: No registered users and 6 guests