Foundation models
Modelling of single-cells to perform multiple tasks.
Recent developments in deep-learning have led to the creation of several “foundation models” for single-cell data. These are large models that have been trained on data from millions of cells and am to fully capture the variability in the single-cell landscape. Typically, they use a transformer architecture (Szałata et al. 2024) and undergo self-supervised pre-training using masking of parts of the input data. Trained foundation models can then be applied to a variety of downstream tasks, either by directly feeding new data into the model or by fine-tuning to better fit a new dataset or to produce a specific output. The general nature of single-cell foundation models and the large amount of data they have been trained on makes them potentially powerful tools for single-cell analysis but their performance is yet to be fully established.
Open Problems plans to build on the existing evaluation (Boiarsky et al. 2023; Liu et al. 2024) of foundation models by incorportating them into our continuous benchmarking framework. Models will be included in existing tasks where they will be compared to more traditional methods. We also plan to compare foundation models across tasks to evaluate their flexibility and compare them to each other.
If you are interested in evaluating foundation models for single-cell data please fill in the form below to get in touch.