Note
Click here to download the full example code
Fit models and plot gene trends
This example shows how to prepare, fit and plot various models to gene expression trends.
We will focus mostly on Generalized Additive Models (GAMs)
and show how to do this for sklearn estimators towards the end. GAMs are flexible models that are well suited to
model non-linear gene trends as they often appear in single-cell data. Further, they have the advantage that it is
relatively straightforward to derive a confidence interval around the main trend.
import cellrank as cr
from sklearn.svm import SVR
adata = cr.datasets.pancreas_preprocessed("../example.h5ad")
adata
Out:
AnnData object with n_obs × n_vars = 2531 × 2000
obs: 'day', 'proliferation', 'G2M_score', 'S_score', 'phase', 'clusters_coarse', 'clusters', 'clusters_fine', 'louvain_Alpha', 'louvain_Beta', 'initial_size_unspliced', 'initial_size_spliced', 'initial_size', 'n_counts', 'velocity_self_transition', 'dpt_pseudotime'
var: 'highly_variable_genes', 'gene_count_corr', 'means', 'dispersions', 'dispersions_norm', 'fit_r2', 'fit_alpha', 'fit_beta', 'fit_gamma', 'fit_t_', 'fit_scaling', 'fit_std_u', 'fit_std_s', 'fit_likelihood', 'fit_u0', 'fit_s0', 'fit_pval_steady', 'fit_steady_u', 'fit_steady_s', 'fit_variance', 'fit_alignment_scaling', 'velocity_genes'
uns: 'clusters_colors', 'clusters_fine_colors', 'diffmap_evals', 'iroot', 'louvain_Alpha_colors', 'louvain_Beta_colors', 'neighbors', 'pca', 'recover_dynamics', 'velocity_graph', 'velocity_graph_neg', 'velocity_params'
obsm: 'X_diffmap', 'X_pca', 'X_umap', 'velocity_umap'
varm: 'PCs', 'loss'
layers: 'Ms', 'Mu', 'fit_t', 'fit_tau', 'fit_tau_', 'spliced', 'unspliced', 'velocity', 'velocity_u'
obsp: 'connectivities', 'distances'
First, we need to compute a set of terminal states and we have to estimate fate probabilities towards these. The fate probabilities will be used as weights when fitting the model - they determine how much each cell contributes to each lineage.
cr.tl.terminal_states(
adata,
cluster_key="clusters",
weight_connectivities=0.2,
softmax_scale=4,
show_progress_bar=False,
)
cr.tl.lineages(adata)
Out:
/home/docs/checkouts/readthedocs.org/user_builds/cellrank/checkouts/latest/examples/other/plot_model.py:24: DeprecationWarning: `cellrank.tl.terminal_states` will be removed in version `2.0`. Please use the `cellrank.kernels` or `cellrank.estimators` interface instead.
cr.tl.terminal_states(
/home/docs/checkouts/readthedocs.org/user_builds/cellrank/checkouts/latest/cellrank/tl/_init_term_states.py:156: DeprecationWarning: `cellrank.tl.transition_matrix` will be removed in version `2.0`. Please use the `cellrank.kernels` or `cellrank.estimators` interface instead.
kernel = transition_matrix(
/home/docs/checkouts/readthedocs.org/user_builds/cellrank/checkouts/latest/examples/other/plot_model.py:31: DeprecationWarning: `cellrank.tl.lineages` will be removed in version `2.0`. Please use the `cellrank.kernels` or `cellrank.estimators` interface instead.
cr.tl.lineages(adata)
0%| | 0/3 [00:00<?, ?/s]
100%|##########| 3/3 [00:00<00:00, 29.42/s]
Models in cellrank.ul.models follow similar patterns as sklearn models.
We begin by initializing and preparing the model for fitting. cellrank.ul.models.BaseModel.prepare() requires
only the gene name and the lineage name and must be called before cellrank.ul.models.BaseModel.fit(). It also
includes various useful parameters, such as time_range or weight_threshold, which determine the start and
end pseudotime and the minimum required threshold for lineage probabilities, respectively.
model = cr.ul.models.GAM(adata)
model.prepare(
gene="Pak3",
lineage="Alpha",
time_key="dpt_pseudotime",
data_key="Ms",
n_test_points=100,
)
Out:
<GAM[gene='Pak3', lineage='Alpha', model=GammaGAM(callbacks=['deviance', 'diffs'], fit_intercept=True, max_iter=2000, scale=None, terms=s(0), tol=0.0001, verbose=False)]>
CellRank allows any pseudotime from adata.obs to be passed via the time_key, e.g. Diffusion Pseudotime (DPT)
[Haghverdi et al., 2016], scvelo’s latent time [Bergen et al., 2020], Palantir’s pseudotime [Setty et al., 2019], etc.
Further, CellRank accepts imputed gene expression values stored in adata.layers via the data_key, i.e. you
can pass MAGIC [van Dijk et al., 2018] imputed data, scvelo.pp.moments() [Bergen et al., 2020] (used below) or any other
form of imputation.
Once the model has been prepared, it is ready for fitting and prediction.
y_hat = model.fit().predict()
y_hat
Out:
/home/docs/checkouts/readthedocs.org/user_builds/cellrank/envs/latest/lib/python3.8/site-packages/pygam/utils.py:649: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
bases = (x >= aug_knots[:-1]).astype(np.int) * \
/home/docs/checkouts/readthedocs.org/user_builds/cellrank/envs/latest/lib/python3.8/site-packages/pygam/utils.py:650: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
(x < aug_knots[1:]).astype(np.int)
array([0.04188791, 0.04126428, 0.04068071, 0.04013712, 0.03963346,
0.03916972, 0.03874596, 0.03836229, 0.0380189 , 0.03771603,
0.03745405, 0.03723337, 0.03705455, 0.03691824, 0.03682523,
0.03677643, 0.03677291, 0.0368159 , 0.03690682, 0.03704727,
0.03723907, 0.0374843 , 0.03778525, 0.03814454, 0.0385651 ,
0.03905017, 0.03960342, 0.04022891, 0.0409312 , 0.04171537,
0.04258705, 0.04355257, 0.04461895, 0.04579404, 0.04708596,
0.04850117, 0.05004616, 0.05172811, 0.05355493, 0.05553525,
0.05767849, 0.05999494, 0.06249573, 0.06519297, 0.06809971,
0.07123006, 0.0745993 , 0.07822377, 0.08212117, 0.08631043,
0.0908119 , 0.09564741, 0.10084029, 0.10641551, 0.1123997 ,
0.11882138, 0.1257108 , 0.1331002 , 0.14102387, 0.14951801,
0.15862122, 0.1683741 , 0.17881963, 0.1900029 , 0.20197162,
0.21477558, 0.22846699, 0.24309957, 0.25872678, 0.27540228,
0.29318067, 0.31211704, 0.33226678, 0.35368466, 0.3764248 ,
0.4005398 , 0.42607975, 0.45309275, 0.48162258, 0.5117088 ,
0.5433851 , 0.5766794 , 0.61161155, 0.6481926 , 0.6864236 ,
0.7262952 , 0.7677848 , 0.81085634, 0.855459 , 0.90152526,
0.9489712 , 0.9976939 , 1.0475703 , 1.098458 , 1.1501939 ,
1.2025928 , 1.2554483 , 1.308532 , 1.3615955 , 1.414369 ],
dtype=float32)
Optionally, we can also get the confidence interval. Models which don’t have a method to compute it, such as
cellrank.ul.models.SKLearnModel wrapper for some sklearn estimators, can use the default the
confidence interval cellrank.ul.models.BaseModel.default_confidence_interval().
conf_int = model.confidence_interval()
conf_int[:5]
Out:
/home/docs/checkouts/readthedocs.org/user_builds/cellrank/envs/latest/lib/python3.8/site-packages/pygam/utils.py:649: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
bases = (x >= aug_knots[:-1]).astype(np.int) * \
/home/docs/checkouts/readthedocs.org/user_builds/cellrank/envs/latest/lib/python3.8/site-packages/pygam/utils.py:650: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
(x < aug_knots[1:]).astype(np.int)
array([[0.0387563 , 0.04527255],
[0.03821013, 0.04456254],
[0.0377004 , 0.04389662],
[0.037227 , 0.04327473],
[0.03678989, 0.04269681]], dtype=float32)
After the prediction and optionally the confidence interval calculation, we can plot the smoothed gene expression.
Cells in this plot have been colored by their fate probability of reaching the Alpha terminal state. We include these probabilities as weights in the loss function when fitting the model. This allows us to weight each cell by its relative contribution to the lineage, without needing to subset cells for each lineage.
model.plot(conf_int=True)
Lastly, wrapping sklearn estimators is fairly simple, we just pass the instance
to cellrank.ul.models.SKLearnModel.
svr = SVR()
model = cr.ul.models.SKLearnModel(adata, model=svr)
model
Out:
<SKLearnModel[gene=None, lineage=None, model=SVR()]>
Total running time of the script: ( 0 minutes 23.812 seconds)
Estimated memory usage: 574 MB