Skip to main content

Population genomic simulations with slendr

Why use simulations?

  1. Developing intuition into statistics
  2. Estimating model parameters
  3. Ground truth for method development

Again, these are relevant for every researcher in population genomics.

. . .


We just haven’t had the appropriate software tools.



. . .

(Not entirely accurate.)

Simulation software

The most famous and widely used are SLiM and msprime.

They are very powerful and (nearly) infinitely flexible.

However, they both require:

  • quite a bit of code for complex simulations
  • relatively high confidence in programming

Our exercises will focus on slendr—a simulation toolkit for population genomic simulations in R which I created.


But, for completeness, let’s look at msprime and SLiM a bit.

What is msprime?

  • A Python module for writing coalescent simulations

  • Extremely fast (genome-scale, population-scale data!)

  • You should know Python fairly well to build complex models

Image modified from Alexei Drummond

Simple simulation using msprime

import msprime

demography = msprime.Demography()
demography.add_population(name="A", initial_size=10_000)
demography.add_population(name="B", initial_size=5_000)
demography.add_population(name="C", initial_size=1_000)
demography.add_population_split(time=1000, derived=["A", "B"], ancestral="C")

ts = msprime.sim_ancestry(
  sequence_length=10e6,
  recombination_rate=1e-8,
  samples={"A": 100, "B": 100},
  demography=demography
)

What is SLiM?

  • A forward-time simulator

  • Has its own programming language

  • Massive library of functions for:

    • demographic events
    • various mating systems
    • natural selection
  • Nearly 1000 pages long manual!

Image modified from Alexei Drummond

Simple neutral simulation in SLiM

initialize() {
    // create a neutral mutation type
    initializeMutationType("m1", 0.5, "f", 0.0);

    // initialize 1Mb segment
    initializeGenomicElementType("g1", m1, 1.0);
    initializeGenomicElement(g1, 0, 999999);

    // set mutation rate and recombination rate of the segment
    initializeMutationRate(1e-8);
    initializeRecombinationRate(1e-8);
}

// create an ancestral population p1 of 10000 diploid individuals
1 early() { sim.addSubpop("p1", 10000); }

// in generation 1000, create two daughter populations p2 and p3
1000 early() {
    sim.addSubpopSplit("p2", 5000, p1);
    sim.addSubpopSplit("p3", 1000, p1);
}

// in generation 10000, stop the simulation and save 100 individuals
// from p2 and p3 to a VCF file
10000 late() {
    p2_subset = sample(p2.individuals, 100);
    p3_subset = sample(p3.individuals, 100);
    c(p2_subset, p3_subset).genomes.outputVCF("/tmp/slim_output.vcf.gz");

    sim.simulationFinished();
}

SLiM and msprime are both
incredible pieces of software…

… so why slendr?

https://bodkan.net/slendr


First motivation: spatial simulations!

A broader motivation for slendr

  • Most researchers are not expert programmers

  • All but the most trivial simulations require lots of code

. . .

  • Yet, 90% [citation needed] of models are basically the same!

    • define populations, admixture events, bottlenecks, etc.

. . .

  • Computing statistics is even more challenging

. . .

  • This is a huge barrier to simulation-based research / learning

. . .

slendr is designed to make this very easy

A quick overview of slendr


(The exercise worksheets have a lot more background information, so don’t worry about us rushing through this.)

Typical slendr workflow in R

We will always start our R scripts with one of these:

library(slendr)
init_env()

    or

library(slendr)
init_env(uv = TRUE)

. . .


Followed by some combination of:

  1. creating populations
  2. programming \(N_e\) size changes
  3. encoding gene-flow events
  4. simulating genomic data
  5. computing statistics

Creating populations

At minimum, we need its name, size and “time of appearance”:

pop1 <- population("pop1", N = 1000, time = 1)

This creates a normal R object! Typing it out gives a summary:

pop1
slendr 'population' object 
-------------------------- 
name: pop1 
non-spatial population
stays until the end of the simulation

population history overview:
  - time 1: created as an ancestral population (N = 1000)

Note: Because slendr uses either msprime or SLiM internally for simulation of genomic data, all individuals are assumed to be diploid.

Programming population splits

Splits are defined by providing a parent = <pop> argument:

pop2 <- population("pop2", N = 100, time = 50, parent = pop1)

Note: Here pop1 is an R object created above, not a string "pop1"!

Scheduling resize events

  • Step size decrease:
pop1 <- population("pop1", N = 1000, time = 1)
pop1_step <- resize(pop1, N = 100, time = 500)
  • Exponential increase:
pop2 <- population("pop2", N = 100, time = 50, parent = pop1)
pop2_exp <- resize(pop2, N = 10000, time = 500, end = 2000)

Tidyverse-style pipe %>% interface

The following leads to a more concise (and “elegant”) code.

  • Step size decrease:
pop1 <-
  population("pop1", N = 1000, time = 1) %>%
  resize(N = 100, time = 500)
  • Exponential increase:
pop2 <-
  population("pop2", N = 1000, time = 1) %>%
  resize(N = 10000, time = 500, end = 2000)

Note: You can read (and understand) a() %>% b() %>% c() as “take the result of the function a, pipe it into function b, and then pipe that to function c”.

A more complex model

Using just the two functions introduced so far:

pop1 <- population("pop1", N = 1000, time = 1)

pop2 <-
  population("pop2", N = 1000, time = 300, parent = pop1) %>%
  resize(N = 100, time = 1000)

pop3 <-
  population("pop3", N = 1000, time = 400, parent = pop2) %>%
  resize(N = 2500, time = 800)

pop4 <-
  population("pop4", N = 1500, time = 500, parent = pop3) %>%
  resize(N = 700, time = 1200, end = 2000)

pop5 <-
  population("pop5", N = 100, time = 600, parent = pop4) %>%
  resize(N = 50, time = 900) %>%
  resize(N = 1000, time = 1600, end = 2200)

Again, each object carries its history!

For instance, this is the summary you will get from the last population from the previous code chunk:

pop5
slendr 'population' object 
-------------------------- 
name: pop5 
non-spatial population
stays until the end of the simulation

population history overview:
  - time 600: split from pop4 (N = 100)
  - time 900: resize from 100 to 50 individuals
  - time 1600-2200: exponential resize from 50 to 1000 individuals

This way, you can build up complex models step by step, checking things as you go by interacting with the R console.

Gene flow / introgression / admixture

We can schedule gene flow from pop1 into pop2 with:

gf <- gene_flow(from = pop1, to = pop2, start = 2000, end = 2200, proportion = 0.13)

Note: Here proportion = 0.13 means 13% migrants over the given time window will come from “pop1” into “pop2”.


Multiple gene-flow events can be gathered in a list:

gf <- list(
  gene_flow(from = pop1, to = pop2, start = 500, end = 600, proportion = 0.13),
  gene_flow(from = ..., to = ..., start = ..., end = ..., proportion = ...),
  ...
)

Model compilation


This is the final step before we can simulate data.


model <- compile_model(
  populations = list(pop1, pop2, pop3, pop4, pop5),
  generation_time = 1,       # (converts the all times into generations)
  simulation_length = 3000,  # (number of generations to run the simulation for)
  direction = "forward"      # (not always necessary but good practice)
)


compile_model() takes a list of components, performs some consistency checks, and returns a single R object

Model compilation


This is the final step before we can simulate data.


model <- compile_model(
  populations = list(pop1, pop2, pop3, pop4, pop5),
  gene_flow = gf,      # <----- in case our model includes gene flow(s)
  generation_time = 1,
  simulation_length = 3000,
  direction = "forward"
)


Gene flow(s) can be included via the gene_flow argument.

Model summary

Typing the compiled model into R prints a brief summary:

model
slendr 'model' object 
--------------------- 
populations: pop1, pop2, pop3, pop4, pop5 
geneflow events: 1 
generation time: 1 
time direction: forward 
time units: generations 
total running length: 3000 time units
model type: non-spatial

configuration files in: /private/var/folders/7f/n68ck8q969z_d628zqqnl4ch0000gn/T/RtmpqeOvR6/filef59a26e19eec_slendr_model 

This can be useful as a quick overview of the model we are working with. However, a better way to check a model is…

Model visualization

plot_model(model)

A note on units of time (and its direction)


Sometimes you want to work with time units such as “years ago” (aDNA radio-carbon dated samples, etc.), but you have to convert those to “generations forward” for some software.

slendr helps by making it possible to use whatever time units or time directions are more natural for a particular project.

“Forward time units”

pop1 <- population("pop1", N = 1000, time = 1)

pop2 <-
  population("pop2", N = 1000, time = 300, parent = pop1) %>%
  resize(N = 100, time = 1000)

pop3 <-
  population("pop3", N = 1000, time = 400, parent = pop2) %>%
  resize(N = 2500, time = 800)

pop4 <-
  population("pop4", N = 1500, time = 500, parent = pop3) %>%
  resize(N = 700, time = 1200, end = 2000)

pop5 <-
  population("pop5", N = 100, time = 600, parent = pop4) %>%
  resize(N = 50, time = 900) %>%
  resize(N = 1000, time = 1600, end = 2200)

model <- compile_model(
  populations = list(pop1, pop2, pop3, pop4, pop5),
  generation_time = 1,
  simulation_length = 3000, # forward-time models need an explicit end
  direction = "forward"
)

We started with pop1 in generation 1, with later events at an increasing time value.

“Forward time units”

plot_model(model) # see time progressing from generation 1 forwards

We started with pop1 in generation 1, with later events at an increasing time value.

“Backward time units”

plot_model(model) # see time progressing from "year" 30000 backwards

Same model, except now in units of “years ago” (using generation_time = 10).

So we built a model…

… how do we simulate data from it?

Built-in simulation “engines”

slendr has two simulation “engine scripts” built-in:

They are designed to “understand” slendr models, meaning that you can simulate data just with this command:

ts <- msprime(model, sequence_length = 10e6, recombination_rate = 1e-8)


No need to write any msprime or SLiM code!

The result of a simulation is a tree sequence (ts)

What is a tree sequence?

  • a record of full genetic ancestry of a set of samples
  • an encoding of DNA sequence carried by those samples
  • an efficient analysis framework

For now, let’s treat it as a “genotype black box”

. . .

How do we work with this ts thing?

slendr’s R interface to tskit statistics

Allele-frequecy spectrum, \(\pi\), \(f\)-statistics, \(F_{ST}\), Tajima’s D, etc.

Find help at this link or in R under ?ts_fst etc.

Extracting names of recorded samples

  1. We can get individuals recorded in ts with ts_recorded():
ts_recorded(ts) %>% head(1) # returns a data frame (one row here, for brevity)
    name time  pop
1 pop1_1 3001 pop1
  1. A shortcut ts_names() can also be useful:
ts_names(ts) %>% head(5) # returns a vector of individuals' names
[1] "pop1_1" "pop1_2" "pop1_3" "pop1_4" "pop1_5"
  1. We can get a per-population list of individuals like this:
ts_names(ts, split = TRUE) # returns a named list of such vectors
$pop1
[1] "pop1_678" "pop1_899" "pop1_77"  "pop1_435" "pop1_407"

All slendr statistics take individuals’ names as their function arguments.


This is modelled after the sample_sets= argument of the respective tskit Python methods (except you use names of individuals directly, not tree-sequence node numbers).

Let’s take a look at how it works in general…

tskit computation – option #1

For functions computed within groups of individuals, we can first get a vector of names to compute on like this:

# a random selection of names of three individuals in a tree sequence
samples <- c("popX_1", "popX_2", "popY_42")


Then we can calculate the statistic of interest like this:

# this computes nucleotide diversity in our set of individuals
df_result <- ts_diversity(ts, sample_sets = list(samples))

Note: Wherever you see list(<vector of names>), you can think of it as “compute a statistic for the entire group of individuals” (you get a single number). Without the list(), it would mean “compute the statistic for each individual separately” (and get a value for each of them individually).

tskit computation – option #2

For functions based on comparisons of multiple groups of individuals, we want a list of vectors of names (one per group):

# when we compute on multiple groups, it's a good idea to name them
samples <- list(
  popX = c("popX_1", "popX_2", "popX_3"),
  popY = c("popY_1", "popY_2", "popY_3"),
  popZ = c("popZ_1", "popZ_2")
)


Then we use this list of vectors in the same way as before:

# this computes a pairwise divergence between all three groups
df_result <- ts_divergence(ts, sample_sets = samples)

tskit computation – option #3

For something like \(f\) statistics, the function arguments must be more precisely specified (here A, B, C, not sample_sets):

df_result <- ts_f3(
  ts,
  A = c("popX_1", "popX_2", "popX_3"),
  B = c("popY_1", "popY_2", "popY_3"),
  C = c("popZ_1", "popZ_2")
)

Doing this manually can be annoying — ts_names() helps by preparing the list of names in the correct format:

# get names of individuals in each population as a named list of vectors
samples <- ts_names(ts, split = TRUE)

# use this list directly by specifying which vectors to take out
ts_f3(ts, A = samples$popX, B = samples$popY, C = samples$popZ)

It’s time for the first exercises!


(There’s lots more to learn about the inner workings of tree sequences, but let’s leave technicalities on the side for now.)

Plan for the rest of this session

  1. Go to bodkan.net/simgen.
  • Start with the chapter “Building slendr models”.

  • Then continue with the chapter “Simulating genomic data”.

  • An interactive slendr showcase (challenge comes later).

. . .

  1. We’ll check-in before delving into statistics in later sessions.

. . .

Reference information (slides, etc.) are under “Resources”.

. . .


Don’t rush. Don’t panic. Restart R often.

 

Let’s pick up
tree sequences
where we left off

What is a tree sequence?

  • a record of full genetic ancestry of a set of samples
  • an encoding of DNA sequence carried by those samples
  • an efficient analysis framework

Why tree sequence?


Why not VCF or a normal genotype table?

What we usually have

What we usually want

An understanding of our samples’ evolutionary history:

This is exactly what a tree sequence is!

Image from the tskit documentation

The magic of tree sequences

They allow us to compute statistics without genotypes!

There is a “duality” between mutations and branch lengths.

Note: See an amazing paper by Ralph et al. (2020) for more detail.

What if we need mutations though?

 

What if we need mutations though?

Coalescent and mutation processes can be decoupled!

. . .

This means we can overlay mutations across our tree sequence after the simulation is over.

In practice, we just go from this…


In our script we’ll have something like this:

library(slendr)
init_env()

# <... population() definitions ...>

# <... gene_flow() definition ...>

# <... compile_model(...) ...>
  
ts <-
  msprime(model, sequence_length = 50e6, recombination_rate = 1e-8)

… and overlay mutations on ts


In our script we’ll have something like this:

library(slendr)
init_env()

# <... population() definitions ...>

# <... gene_flow() definition ...>

# <... compile_model(...) ...>
  
ts <-
  msprime(model, sequence_length = 50e6, recombination_rate = 1e-8) %>%
  ts_mutate(mutation_rate = 1e-8)
Note: In most situations, mutations aren’t actually necessary. But in our exercises, let’s always add them because they don’t cost us much.

More details on
tree-sequences (ARGs)

Why are tree sequences so efficient?


This tiny script simulates 2 \(\times\) 10000 chromosomes of 100 Mb:


library(slendr)
init_env()

pop <- population("pop", time = 100000, N = 10000)
model <- compile_model(pop, generation_time = 30, direction = "backward")
ts <- msprime(model, sequence_length = 100e6, recombination_rate = 1e-8)


Runs in less than 30 seconds on my laptop!

Takes only about 66 Mb of memory!

How is this possible?!

. . .

Representation of tree sequences

A single tree can be represented by:

  • a table of nodes,

  • a table of edges between nodes,

  • a table of mutations on edges


. . .

Tree sequence is just a collection of tables!

Tree-sequence tables in practice

Nodes:

  node     time
1  360 871895.1
2  256 475982.3
3  255 471179.5

Edges:

  child parent
1   256    360
2   255    256
3    69    256

Mutations:

   id node     time
1  69   74 125539.4
2 272   22 242337.9
3 277   22 129474.1

Other data formats


Tree sequence is a useful, cutting-edge data structure, but there are many well-established bioinformatics tools out there.


slendr offers functions to help integrate simulation results with third-party software (IBDseq, PCA, PLINK, ADMIXTOOLS, etc.).

Standard genotype formats

If a tree sequence doesn’t cut it, you can always…

  • export genotypes to a VCF file:
ts_vcf(ts, path = "path/to/a/file.vcf.gz")
  • export genotypes in the EIGENSTRAT format:
ts_eigenstrat(ts, prefix = "path/to/eigenstrat/prefix")
  • access genotypes as a data frame:
ts_genotypes(ts)
    pos pop_1_chr1 pop_1_chr2 pop_2_chr1 pop_2_chr2 pop_3_chr1 pop_3_chr2
1 12977          0          1          0          0          0          0
2 15467          1          0          1          1          1          1