Developing novel cancer drugs or therapies requires years of preclinical work before translation to clinical trials and ultimately the market. Unfortunately, an overwhelming majority of compounds will fail to make this transition and will show no benefit in trials. To reduce attrition along the drug development pipeline, mathematical modelling is increasingly used in preclinical work to investigate and optimize treatment scenarios, in the hope of improving the success rate of potential therapies. Mechanistic models aim to incorporate the mechanisms of actions of drugs and physiological/cellular interactions to provide a deeper understanding of the system and rationally investigate therapeutic effectiveness. This thesis focuses on the implementation of heterogeneous, mechanistic mathematical models in preclinical contexts in cancer drug development. The first chapter of this thesis provides an overview of mathematical oncology and the drug discovery pipeline by presenting different tumour growth models and the integration of therapeutic effect through pharmacokinetic/pharmacodynamic (PK/PD) models. The second chapter of this thesis discusses granulosa cell tumour (GCT) of the ovary and the development of a mathematical model to investigate the potential of a combination therapy using a chemotherapy and an immunotherapy that produces tumour necrosis factor-related apoptosis-inducing ligand (TRAIL) through an oncolytic virus (OV). The model considers tumour cells throughout the phases of the cell cycle, the infection of these cancer cells by the OV, and the innate-immune pressure from the body. It also incorporates detailed PK/PD models for TRAIL and the chemotherapeutic drug, procaspase activating compound-1 (PAC-1). This includes a mechanistic receptor binding PK model for TRAIL as well as a two-compartment PK model for PAC-1 to properly integrate the concentrations of both compounds in the combination effect function applied to the cancer cell populations. Through simulations and hypothesis testing, we determined the minimal doses and ideal dosing regimens for PAC-1 that best controlled tumour growth. We also established how to successfully eradicate the tumour under the assumption of a much higher infection rate of the OV. 6 In the third chapter, we present different approaches to include inter-individual variability into mechanistic mathematical models, each with their own benefits and challenges. We describe how population PKs (PopPK) inform on cohort averages and variability due to covariates, and how to use this heterogeneity to recover the dynamics of drug treatment in patient populations. Variability in cohorts can also be generated through algorithms ensuring that virtual patients have realistic parameters and outcomes. We also touch upon in silico trials that help to predict a range of outcomes and treatment scenarios. These in silico clinical trials are highly valuable in quantitative system pharmacology (QSP) due to their predictive nature. Lastly, we present an application of PopPK using 300 generated patients in a QSP model for mammary stem cell differentiation under treatment with estrogen (estradiol). We investigate the effect of hormone therapy on mammary cell differentiation due to its potential application in triple negative breast cancer (TNBC), as prolactin has been proposed in experimental models to induce differentiation in TNBC stem cells. Our model and results serve as proof of concept for the continued investigation into pharmacological means of inducing stem cell differentiation to reduce cancer plasticity and severity.

The signaling system for the transcription factor NF-κB is involved in over 150 genes in a mammal cell. This leads scientists to try to analyse this molecule to understand its effect on a cell. Many scientists, including Krishna and al., noticed oscillations in the amount of nucleic NF-κB. Before anyone noticed those oscillations, the quantities were thought to be somewhat stable, and they are, but not in every condition. This change of condition creates this instability and the transition of such stability for the stationary solution is caused by a Hopf bifurcation. To determine the existence of the stationary state in the tridimensional system and to analyse the bifurcation is important to predict the oscillations that might appear in certain conditions. It is then necessary to determine what kind of cycle appears or disappears at the bifurcation to understand the stability of those periodic solutions, of those oscillations. Finally, we simulate numercially the bifurcation diagrams for two models and differents parameters to observe the local similarities and global divergence of the diagrams.