Generalized linear models (GLMs) are among the most popular classes of statistical models because they include a large class of frequently used regression models such as the linear regression model and the logistic regression model. One of their key limitations is their lack of robustness against outliers. We show that, for the logistic regression, the skewed inference and prediction in case there is a conflict in the trends in the bulk of the data and in the outliers is caused by the exponentially decaying tails of the Sigmoid function. We fix the problem by replacing the Sigmoid function by a robust version of it that changes the exponential tails to slowly varying tails. To our knowledge, there is no Bayesian robust approach for GLMs in general. The most popular method is Cantoni and Ronchetti (2001), which is frequentist and is based on M-estimators. The only Bayesian robust approach for logistic regression we have found is the misclassification model proposed by Russo and Greco (2022). Compared to Cantoni and Ronchetti’s method, which does not provide a statitical model and often ignores completely the information provided by the outliers, we do provide a model useful for both frequentist and Bayesian analyses while still keeping sensibility to the positions of outliers. Compared to Russo and Greco’s model, our model is simpler, has no latent variables, and is justified in a more theoretical way. Our approach is rooted in the robust heavy-tailed modeling in Bayesian statistics (O’Hagan and Pericchi, 2012; Desgagné, 2015; Gagnon et al., 2020; Gagnon and Wang, 2024). In our context, our robust version of the Sigmoid function is the sum of a central term and slowly varying tails behaving like the function 1/ log |x|. It is the first time that such an approach appears in the context of regression for a binary response variable. The performance is examined both theoretically and empirically, with an analysis of the leukemia data set.

Generalized linear models (GLMs) form one of the most popular classes of models in statistics. This class contains a large variety of commonly used regression models, such as normal linear regression, logistic regression and gamma GLMs. In GLMs, the response variable distribution defines an exponential family. A drawback of these models is that they are non-robust against outliers. For models like the normal linear regression and gamma GLMs, the non-robustness is a consequence of the exponential tails of the densities. The difference in trends in the bulk of the data and the outliers yields skewed inference and prediction. To our knowledge, there is no Bayesian robust approach specifically for GLMs. The most popular method is frequentist; it is that of Cantoni and Ronchetti (2001). Their approach is to adapt the robust M-estimators for linear regression to the context of GLMs. However, their estimator is derived from a modification of the derivative of the log-likelihood, instead of from a modification of the likelihood (as with robust M-estimators for linear regression). As a consequence, it is not possible to establish a clear correspondence between the modified function to optimize and a model. Having a robust model has two advantages. First, it allows for an understanding and an interpretation of the modelling. Second, it allows for both frequentist and Bayesian analysis. The method we propose is based on ideas from Bayesian robust linear regression. We adapt the approach proposed by Gagnon et al. (2020), which consists of using a modified normal distribution with heavier tails for the error term. In our context, the distribution of the response variable is a modified version where the central part of the density is kept as is, while the extremities are replaced by log-Pareto tails, behaving like (1/|x|)(1/ log |x|)λ. The focus of this thesis is on gamma GLMs. The performance is measured both theoretically and empirically, with an analysis of hospital costs data.

Model selection and parameter estimation are two main aspects of statistical analysis. This thesis discusses these aspects from a Bayesian point of view via three papers. The first one deals with a computational procedure, named the reversible jump algorithm, that allows to simultaneously select models and estimate parameters. This sampler being difficult to tune in practice, we aim at providing guidelines to users for an optimal implementation. An optimally tuned sampler corresponds to a sampler that generates Markov chains that optimally explore their state space. Our goal is achieved through the optimisation of a stochastic process that corresponds to the limit (in distribution) of the sequence of stochastic processes engendered by the algorithm. In the second paper, a strategy leading to robust estimation of the parameters of a linear regression model in presence of outliers is presented. The strategy is to make assumptions that are more adapted to the eventual presence of outliers, compared with the traditional model assuming normality of errors. This normality assumption is indeed replaced by a super heavy-tailed distribution assumption. Robustness, which is represented by the convergence of the posterior distribution of the parameters (based on the whole sample) towards that arising from the nonoutliers only, is guaranteed when the number of outliers does not exceed a given threshold. Finally, the results presented in the first two papers are combined to introduce a Bayesian robust principal component regression approach that involves model selection in the prediction process. The characteristics of this approach contribute to increase the accuracy of the predictions produced.