Boosting nonlinear penalized least squares

For some reasons I couldn’t foresee, there’s been no blog post here on november 13 and november 20. So, here is the post about LSBoost announced here a few weeks ago.

First things first, what is LSBoost? Gradient boosted nonlinear penalized least squares. More precisely in LSBoost, the ensembles’ base learners are penalized, randomized neural networks.

These previous posts, with several Python and R examples, constitute a good introduction to LSBoost:

• https://thierrymoudiki.github.io/blog/2020/07/24/python/r/lsboost/explainableml/mlsauce/xai-boosting

• https://thierrymoudiki.github.io/blog/2020/07/31/python/r/lsboost/explainableml/mlsauce/xai-boosting-2

More recently, I’ve also written a more formal, short introduction to LSBoost:

• LSBoost, gradient boosted penalized nonlinear least squares (pdf).

The paper’s code – and more insights on LSBoost – can be found in the following Jupyter notebook:

• https://github.com/Techtonique/mlsauce/blob/master/mlsauce/demo/thierrymoudiki_211120_lsboost_sensi_to_hyperparams.ipynb

Comments, suggestions are welcome as usual.