Search results for “IRLS”

Nonlinear regression: the Newton–Raphson and Gauss–Newton methods as a series of linear regressions; logistic regression and iteratively reweighted least squares (IRLS); the generalized additive model and backfitting; the probabilistic meaning of least squares; the generalized linear model (GLM), the exponential family of distributions and the link function; logistic regression as a special case of GLM; the median, quantile regression and robust loss functions.

Anomaly and novelty detection; heuristic anomaly scores: local density, isolation forest, a mixture with a background component, robust principal component analysis; outlier removal in nonparametric regression, the LOWESS algorithm; anomaly as the value of the loss function; the theory of robust learning: slowly growing functions, iterative reweighting of objects with IRS and IRLS, robust loss functions, the median, the quantile and the censored mean, robust aggregation functions and their smoothing, the IR-ERM algorithm; one-class SVM, learning from positive and unlabeled objects (PU-learning), problems with new and unknown classes.

Objects, features and answers; the types of supervised learning problems; parametric models and the learning method; loss functions and empirical risk minimization; the Runge example and overfitting; hold-out, leave-one-out and cross-validation; applied problems from medical diagnosis to sales forecasting; the CRISP-DM standard.

The interpretability of models; decision trees and the greedy ID3 algorithm; the axioms of impurity, the entropy and Gini criteria; handling missing values; the merits and drawbacks of the greedy strategy; tree pruning (post-pruning); CART regression trees and Minimal Cost-Complexity Pruning; rule induction: logical regularities, families of rules (conjunctions, syndromes, half-planes, balls), search metaheuristics, a tree as a covering set of conjunctions; the Pareto front in (p,n) space, logical and statistical regularities, Fisher's exact test, informativeness criteria; building a classifier from regularities.