Search results for “обобщающая способность”

The problem of estimating generalization ability; the error matrix and the probability of overfitting; the case of a single algorithm and the hypergeometric distribution; the principle of uniform convergence of frequencies and Boole's inequality; the Vapnik–Chervonenkis bound, the growth function and the capacity of a family of algorithms; inverting the bound and the complexity penalty; structural risk minimization; the splitting and similarity effects; the Occam's razor bound and its link to regularization.

Measures of classification quality: type I and type II errors, the ROC curve and AUC, gradient maximization of AUC, log-loss, precision and recall, micro- and macro-averaging, the F-measure; internal and external criteria of generalization ability; hold-out, leave-one-out, q-fold and t×q-fold cross-validation, stratification; model consistency; regularization, AIC, BIC and the Vapnik–Chervonenkis bound; multi-criteria model selection; feature selection: full search, Add, Add-Del, depth-first search, beam search (GMDH), the genetic algorithm and random search with adaptation.

Bias–variance analysis: the decomposition of the error into noise, bias and variance, the bias and variance of simple voting, why complex ensembles do not overfit; gradient boosting for an arbitrary loss function as a parametric approximation of a gradient step; stochastic gradient boosting; special cases: regression, AdaBoost, AnyBoost, variants of loss functions; XGBoost: the tree as a linear model, L0 and L2 regularisation, analytical leaf values and the splitting criterion; CatBoost: leakage in the gradients, ordered boosting on subsamples of doubling length, unbiased target statistics for categorical features.