TRexSelector R package guide

T-Rex+GVS: grouped variable selection

Groups of correlated variables

In genomics and finance, variables come in groups of highly correlated members that carry the same signal. The lasso picks one member of a group and drops the others, while the elastic net [1] spreads the coefficient over the group and lets the members enter together. The T-Rex selector for grouped variable selection (GVS) [2] runs the terminating elastic net (T-ENET) inside the random experiments and draws its dummies in groups too, so that the dummies mirror the group structure of the null variables. The informed elastic net (IEN) [3] goes one step further and encodes the known or estimated groups in the penalty. It keeps the grouping effect of the elastic net at a lower computation time, and it is the elastic net when every variable is its own group.

Data with groups

Twenty groups of five variables, each group driven by one latent factor. The first two groups are active.

set.seed(11)
n <- 100
p <- 100
group_size <- 5
groups <- rep(seq_len(p / group_size), each = group_size)
latent <- matrix(rnorm(n * max(groups)), n, max(groups))
X <- 0.9 * latent[, groups] + sqrt(1 - 0.9^2) * matrix(rnorm(n * p), n, p)
true_support <- which(groups %in% 1:2)
beta <- numeric(p)
beta[true_support] <- 1
y <- as.numeric(X %*% beta + rnorm(n))

The elastic net policy

gvs_type = "EN" is the default. The selector clusters the variables by correlation (single linkage, cut at corr_max), draws its dummies per cluster and runs the T-ENET solver chosen by en_solver ("TENET" by default and "TENET_AUG", the row-augmented form, which select the same variables). The elastic-net penalty \(\lambda_2\) is fixed here. A base control that is passed has to name a solver of the selector, on this track "TENET", because trex_control() names "TLARS" by default, which the selector refuses. Without a base control the selector runs the defaults of trex_control() with "TENET".

ctrl <- trex_control(solver = "TENET", K = 10)
gvs <- TRexGVSSelector$new(X, y, tFDR = 0.1, seed = 5, verbose = FALSE,
                           gvs_control = trex_gvs_control(gvs_type = "EN", lambda_2 = 1),
                           control = ctrl)
gvs$select()
sort(gvs$selected_indices)
#>  [1]  1  2  3  4  5  6  7  8  9 10
trex_fdp(gvs$selected_indices, true_support)
#> [1] 0
trex_tpp(gvs$selected_indices, true_support)
#> [1] 1

The fields record the clustering and the penalty.

gvs$gvs_type
#> [1] "EN"
gvs$max_clusters
#> [1] 20
gvs$hc_method_used
#> [1] "single"
gvs$lambda2_used
#> [1] 1
table(gvs$groups)[1:5]
#> 
#> 1 2 3 4 5 
#> 5 5 5 5 5

Choosing lambda_2

A negative lambda_2 (the default -1) lets the selector determine the penalty. lambda2_method names the rule. For the elastic net the default is "CV_1SE_CCD", a ridge cross-validation by coordinate descent read with the one-standard-error rule, with "CV_MIN_CCD", "CV_1SE_SVD" and "CV_MIN_SVD" as alternatives. The one-standard-error rule is preferred because the minimum rule tends to pin the penalty to the floor of the grid. cv_n_folds, cv_n_lambda and cv_seed control the cross-validation. lambda2_cv_diagnostics is the health record of the tuner: whether it ran, its seconds, and n_profile_at_edge, the number of \(\lambda_2\) values whose profiled \(\lambda_1\) optimum sat on a truncation edge of the path (0 is healthy).

trex_gvs_lambda2_power_rule() chooses the penalty by the selector’s own principle instead. It runs one selection per value of a grid with common dummies and returns the smallest \(\lambda_2\) whose selection is within one selection of the largest, together with the selection of every grid value.

cv <- TRexGVSSelector$new(X, y, tFDR = 0.1, seed = 5, verbose = FALSE,
                          gvs_control = trex_gvs_control(gvs_type = "EN", cv_n_lambda = 100),
                          control = ctrl)
cv$select()
cv$lambda2_used
#> [1] 100.2816
str(cv$lambda2_cv_diagnostics)
#> List of 12
#>  $ ran                   : logi TRUE
#>  $ seconds               : num 0.0726
#>  $ n_profile_at_edge     : int 0
#>  $ support_cap           : int -1
#>  $ dfmax_ratio           : num 0
#>  $ low_dimensional       : logi FALSE
#>  $ grid_index            : int 83
#>  $ grid_size             : int 100
#>  $ at_grid_edge          : logi FALSE
#>  $ profile_at_edge       : logi FALSE
#>  $ profile_at_support_cap: logi FALSE
#>  $ boundary_pick         : logi FALSE

The informed elastic net

gvs_type = "IEN" encodes the groups in the penalty. The groups come from groups in the control (one label per variable, numbered 1 to \(M\) without gaps) or from the correlation clustering. The solver of this track is named in the base control, trex_control(solver = ): "TIENET" (the native pathwise form) or "TIENET_AUG" (row augmented). The selector refuses every other solver there, so gvs_type = "IEN" needs a base control. The IEN rules of lambda2_method ("CV_1SE_IEN_CCD", the default on this track, "CV_MIN_IEN_CCD", "CV_1SE_TIK_SVD", "CV_MIN_TIK_SVD") consume the group structure and return \(\lambda_2\) on the solver scale.

ien <- TRexGVSSelector$new(X, y, tFDR = 0.1, seed = 5, verbose = FALSE,
                           gvs_control = trex_gvs_control(gvs_type = "IEN", groups = groups,
                                                          lambda_2 = 1),
                           control = trex_control(solver = "TIENET", K = 10))
ien$select()
sort(ien$selected_indices)
#> [1]  1  2  4  5  6  7 10
trex_fdp(ien$selected_indices, true_support)
#> [1] 0
trex_tpp(ien$selected_indices, true_support)
#> [1] 0.7

With the groups known and the penalty informed, the active groups enter complete.

Examples

The TRexSelector_Examples repository holds runnable demos and Monte Carlo studies of T-Rex+GVS on block designs of several kinds, with their results and a README per demo, in R, Python and C++.

Bibliography

[1]
H. Zou and T. Hastie, “Regularization and variable selection via the elastic net,” J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 67, no. 2, pp. 301–320, 2005, doi: 10.1111/j.1467-9868.2005.00503.x.
[2]
J. Machkour, M. Muma, and D. P. Palomar, “False discovery rate control for grouped variable selection in high-dimensional linear models using the T-Knock filter,” in Proc. 30th eur. Signal process. Conf. (EUSIPCO), 2022, pp. 892–896. doi: 10.23919/EUSIPCO55093.2022.9909883.
[3]
J. Machkour, M. Muma, and D. P. Palomar, “The informed elastic net for fast grouped variable selection and FDR control in genomics research,” in Proc. IEEE 9th int. Workshop comput. Adv. Multi-sensor adapt. Process. (CAMSAP), 2023, pp. 466–470. doi: 10.1109/CAMSAP58249.2023.10403489.