The coordinate-descent kernel behind space() did not
synchronize its record of the previous coefficients with the starting
coefficients. With the default soft-threshold start (and with warm
starts), the first sweep updated every pair relative to zero instead of
its starting value, so the algorithm could converge to a point that does
not solve the lasso problem. Estimates from space() and
matSPACE() change accordingly.
The BIC used by matSPACE() to select the penalty
applied the sig_i / sig_j rescaling twice (once when
converting partial correlations to regression coefficients and again
inside the BIC), so the residual sums of squares, and in some cases the
selected penalty, were wrong. The BIC is now
sum_j N * sf * log(RSS_j) + log(N) * deg_j / 2 with
RSS_j the residual sum of squares of variable
j under
b_kj = rho_kj * sqrt(sig_k / sig_j).
space() now centers the data before re-estimating
the residual precisions (the compiled kernel already centered
internally), and starts the residual precisions at the inverse sample
variances instead of 1.
lambda.bound() now builds the penalty sequence from
the column-centered data, the same centering as the fit.
matSPACE() fits each penalty path in compiled code from
the q x q Gram matrix of the centered data, so the
n * p * q data are touched only once per path. The fits are
the same as calling space() at every penalty, and are much
faster to compute.The outer iterations (residual precision and weight updates) now
run until convergence. A new argument outer_tol (default
1e-8) has been added to space() and
matSPACE(), and iter, now the maximum number
of outer iterations, defaults to 500 instead of 2. Use
iter = 2, outer_tol = 0 to reproduce the previous fixed two
iterations.
The default column weighting is now "variance"
(f_type of space(), f_type_V and
f_type_U of matSPACE()); it was
"equal".
matSPACE() gains sf, the scale factor
of the log-RSS term of the BIC (default 1), and eps, which
is passed to lambda.bound(). The default K is
now 80 (was 30).
lambda.bound() now defaults to
eps = 1e-4 (was 1e-6) and K = 80
(was 30).
space() now returns outer_iter, and it
no longer prints a message at every outer iteration.
Each fit stored in attr(matSPACE(...), "path") now
holds ParCor, sig, f,
rss, deg, outer_iter,
total_iter and cd_capped. The residual
matrices E and beta are no longer stored; call
space() at the selected penalty to obtain them.
matSPACE(), space() and
lambda.bound() now check data.
compute_BIC() (used internally by
matSPACE() to select the BIC-minimizing penalty) computed
residuals on raw data while the coefficients came from column-centered
data. It now uses centered data.matSPACE()’s API is simplified: the
sf_vec argument is removed, and the return value is a
single result list (fit$V) instead of a list keyed by
scaling factor (fit$sf_1$V).
An experimental active-set variant of the shooting algorithm was removed in favor of the full-sweep implementation.