OptimalFactor 1.5.0

Resubmission after CRAN review (2026-10-15)

• print.specification_search() documents its return value (\value).
• The internal helper that lets a user-written algorithm run on parallel workers in algorithm_stability() no longer refers to the global environment: it copies the objects the function needs into a new private environment and never modifies the user’s workspace.

Preparation for CRAN

• Title changed to “Item Selection in Factor Analysis by Boosting and Specification Search”. The Description cites Kenny and McCoach (2003) and Shi, Lee and Maydeu-Olivares (2019) for the adaptive fit weights; a non-existent “Kenny, Shi and Savalei (2022)” reference was removed from the Description, README and efa_boosting() help page.
• algorithm_stability(), cross_validate_cfa(), item_stability(), simulate_cfa_recovery() and simulate_recovery() no longer fix a seed by default (seed = NULL).
• bifactor_indices() and cross_validate_cfa() gain verbose; console output of non-print functions can be silenced.
• export_conceptual_analysis() no longer writes to the working directory by default: file must be supplied.
• optimal_cfa_with_ai(), optimal_efa_with_ai(), print_cfa_results() and print_efa_results() are now exported.
• All examples are self-contained; only the calls to the external language model service remain in \dontrun{}.
• Bug fixes: report_efa_results() failed on data frames (n partially matched na.print); optimal_cfa_with_ai() failed with the ML estimator (it requested rmsea.scaled).

Choosing between algorithms: how much theory survives, and does the choice replicate

The package offers several purification routines, and on a given scale they disagree. Two functions put them on the same footing so the choice can be argued rather than asserted.

• theory_recovery() measures how much of the theoretical key a solution keeps. A solution departs from the theory by dropping items or by moving them, and a single index confounds the two, so it reports retention, accuracy (matching factors to dimensions by the exhaustive optimum, not greedily), recovery (their product), the Adjusted Rand Index and, when loadings are available, Tucker congruence per dimension. It reads the output of any routine in the package, a plain partition or a loading matrix. Accuracy alone is misleading for confirmatory routines: cfa_boosting() can only drop items, so its accuracy is 1 by construction.

• algorithm_stability() runs any item-selection algorithm on random halves of the data and fits the structure it chooses on the other half. It reports whether the decision replicates (Jaccard overlap with the full-sample solution, per-item retention rates) and whether it holds out of sample (fit and largest interfactor correlation on the validation half), and, given a theory, the recovery on every split. Unlike item_stability() it is not tied to efa_boosting(), and unlike cross_validate_cfa() it is not limited to one factor. Splits run in parallel with n_cores. A timeout (default 300 s) counts a split whose algorithm does not finish as failed instead of stalling the run: on small derivation halves an iterative search can loop indefinitely.

Bug fix

• bifactor_indices() returned NA for omega, omega_H, PUC and every specific ECV when some items load on the general factor only, as in a bifactor S-1 model (Eid et al., 2017) or after dropping a specific factor that collapsed. It also reported an inadmissible solution without complaint (an omega of 2.4 in one case); it now warns when a standardized loading reaches 1 or a residual variance is not positive.

• discriminant_boosting() stopped with “arguments imply differing number of columns” when a structure in its ladder did not converge: the row for a non-converged model had two columns instead of all of them. It now carries every column, filled with NA.

OptimalFactor 1.4.0

Two routines for scales that fit but do not measure what they claim

A model can fit well and still be uninterpretable. Four factors correlating .97 reproduce the covariance matrix as well as one factor does, so CFI and RMSEA say nothing about whether the factors exist. These two functions target what the fit indices miss.

• discriminant_boosting() rescues the discriminant validity of a multidimensional scale whose interfactor correlations are above the threshold — or above 1, which makes the solution inadmissible — with the smallest possible departure from the theoretical structure. It walks a ladder of specifications ordered by distance from the original (theoretical model, plus a method factor, bifactor, and every partition of the factors into fewer blocks), prunes each level guided by the interfactor correlation, then by CFI, and keeps the model that meets every criterion with the most factors.

  Three design decisions come from what does not work. Pruning by fit leaves the interfactor correlation untouched, so the search is guided by it instead. Letting the data assign items surfaces the wording polarity of reverse items as if it were a dimension, so item-to-factor assignment stays theoretical and only whole factors are merged. Optimising fit and correlation at once lets fit dominate, so they are optimised in sequence.

• local_fit_search() searches for a specification of a one-factor model that reaches the fit targets, combining item removal with freed residual covariances. Where redundancy_short_form() prunes near-duplicate items, this one also weighs keeping the pair and modelling its covariance, and decides by the effect on fit.

  It encodes three steps normally taken by hand: items below the loading floor are removed without requiring that fit improve (an item that does not load cannot stay, and dropping it often worsens RMSEA at first); an item taking part in two or more covariances is dropped rather than kept, since the item is the problem and not the relations; and every retained covariance is tested for removal at the end, because each freed parameter is a debt.

  It does not judge whether a covariance makes substantive sense. It returns the estimate, its interval, its sign and the wording of both items. A negative residual covariance between items at opposite poles of a construct is often the trace of a dimension the unidimensional model absorbed, and is worth reading before treating it as noise.

Reliability and loading floors are now adaptive

Both routines treat omega_min and min_loading as adaptive: a model already below a floor is not frozen, its effective bar becomes its current value. A floor set above the starting value rejects every candidate and stalls the search on the first iteration — a failure mode indistinguishable, from the outside, from a genuine negative result.

Documentation

• specification_search_theory(): only_within_factor is documented, including why a residual correlation between items of different factors absorbs covariance that belongs at the factor level and makes discriminant validity look better than it is.

OptimalFactor 1.3.0

Breaking-free change: no more PsyMetricTools dependency

• The package no longer depends on PsyMetricTools. Every exploratory model was fitted through PsyMetricTools::EFA_modern(), a package that is not on CRAN, which blocked OptimalFactor’s own submission. The engine (model syntax → lavaan fit → fit measures → standardized pattern matrix) is now internal (R/efa_modern_internal.R) and calls lavaan::cfa() with exactly the same arguments (mimic = "Mplus", ordered = TRUE), so results are reproduced: on Data_Personality the largest loading difference against the old engine is 1.6e-05, which is smaller than the 1.9e-05 that lavaan itself produces between two identical runs. The internal version drops the purrr and magrittr usage of the original and returns a plain data.frame instead of a tibble.

New functions

• item_stability(): runs the whole efa_boosting() pipeline over bootstrap or subsample resamples and reports, per item, the retention and removal rates plus the stability of its factor assignment. Factor labels are aligned to a full-sample reference before agreement is computed, since rotation names factors arbitrarily. This is the empirical answer to the capitalization-on-chance objection (MacCallum, Roznowski & Necowitz, 1992): an item dropped in 97 of 100 resamples is a defensible decision, one dropped in 55 is not. Ships with a print method that flags decisions taken in 25–75 % of resamples as unstable. Resampling defaults to method = "subsample": drawing with replacement duplicates cases, which distorts the polychoric correlations ordinal estimation relies on, and on a sample of 100 leaves as few as 63 distinct cases, enough to send a WLSMV fit grinding for minutes. Measured on Data_Personality with R = 40 over 8 cores, bootstrap took 5m46s with 3 replications hitting the cap against 2m31s with none under subsampling.

• simulate_recovery(): Monte Carlo evidence on when the pipeline recovers a known structure. Simulates ordinal data from a population model with good, cross-loading and weak items, crosses conditions over sample size, loading size and items per factor, and reports recovery rate, sensitivity (contaminated items removed) and specificity (good items kept). Replications are fitted in parallel through n_cores, which is what makes a crossed design feasible; the datasets are always drawn sequentially from seed, so results do not depend on how many cores ran them. Ships print and plot methods. plot() draws the recovery curve against sample size, one line per population loading and one panel per number of items per factor, which is the figure a validation paper reports.

• simulate_cfa_recovery(): the confirmatory counterpart, running the same population model through cfa_boosting(). The question is not the same one: the structure is imposed rather than discovered, so the assignment cannot be wrong and what is measured is the item selection. Reports exact_rate (the removed set equals the contaminated set item for item), sensitivity, specificity and mean_covs_added, the residual covariances introduced where the population has none, which turns the MacCallum et al. (1992) objection into a number.

cfa_boosting() enforces its own loading floor

The confirmatory simulation was built to evaluate the pipeline and immediately found a defect in it. The optimisation loop tested the fit targets first and broke as soon as they were met, so the low-loading check further down was unreachable: an item the user’s own thresholds$loading declares inadmissible survived untouched whenever global fit happened to be acceptable. And global fit is acceptable far more often than one would hope. In simulation, a population item loading .20 sat inside a model with RMSEA = .03, and adding cross-loading items barely moved the index at all: with six items cross-loading at .60, RMSEA stayed at .053 while CFI rose to .997, because omitted cross-loadings are absorbed by the interfactor correlation instead of degrading fit.

• thresholds$enforce_loading (default TRUE): an item below the loading floor is now removed regardless of the fit indices, weakest first and one per iteration since the loadings are re-estimated after each refit. This is an admissibility criterion, not a fit optimisation, so the removal is unconditional rather than contingent on improving the loss. Items whose removal would breach min_items_per_factor are protected. FALSE restores the previous behaviour.
• Measured on the confirmatory simulation (12 replications, 12 good items at .60 plus one cross-loading and one weak item): sensitivity rose from .167 to .667 at N = 300 and from .208 to .583 at N = 600, with specificity unchanged or better (.965 to .986, .965 to .965). The floor finds the contaminated items without taking good ones along.
• The trade is real and worth stating. On Data_Personality the previous behaviour removed one item and bought its fit with a residual covariance (RMSEA .083); enforcing the floor removes the weak item as well, needs no covariance at all, and ends at RMSEA .091. Slightly worse global fit, a defensible retained set, and one less parameter that the population may not have.
• thresholds$enforce_simple_structure (default TRUE) and thresholds$cross_loading (default 0.30): the floor catches the item that measures nothing, not the one that measures two things. A cross-loading item still loads acceptably on its own factor, so the floor never sees it. Detection is by the modification index of the absent loading, but never by significance alone: the standardized EPC must also reach the threshold, since a significant index with a trivial EPC is the capitalization on chance the whole exercise is meant to avoid. Adding this took the confirmatory simulation from sensitivity .667/.583 to .958/.958 and exact recovery from .250/.167 to .917/.917, with specificity rising to 1.000 at both sample sizes, fewer spurious residual covariances (.17 to .08 and .08 to .00) and better final fit (RMSEA .031 to .013 and .035 to .004). Nothing was traded away for it.
• stop_reason is now returned, with values all_targets_met, targets_met_loading_protected, no_improving_action and max_iterations, so the caller no longer has to infer it from the log.

Misspecified dimensionality: n_factors_fitted

Every condition in a recovery design hands the pipeline the true number of factors, and in practice nobody knows it. simulate_recovery() gains n_factors_fitted, crossed like the other conditions, so the population keeps its own dimensionality while the algorithm is told something else.

The answer is asymmetric, and the asymmetry is the practical finding. Against a three-factor population with 12 good items, one cross-loading item and one weak one (12 replications):

Extracting too few factors is destructive: specificity collapses to .65-.73, so between three and four good items are discarded to force the data into a structure that cannot hold them. Extracting too many makes the pipeline inert instead: sensitivity falls to zero, no contaminated item is removed at all, and 18 of 24 replications stop at min_items_per_factor_protected, because with four factors and fourteen items almost any removal breaches the minimum. The scale survives untouched, which is the harmless failure.

The practical reading: when the dimensionality is uncertain, erring towards more factors leaves the algorithm conservative, while erring towards fewer leaves it removing good items.

Population model: cross-loading items no longer share a factor pair

simulate_recovery() and simulate_cfa_recovery() used to place every cross-loading item on the same two factors. With more than one such item that makes them clones: they correlate strongly, form a coherent cluster, and the rotation reports the cluster as a factor of its own, each item showing a single clean loading near .80. No item-level criterion can call that a cross-loading, because in the estimated solution it no longer is one, and the good items get displaced to make room for it. They are now spread over different pairs.

The correction matters because it inverts the verdict. Under three cross-loading items at .55 plus two weak ones, the old generator made efa_boosting() look defective (sensitivity .617/.700, recovery falling to .75 at N = 600, contaminated items left in the pool). With the population correctly specified the same conditions give sensitivity .983/1.000, recovery 1.000 and specificity 1.000, retaining 12.08 and 12.00 items against the 12 good ones in the population. The apparent defect was the measuring instrument.

Progress, timeouts and parallel robustness

Resampling exposed three things that made these functions unusable in practice rather than merely slow.

• Progress is now reported while a cluster works. The parallel branch used to print nothing at all, which is indistinguishable from a hung session. Both simulations and item_stability() now print a bar with elapsed time and an estimate of what is left, announce the cluster starting up, and say in advance whether the bar will advance per round or jump at the end. Status lines go through flush.console(), without which the R console holds them until the computation ends.
• timeout (default 120 s per replication) in item_stability(), simulate_recovery() and simulate_cfa_recovery(). A bootstrap draw of N = 100 can leave 63 distinct cases, and a WLSMV fit on the near singular matrix that follows runs for minutes: measured, one such replication passed 180 s without finishing, while the same one capped returns in 69 s. One replication like that stalls an entire run. Capped replications are counted and reported rather than hidden.
• Workers never outnumber tasks, and the master’s .libPaths() is pushed to them before loading the package, so a session that added a library at runtime (RStudio project libraries, renv) does not produce workers that cannot find OptimalFactor. That failure previously surfaced as an opaque checkForRemoteErrors() message, because the guard tested for an error while requireNamespace(quietly = TRUE) merely returns FALSE.

Performance

• performance$fit_target_only (default TRUE) in efa_boosting(). Evaluating a candidate reads only the n_factors solution, yet the engine fitted the 1..k-1 solutions as well and discarded them. Verified identical on Data_Personality and Data_Expectativas (structure, removed items, stop reason, RMSEA, omega and the fit table), with 38 % fewer lavaan fits and a 1.43x speedup where the greedy loop evaluates candidates. FALSE restores the previous behaviour.
• Cluster work is dispatched with parLapplyLB: replication times are heavy tailed, so load balancing matters more than it would with uniform tasks.

Bug fixes

• fits[[r]] <- NULL deleted the slot instead of filling it, and a failed replication returns exactly NULL. Any condition where a replication failed aborted with a subscript error; the same idiom in item_stability() kept its progress counter permanently at “0 failed”.
• The fitting workers addressed OptimalFactor::efa_boosting even when running in process, which fails under devtools::load_all() because nothing is exported yet. The error was swallowed and every replication was recorded as a silent non-convergence, a result that looked like evidence.
• The skewed response thresholds were not cumulative proportions, so the skew came out reversed with about 70 % of responses in the top category.
• Passing performance through ... to the resampling functions raised “formal argument matched by multiple actual arguments” instead of overriding the defaults. It is now merged.

Tests

• The package ships tests/testthat for the first time: 170 checks over the population model, the ordinal simulation, factor alignment, recovery scoring for both pipelines, the progress helpers and the timeout translation, with regression tests for each of the fixes above.

Reliability floor

• thresholds$min_omega in efa_boosting() and min_omega in redundancy_short_form(): an item is removed only if every factor keeps McDonald’s omega at or above the floor. It is deliberately a constraint and not a term in the loss — reliability is not interpretable in the ill-fitting intermediate models the loss visits, so omega vetoes individual removals instead of trading against the fit indices. A factor already below the floor is not frozen: its effective bar becomes its current omega, so removals that do not reduce reliability remain available. Heywood cases are exempt, because an inadmissible solution must be fixed regardless. NULL (default) reproduces the previous behaviour exactly.
• New stop reason min_omega_protected.
• efa_boosting() now always returns omega_final (omega per factor) and omega_check, whether or not the floor is active.
• redundancy_short_form() gained an omega column in its trajectory plus stop_reason and omega_blocked fields.

OptimalFactor 1.2.2

CRAN readiness

• R CMD check --as-cran now passes with 0 errors and 0 warnings (apart from the expected note about PsyMetricTools not yet being on CRAN): non-ASCII characters in R code were escaped to \uXXXX, runtime calls to install.packages() / devtools::install_github() were replaced by informative errors, library() calls inside functions were removed, and the Remotes field was dropped from DESCRIPTION.
• imprimir_items_eliminados() is now exported.
• Documentation fixes: usage sections synchronized with the real default arguments of cfa_boosting() and efa_boosting(), examples updated to the current cfa_boosting(data, model) signature, duplicated alias for print.specification_search removed, and the new print argument of report_efa_results() documented.
• parallel and stats are now declared in Imports.

OptimalFactor 1.2.1

• All AI calls (conceptual analysis in efa_boosting(), efa_optimizer(), optimal_efa_with_ai() and optimal_cfa_with_ai()) now honour the OPENAI_BASE_URL environment variable, so they can be redirected to any OpenAI-compatible provider — e.g. the Hugging Face router (https://router.huggingface.co/v1) to use models such as Qwen/Qwen2.5-72B-Instruct with an hf_ token. Default behaviour (variable unset) is unchanged.

OptimalFactor 1.2.0

Breaking changes

• The Shiny apps moved out of the package. run_efa_boosting() and run_efa_boosting_wizard() (plus inst/shiny-apps/ and the two app vignettes) were removed: the interactive OptimalFactor Wizard is now distributed separately as a web application on Posit Connect Cloud. The package keeps the analysis engine only, which also trims the suggested dependencies (shiny, bslib, DT, officer, flextable, readxl, readr, commonmark, later, future, promises are no longer suggested).

New functions

• specification_search_theory(): theory-guided specification search. Extends the MacCallum (1986) hill-climbing search with a theory-congruence term in the loss (penalizes moving items away from their theoretical factor and dropping theoretical items). A single theory_weight parameter grades how much theory counts relative to fit (0 reproduces the fit-only search).
• cross_validate_cfa(): split-half cross-validation of a factor model.
• bifactor_indices(): bifactor statistical indices.
• redundancy_short_form(): redundancy-guided short form of a unidimensional scale.

Deprecated

• specification_search() is deprecated in favour of specification_search_theory() and now emits a deprecation warning on every call. It is kept for backward compatibility.

OptimalFactor 1.1.0

New: Guided wizard for EFA-Boosting

• run_efa_boosting_wizard(): a second Shiny app that coexists with the original run_efa_boosting() studio. Provides a guided 5-phase flow (Data → Parallel diagnostic → EFA boosting → Reliability → External and convergent/discriminant validity), each with a “proposed action / what will happen” panel.

Wizard features

• Multi-method consensus for number of factors: Kaiser, parallel analysis (Horn, 1965), MAP (Velicer, 1976) and BIC. The user can accept the consensus recommendation or override it with a theoretical value.
• Trace tab: captures the verbose stdout of efa_boosting() plus the print_conceptual_analysis() output so the entire item-purification process is auditable in a single pane. Downloadable as .txt.
• AI integration: optional autopilot mode that uses OpenAI (gpt-4.1 default) to drive the wizard step by step. Includes a chat panel with Markdown rendering of replies (via commonmark).
• Reliability: omega (semTools::compRelSEM, ord.scale = TRUE) and Cronbach’s alpha per factor, reported alongside CFI/TLI/RMSEA/SRMR of the EFA-derived CFA model.
• Convergent / discriminant validity with multidimensional comparators:
  • Automatic detection of comparison instruments from column names via hierarchical regex (e.g. DERS_AC1, DERS_OB1 → multidimensional instrument with two subscales).
  • Mini parallel analysis on each candidate confirms or challenges the detected dimensionality.
  • AI auto-classification of each instrument as convergent or discriminant based on its label, using response_format = "json_object".
  • Per-pair verdict using Cohen’s (1988) magnitude conventions: convergencia fuerte / moderada / débil / no significativa for convergent expectations; discrimina / dudosa / NO discrimina for discriminant expectations.
  • Score construction preserves the comparator’s internal structure (sub-totals when multidimensional, optional grand total).
  • Heatmap visualization (ggplot2 geom_tile) with asterisks for p < .05.
• Downloadable session log (.txt): full audit trail of every phase, numerical detail, items removed, fit indices, omega/alpha and validity correlations.
• Downloadable manuscript (.docx): AI-drafted APA-7 sections “2.4 Análisis de datos” and “3. Resultados” (subsections 3.1–3.6 when convergent instruments are present). Tables are inserted from real session data via placeholders ({{TABLE_LOADINGS}}, {{TABLE_FIT}}, {{TABLE_OMEGA}}, {{TABLE_EXTERNAL}}, {{TABLE_CONVERGENT}}). The AI never invents numbers — it only writes prose around the embedded tables. Requires an OpenAI API key.
• Autopilot controls: each step exposes ← Back, ⏸ Pause autopilot and ▶ Resume AI in a persistent toolbar. The user can rewind to any previous phase (even from the “wizard completed” state) without losing previously computed results.

Improvements to efa_boosting()

• Added performance$max_candidates_eval (default 12) and performance$smart_pruning = TRUE: ranks candidate items by their maximum factor loading and evaluates only the top-K, giving large speed-ups on long instruments (e.g. 32 items: >30 min → ~2–3 min).
• New return field stop_reason with canonical values (all_criteria_met, min_items_per_factor_protected, max_iterations, not_enough_items, efa_convergence_failed, fit_target_reached, fit_zero_no_structural_problem, timeout). Allows downstream code to give precise messages without inferring why the loop stopped.

New helpers

• report_efa_results() / report_cfa_results(): structured text-and-data reports of the optimization process. Each returns an invisible list with a $text field suitable for embedding in manuscripts or printing to console.

OptimalFactor 1.0.0

Initial CRAN Release

Main Features

• efa_boosting(): Advanced iterative EFA optimization algorithm
  • Greedy and global search strategies for optimal item selection
  • Adaptive composite fit indices (RMSEA, SRMR, CFI) with weights based on df x N
  • Automatic Heywood case and near-Heywood detection
  • Cross-loading identification and removal
  • Minimum items per factor enforcement
  • Interfactor correlation verification
  • Optional AI-assisted conceptual analysis via GPT models
• run_efa_boosting(): Launch EFA-Boosting Studio
  • Interactive Shiny application for EFA optimization
  • Real-time console output with auto-scroll
  • Modern Bootstrap 5 interface
  • CSV/Excel data import
  • Configurable fit targets and thresholds
  • Option to exclude specific items
  • Export results to CSV
• cfa_boosting(): Confirmatory Factor Analysis optimization
  • Modification indices-based model improvement
  • Automatic covariance addition
  • Fit index monitoring
• print_conceptual_analysis(): Formatted output for AI analyses
  • Bilingual support (English/Spanish)
  • Technical statistics display
  • Narrative formatting
• export_conceptual_analysis(): Export analysis to various formats
  • Word document export
  • Plain text export
  • Formatted tables

Data

• Data_Expectativas: Example dataset for expectation scale
• Data_Personality: Example personality assessment dataset

Dependencies

• Requires PsyMetricTools for EFA_modern function
• lavaan for structural equation modeling
• psych for factor analysis utilities

References

• Kenny, D. A., & McCoach, D. B. (2003). Effect of the number of variables on measures of fit in structural equation modeling. Structural Equation Modeling, 10(3), 333-351.
• Shi, D., Lee, T., & Maydeu-Olivares, A. (2019). Understanding the model size effect on SEM fit indices. Educational and Psychological Measurement, 79(2), 310-334.