---
title: "Data structures for rasch models"
author: "Joshua A. McGrane"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Data structures for rasch models}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

```{r library}
library(rasch)
```

This vignette shows the data required by each model. The examples use plain
data frames so that the required columns are visible. Column names can differ;
name their roles in the fitting function.

## Rasch, partial credit and rating scale models

Use one row per person and one column per item. Scores begin at zero. A
dichotomous item is coded 0/1; an item with four categories is coded 0, 1, 2
or 3. Missing values may be `NA`; negative missing-value codes are also
accepted through `na_codes`.

```{r wide-data}
responses <- data.frame(
  person = c("P01", "P02", "P03", "P04"),
  group = c("control", "control", "treated", "treated"),
  I1 = c(0, 1, 1, 1),
  I2 = c(0, 1, 2, 2),
  I3 = c(1, 2, 3, NA)
)
responses
```

The identifier and person factors are not item responses. Name them
explicitly, or nominate the item columns.

```{r ordinary-fit, eval=FALSE}
fit <- rasch(
  responses,
  id = "person",
  factors = "group",
  items = c("I1", "I2", "I3"),
  model = "PCM"
)
```

Use `model = "RSM"` only when the items share one rating-scale threshold
structure. Items may have different maximum scores in the partial credit
model.

External weights are supplied after fitting. Item weights are a named numeric
vector. Set weights additionally need an item-to-set map, except for an EFRM
fit, which already contains one.

```{r person-weights, eval=FALSE}
item_weights <- c(I1 = 2, I2 = 1, I3 = 0.5)
weighted <- weighted_person_estimates(fit, item_weights)

set_of <- c(I1 = "core", I2 = "core", I3 = "extension")
set_weights <- c(core = 2, extension = 1)
weighted_person_estimates(fit, set_weights, by = "set", sets = set_of)

save_outputs(fit, "analysis", person_weights = weighted) # new or empty folder
```

Only relative weights matter. The weighted estimates are supplementary; they
do not replace the Rasch estimates used for fit, reliability, targeting or
DIF. In the application, use a CSV with `item,weight`, or
`item,set,weight` for set weights. The computed table can also be passed to
`save_outputs()`, `report_html()` or `report_document()`.

## Explanatory item and threshold models

The response data remain in the wide form above. A second data frame --- the
one uploaded as a CSV in the application --- describes the items. Its shape
depends on the level being explained, and that is the distinction to get
right before anything else.

An **item-level** design has one row per item. The predictors describe the
item as a whole, and the model explains the item's location.

A **threshold-level** design has one row per fitted *threshold*, identified by
`item` and `threshold` together. The predictors may vary within an item, and
the model explains each threshold separately.

The number of rows a threshold design needs is fixed by the data, not chosen:
an item scored 0 to *m* has *m* thresholds. For the responses above that is
one for `I1`, two for `I2` and three for `I3` --- six rows, not three.

```{r threshold-count}
items <- c("I1", "I2", "I3")
maxima <- vapply(responses[items], max, 0, na.rm = TRUE)
data.frame(item = items, max_score = maxima, thresholds = maxima,
           row.names = NULL)
sum(maxima)          # rows a threshold-level design must have
```

### One row per item

```{r item-predictors}
item_design <- data.frame(
  item = items,
  format = c("selected", "constructed", "constructed"),
  demand = c(0.2, 0.7, 1.1)
)
item_design
```

```{r explanatory-item-fit, eval=FALSE}
fit <- rasch_explanatory(
  responses,
  predictors = item_design,
  formula = ~ format + demand + format:demand,
  level = "item",
  id = "person",
  factors = "group",
  items = items
)
```

### One row per threshold

The same two predictors at threshold level. `format` is a property of the
item, so it simply repeats down that item's thresholds; `demand` now varies
between them, which is the reason to work at this level at all. An item-level
predictor is not excluded from a threshold design --- it is constant within
each item.

```{r threshold-predictors}
threshold_design <- data.frame(
  item = rep(items, maxima),
  threshold = sequence(maxima),
  format = rep(c("selected", "constructed", "constructed"), maxima),
  demand = c(0.2,
             0.5, 0.9,
             0.4, 0.8, 1.4)
)
threshold_design
nrow(threshold_design) == sum(maxima)
```

```{r explanatory-threshold-fit, eval=FALSE}
fit <- rasch_explanatory(
  responses,
  predictors = threshold_design,
  formula = ~ format + demand,
  level = "threshold",
  id = "person",
  factors = "group",
  items = items
)
```

`threshold` numbers the thresholds within an item, from 1, in order. It is
not a running count across the instrument: `I2` has thresholds 1 and 2, and
so does the first half of `I3`. `sequence(maxima)` above generates exactly
that pattern, and is worth using in place of typing the column out.

Three mistakes account for most refused designs. Each is named precisely
when it happens, so the message identifies the rows to fix:

- one row per item where the level is `"threshold"`, leaving the design short
  by every threshold after the first, and every threshold missing where the
  numbering does not match. Both report *threshold-level predictors are
  missing*, listing the item and threshold of each;
- thresholds numbered continuously across the instrument (1 to 6 here)
  rather than from 1 within each item, which reports the same missing
  combinations alongside *predictor rows do not correspond to an observed
  item threshold* for the numbers that exist nowhere;
- a repeated `item` and `threshold` pair, which reports *threshold-level
  predictors need one row per item and threshold*.

Categorical predictors should be factors, continuous predictors numeric, and
ordinal predictors ordered factors with their substantive order declared. In
the application, upload the same table as a CSV: `item` plus the predictor
columns at item level, and `item,threshold` plus the predictor columns at
threshold level. The predictor type of each column is set after the upload,
so a column read as text can still be declared ordinal with its order given.

## Multiple ratings

Long data use one row per observed rating. The required roles are person,
item, score and at least one facet. A rater, task or occasion can be a facet.
A person-group variable is instead supplied through `factors`.

```{r mfrm-long}
ratings <- data.frame(
  person = c("P01", "P01", "P02", "P02"),
  item = c("Essay1", "Essay2", "Essay1", "Essay2"),
  rater = c("R1", "R2", "R2", "R1"),
  occasion = c("first", "first", "first", "first"),
  score = c(2, 3, 1, 2)
)
ratings
```

```{r mfrm-fit, eval=FALSE}
fit <- rasch_mfrm(
  ratings,
  person = "person",
  item = "item",
  score = "score",
  facets = c("rater", "occasion")
)
```

Wide multiple-ratings data use one row per person-by-facet combination and
one score column per item.

```{r mfrm-wide}
wide_ratings <- data.frame(
  person = c("P01", "P01", "P02", "P02"),
  rater = c("R1", "R2", "R1", "R2"),
  Essay1 = c(2, 3, 1, 2),
  Essay2 = c(3, 2, 2, 2)
)
wide_ratings
```

```{r mfrm-wide-fit, eval=FALSE}
fit <- rasch_mfrm(
  wide_ratings,
  person = "person",
  facets = "rater",
  items = c("Essay1", "Essay2")
)
```

Long data are usually easier to inspect when a facet varies within items.

## Extended Frames

EFRM response data use one row per person and one column per item, together
with a person-group column. A separate named map assigns each item to exactly
one item set. Supplied person identifiers must not repeat; missing or blank
identifiers are treated as different unknown persons.

```{r efrm-structure}
frame_data <- data.frame(
  person = paste0("P", 1:6),
  group = rep(c("A", "B"), each = 3),
  S1I1 = c(0, 1, 1, 0, 1, 1),
  S1I2 = c(0, 0, 1, 0, 1, 1),
  S2I1 = c(0, 1, 1, 0, 0, 1),
  S2I2 = c(0, 1, 1, 0, 1, 1)
)
frame_data

item_sets <- list(
  set1 = c("S1I1", "S1I2"),
  set2 = c("S2I1", "S2I2")
)
item_sets
```

```{r efrm-fit, eval=FALSE}
fit <- rasch_efrm(
  frame_data,
  item_sets = item_sets,
  groups = "group",
  id = "person"
)
```

Persons must connect the item sets, and common items across person groups must
identify the group units. Crossed person-group factors can be supplied as
several column names in `groups`.

## Comparative Judgement

Use one row per comparison. `object_a` and `object_b` identify the pair. For a
dichotomous comparison, `winner` contains one of those two object names.

```{r cj-data}
comparisons <- data.frame(
  object_a = c("A", "A", "B", "A"),
  object_b = c("B", "C", "C", "C"),
  winner = c("A", "C", "B", "A"),
  judge = c("J1", "J1", "J2", "J2")
)
comparisons
```

```{r cj-fit, eval=FALSE}
fit <- btl(
  comparisons,
  object_a = "object_a",
  object_b = "object_b",
  winner = "winner",
  judge = "judge"
)
```

For ordered comparisons, replace `winner` with an ordered response column
whose categories run from preference for object B to preference for object A.
A frequency column can represent repeated identical rows. A judgement-order
column is needed to estimate exposure or carry-over dependence.

Explanatory Comparative Judgement adds one metadata row per object.

```{r cj-explanatory}
object_design <- data.frame(
  object = c("A", "B", "C"),
  genre = factor(c("essay", "essay", "report")),
  length = c(800, 950, 700)
)
object_design
```

```{r cj-explanatory-fit, eval=FALSE}
fit <- btl_explanatory(
  comparisons,
  predictors = object_design,
  formula = ~ genre + length,
  object_a = "object_a",
  object_b = "object_b",
  winner = "winner",
  judge = "judge"
)
```

## Extended Frames for Comparative Judgement

The comparison rows additionally identify each judge's panel. A named list
assigns every object to one object set. Both within-set and cross-set
comparisons are required to identify set units and origins.

```{r cj-frames}
object_sets <- list(
  set1 = c("S1A", "S1B", "S1C"),
  set2 = c("S2A", "S2B", "S2C")
)
object_sets

# the first four rows compare within a set, the last two across sets
frame_comparisons <- data.frame(
  object_a = c("S1A", "S1B", "S2A", "S2B", "S1A", "S1C"),
  object_b = c("S1B", "S1C", "S2B", "S2C", "S2A", "S2C"),
  winner   = c("S1A", "S1C", "S2A", "S2B", "S2A", "S1C"),
  judge    = c("J1", "J1", "J2", "J2", "J3", "J3"),
  panel    = c("east", "east", "west", "west", "east", "east")
)
frame_comparisons
```

```{r cj-frames-fit, eval=FALSE}
fit <- btl_efrm(
  frame_comparisons,
  object_a = "object_a",
  object_b = "object_b",
  winner = "winner",
  judge = "judge",
  panels = "panel",
  object_sets = object_sets
)
```

## Simulated data

The **Simulate** page in the Shiny application generates every structure
above, assigns its roles and retains the generating values. The corresponding
R functions are `simulate_rasch()`, `simulate_btl()`, `simulate_mfrm()`,
`simulate_efrm()` and `simulate_btl_efrm()`. Explanatory simulations add the
item or object metadata used to generate the locations. Each simulator stores
its parameter values and planted departures in `attr(data, "truth")`. The app
can download the data as a CSV or as a bundle containing the generating call,
true values and explanatory metadata.

```{r simulation-map}
d <- simulate_efrm(n_per_group = 100, items_per_set = 5, seed = 4)
names(attr(d, "truth"))
```