---
title: "Getting Started with NNS: Overview"
author: "Fred Viole"
output: html_vignette
vignette: >
  %\VignetteIndexEntry{Getting Started with NNS: Overview}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r}
#| label: setup
#| include: true
# Prereqs (uncomment if needed):
# install.packages("NNS")
# install.packages(c("data.table","xts","zoo","Rfast"))

suppressPackageStartupMessages({
  library(NNS)
  library(data.table)
})
set.seed(42)
```


```{r, include=FALSE, message=FALSE}
data.table::setDTthreads(1L)
options(mc.cores = 1)
RcppParallel::setThreadOptions(numThreads = 1)
Sys.setenv("OMP_THREAD_LIMIT" = 1)
```

# Orientation

**Goal.** 
A complete, hands‑on curriculum for Nonlinear Nonparametric Statistics (NNS) using **partial moments**. Each section blends narrative intuition, precise math, and executable code.

**Structure.** 

1. Foundations — partial moments & variance decomposition
2. Descriptive & distributional tools 
3. Dependence & nonlinear association 
4. Hypothesis testing & ANOVA (LPM‑CDF) 
5. Regression, boosting, stacking & causality 
6. Time series & forecasting 
7. Simulation (max‑entropy), Monte Carlo & risk‑neutral rescaling 
8. Portfolio & stochastic dominance

**Notation.** 
For a random variable $X$ and threshold/target $t$, the population $n$‑th **partial moments** are defined as

$$
\operatorname{LPM}(n,t,X) 
= \int_{-\infty}^{t} (t-x)^{n} \, dF_X(x),
\qquad
\operatorname{UPM}(n,t,X) 
= \int_{t}^{\infty} (x-t)^{n} \, dF_X(x).
$$

The **empirical** estimators replace $F_X$ with the empirical CDF $\hat F_n$ (or, equivalently, use indicator functions):

$$
\widehat{\operatorname{LPM}}_n(t;X) = \frac{1}{n} \sum_{i=1}^n (t-x_i)^n \, \mathbf{1}_{\{x_i \le t\}},
\qquad
\widehat{\operatorname{UPM}}_n(t;X) = \frac{1}{n} \sum_{i=1}^n (x_i-t)^n \, \mathbf{1}_{\{x_i > t\}}.
$$

These correspond to integrals over the measurable subsets $\{X \le t\}$ and $\{X > t\}$ in a $\sigma$‑algebra; the empirical sums are discrete analogues of Lebesgue integrals.

------------------------------------------------------------------------

# 1. Foundations — Partial Moments & Variance Decomposition

## 1.1 Why partial moments

-   Classical variance treats upside and downside symmetrically. Partial moments separate them, allowing **asymmetric risk/reward** analysis around a chosen target $t$ (often the mean or a benchmark).
-   At $t=\mu_X$: 
$$
\operatorname{Var}(X) = \operatorname{UPM}(2,\mu_X,X) + \operatorname{LPM}(2,\mu_X,X)\quad\text{(exact empirical identity)}.
$$
This **is not** the same as splitting conditional variances around a threshold; partial moments use a *global* reference, preserving the between‑group contribution.

## 1.2 Core functions and headers

-   `LPM(degree, target, variable)`
-   `UPM(degree, target, variable)`


## 1.3 Code: variance decomposition & CDF

```{r}
# Normal sample
y <- rnorm(3000)
mu <- mean(y)
L2 <- LPM(2, mu, y); U2 <- UPM(2, mu, y)
cat(sprintf("LPM2 + UPM2 = %.6f vs var(y)=%.6f\n", (L2+U2)*(length(y) / (length(y) - 1)), var(y)))

# Empirical CDF via LPM.ratio(0, t, x)
for (t in c(-1,0,1)) {
  cdf_lpm <- LPM.ratio(0, t, y)
  cat(sprintf("CDF at t=%+.1f : LPM.ratio=%.4f | empirical=%.4f\n", t, cdf_lpm, mean(y<=t)))
}

# Asymmetry on a skewed distribution
z <- rexp(3000)-1; mu_z <- mean(z)
cat(sprintf("Skewed z: LPM2=%.4f, UPM2=%.4f (expect imbalance)\n", LPM(2,mu_z,z), UPM(2,mu_z,z)))
```

**Interpretation.** The equality `LPM2 + UPM2 == var(x)` (Bessel adjustment used) holds because deviations are measured against the *global* mean. `LPM.ratio(0, t, x)` constructs an empirical CDF directly from partial‑moment counts.

------------------------------------------------------------------------

# 2. Descriptive & Distributional Tools

## 2.1 Higher moments from partial moments

Define asymmetric analogues of skewness/kurtosis using $\operatorname{UPM}_3$, $\operatorname{LPM}_3$ (and degree 4), yielding robust tail diagnostics without parametric assumptions.

**Header.** `NNS.moments(x)`

```{r}
M <- NNS.moments(y)
M
```

## 2.2 Mode estimation (no bin‑or‑bandwidth angst)

**Header.** `NNS.mode(x)`

```{r}
set.seed(23)
multimodal <- c(rnorm(1500,-2,.5), rnorm(1500,2,.5))
NNS.mode(multimodal,multi = TRUE)
```

## 2.3 CDF tables via LPM ratios

**Headers.**

-   `LPM.ratio(degree = 0, target, variable)` (empirical CDF when `degree=0`)
-   `UPM.ratio(degree = 0, target, variable)`
-   `LPM.VaR(p, degree, variable)` (quantiles via partial‑moment CDFs)
-   `UPM.VaR(p, degree, variable)`

```{r}
qgrid <- LPM.VaR(seq(0.05,0.95,.1),0,z) # equivalent to quantile(z,probs = seq(0.05,0.95,by=0.1))
CDF_tbl <- data.table(threshold = as.numeric(qgrid), CDF = LPM.ratio(0,qgrid,z))
CDF_tbl
```

------------------------------------------------------------------------

# 3. Dependence & Nonlinear Association

## 3.1 Why move beyond Pearson $r$

Pearson captures linear monotone relationships. Many structures (U‑shapes, saturation, asymmetric tails) produce near‑zero $r$ despite strong dependence. Partial‑moment dependence metrics respond to such structure.

**Headers.** 

- `Co.LPM(degree, target, x, y)` / `Co.UPM(...)` (co‑partial moments) 
- `PM.matrix(l_degree, u_degree, target=NULL, variable, pop_adj=TRUE)` 
- `NNS.dep(x, y)` (scalar dependence coefficient) 
- `NNS.copula(X, target=NULL, continuous=TRUE, plot=FALSE, independence.overlay=FALSE)`

## 3.2 Code: nonlinear dependence

```{r}
set.seed(1)
x <- runif(2000,-1,1)
y <- x^2 + rnorm(2000, sd=.05)
cat(sprintf("Pearson r = %.4f\n", cor(x,y)))
cat(sprintf("NNS.dep  = %.4f\n", NNS.dep(x,y)$Dependence))

X <- data.frame(a=x, b=y, c=x*y + rnorm(2000, sd=.05))
pm <- PM.matrix(1, 1, target = "means", variable=X, pop_adj=TRUE)
pm

cop <- NNS.copula(X, continuous=TRUE, plot=FALSE)
cop
```

## 3.3 Code: copula

```{r, eval=FALSE}
# Data
set.seed(123); x = rnorm(100); y = rnorm(100); z = expand.grid(x, y)

# Plot
rgl::plot3d(z[,1], z[,2], Co.LPM(0, z[,1], z[,2], z[,1], z[,2]), col = "red")

# Uniform values
u_x = LPM.ratio(0, x, x); u_y = LPM.ratio(0, y, y); z = expand.grid(u_x, u_y)

# Plot
rgl::plot3d(z[,1], z[,2], Co.LPM(0, z[,1], z[,2], z[,1], z[,2]), col = "blue")
```

**Interpretation.** `NNS.dep` remains high for curved relationships; `PM.matrix` collects co‑partial moments across variables; `NNS.copula` summarizes higher‑dimensional dependence using partial‑moment ratios. Copulas are returned and evaluated via `Co.LPM` functions.

------------------------------------------------------------------------

# 4. Hypothesis Testing & ANOVA (LPM‑CDF)

## 4.1 Concept

Instead of distributional assumptions, compare groups via **LPM‑based CDFs**. Output is a *degree of certainty* (not a p‑value) for equality of populations or means.

**Header.** 

- `NNS.ANOVA(control, treatment, means.only=FALSE, medians=FALSE, confidence.interval=.95, tails=c("Both","left","right"), pairwise=FALSE, plot=TRUE, robust=FALSE)`

## 4.2 Code: two‑sample & multi‑group

```{r}
ctrl <- rnorm(200, 0, 1)
trt  <- rnorm(180, 0.35, 1.2)
NNS.ANOVA(control=ctrl, treatment=trt, means.only=FALSE, plot=FALSE)

A <- list(g1=rnorm(150,0.0,1.1), g2=rnorm(150,0.2,1.0), g3=rnorm(150,-0.1,0.9))
NNS.ANOVA(control=A, means.only=TRUE, plot=FALSE)
```

**Math sketch.** For each quantile/threshold $t$, compare CDFs built from `LPM.ratio(0, t, •)` (possibly with one‑sided tails). Aggregate across $t$ to a certainty score.

------------------------------------------------------------------------

# 5. Regression, Boosting, Stacking & Causality

## 5.1 Philosophy

`NNS.reg` learns **partitioned** relationships using partial‑moment weights — linear where appropriate, nonlinear where needed — avoiding fragile global parametric forms.

**Headers.** 

- `NNS.reg(x, y, order=NULL, smooth=TRUE, ncores=1, ...)` → `$Fitted.xy`, `$Point.est`, … 
- `NNS.boost(IVs.train, DV.train, IVs.test, epochs, learner.trials, status, balance, type, folds)` 
- `NNS.stack(IVs.train, DV.train, IVs.test, type, balance, ncores, folds)` 
- `NNS.caus(x, y)` (directional causality score via conditional dependence)

## 5.2 Code: classification via regression + ensembles

```{r, fig.width=7, fig.height=5, fig.align='center'}
# Example 1: Nonlinear regression
set.seed(123)
x_train <- runif(200, -2, 2)
y_train <- sin(pi * x_train) + rnorm(200, sd = 0.2)

x_test <- seq(-2, 2, length.out = 100)

NNS.reg(x = data.frame(x = x_train), y = y_train, order = NULL)
```


```{r, eval = FALSE}
# Simple train/test for boosting & stacking
test.set = 141:150
 
boost <- NNS.boost(IVs.train = iris[-test.set, 1:4], 
              DV.train = iris[-test.set, 5],
              IVs.test = iris[test.set, 1:4],
              epochs = 10, learner.trials = 10, 
              status = FALSE, balance = TRUE,
              type = "CLASS", folds = 5)


mean(boost$results == as.numeric(iris[test.set,5]))
[1] 1


boost$feature.weights; boost$feature.frequency

stacked <- NNS.stack(IVs.train = iris[-test.set, 1:4], 
                     DV.train = iris[-test.set, 5],
                     IVs.test = iris[test.set, 1:4],
                     type = "CLASS", balance = TRUE,
                     ncores = 1, folds = 1)
mean(stacked$stack == as.numeric(iris[test.set,5]))
[1] 1
```

## 5.3 Code: directional causality

```{r}
NNS.caus(mtcars$hp,  mtcars$mpg)  # hp -> mpg
NNS.caus(mtcars$mpg, mtcars$hp)   # hp -> mpg
```

**Interpretation.** Examine asymmetry in scores to infer direction. The method conditions partial‑moment dependence on candidate drivers.

------------------------------------------------------------------------

# 6. Time Series & Forecasting

**Headers.**

- `NNS.ARMA`
- `NNS.ARMA.optim`
- `NNS.seas`
- `NNS.VAR`

```{r , fig.width=7, fig.align='center'}
# Univariate nonlinear ARMA
z <- as.numeric(scale(sin(1:480/8) + rnorm(480, sd=.35)))

# Seasonality detection (prints a summary)
seasonal_period <- NNS.seas(z, plot = FALSE)
head(seasonal_period$all.periods)

# Validate seasonal periods
NNS.ARMA.optim(z, h=48, seasonal.factor = seasonal_period$periods, plot = TRUE, ncores = 1)
```

**Notes.** NNS seasonality uses coefficient of variation instead of ACF/PACFs, and NNS ARMA blends multiple seasonal periods into the linear or nonlinear regression forecasts.

------------------------------------------------------------------------

# 7. Simulation, Bootstrap & Risk‑Neutral Rescaling

## 7.1 Maximum entropy bootstrap (shape‑preserving)

**Header.** 

- `NNS.meboot(x, reps=999, rho=NULL, type="spearman", drift=TRUE, ...)`

```{r}
x_ts <- cumsum(rnorm(350, sd=.7))
mb <- NNS.meboot(x_ts, reps=5, rho = 1)
dim(mb["replicates", ]$replicates)
```

## 7.2 Monte Carlo over the full correlation space

**Header.** 

- `NNS.MC(x, reps=30, lower_rho=-1, upper_rho=1, by=.01, exp=1, type="spearman", ...)`

```{r}
mc <- NNS.MC(x_ts, reps=5, lower_rho=-1, upper_rho=1, by=.5, exp=1)
length(mc$ensemble); names(mc$replicates)

head(mc$replicates$`rho = 0`)
```

## 7.3 Risk‑neutral rescale (pricing context)

**Header.** 

- `NNS.rescale(x, a, b, method=c("minmax","riskneutral"), T=NULL, type=c("Terminal","Discounted"))`

```{r}
px <- 100 + cumsum(rnorm(260, sd = 1))
rn <- NNS.rescale(px, a=100, b=0.03, method="riskneutral", T=1, type="Terminal")
c( target = 100*exp(0.03*1), mean_rn = mean(rn) )
```

**Interpretation.** `riskneutral` shifts the mean to match $S_0 e^{rT}$ (Terminal) or $S_0$ (Discounted), preserving distributional shape.

------------------------------------------------------------------------

# 8. Portfolio & Stochastic Dominance

Stochastic dominance orders uncertain prospects for broad classes of risk‑averse utilities; partial moments supply practical, nonparametric estimators.

**Headers.** 

- `NNS.FSD.uni(x, y)`
- `NNS.SSD.uni(x, y)`
- `NNS.TSD.uni(x, y)` 
- `NNS.SD.cluster(R)`
- `NNS.SD.efficient.set(R)`

```{r}
RA <- rnorm(240, 0.005, 0.03)
RB <- rnorm(240, 0.003, 0.02)
RC <- rnorm(240, 0.006, 0.04)

NNS.FSD.uni(RA, RB)
NNS.SSD.uni(RA, RB)
NNS.TSD.uni(RA, RB)

Rmat <- cbind(A=RA, B=RB, C=RC)
try(NNS.SD.cluster(Rmat, degree = 1))
try(NNS.SD.efficient.set(Rmat, degree = 1))
```

------------------------------------------------------------------------

# Appendix A — Measure‑theoretic sketch (why partial moments are rigorous)

Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space, $X: \Omega\to\mathbb{R}$ measurable. For any fixed $t\in\mathbb{R}$, the sets $\{X\le t\}$ and $\{X>t\}$ are in $\mathcal{F}$ because they are preimages of Borel sets. The **population** partial moments are

$$
\operatorname{LPM}(k,t,X) = \int_{-\infty}^{t} (t-x)^k\, dF_X(x),
\qquad
\operatorname{UPM}(k,t,X) = \int_{t}^{\infty} (x-t)^k\, dF_X(x).
$$

The **empirical** versions correspond to replacing $F_X$ with the empirical measure $\mathbb{P}_n$ (or CDF $\hat F_n$):

$$
\widehat{\operatorname{LPM}}_k(t;X) = \int_{(-\infty,t]} (t-x)^k\, d\mathbb{P}_n(x),
\qquad
\widehat{\operatorname{UPM}}_k(t;X) = \int_{(t,\infty)} (x-t)^k\, d\mathbb{P}_n(x).
$$

Centering at $t=\mu_X$ yields the variance decomposition identity in Section 1.

------------------------------------------------------------------------

# Appendix B — Quick Reference (Grouped by Topic)

## 1. Partial Moments & Ratios
- `LPM(degree, target, variable)` — lower partial moment of order `degree` at `target`.
- `UPM(degree, target, variable)` — upper partial moment of order `degree` at `target`.
- `LPM.ratio(degree, target, variable)`; `UPM.ratio(...)` — normalized shares; `degree=0` gives CDF.
- `LPM.VaR(p, degree, variable)` — partial-moment quantile at probability `p`.
- `Co.LPM(degree, target, x, y)` — co-lower partial moment between two variables.
- `Co.UPM(degree, target, x, y)` — co-upper partial moment between two variables.
- `D.LPM(degree, target, variable)` — divergent lower partial moment (away from `target`).
- `D.UPM(degree, target, variable)` — divergent upper partial moment (away from `target`).
- `NNS.CDF(x, target = NULL, points = NULL, plot = TRUE/FALSE)` — CDF from partial moments.
- `NNS.moments(x)` — mean/var/skew/kurtosis via partial moments.

## 2. Descriptive Statistics & Distributions
- `NNS.mode(x, multi=FALSE)` — nonparametric mode(s).
- `PM.matrix(l_degree, u_degree, target, variable, pop_adj)` — co-/divergent partial-moment matrices.
- `NNS.gravity(x, w = NULL)` — partial-moment weighted location (gravity center).
- `NNS.norm(x, method = "moment")` — normalization retaining target moments.

See NNS Vignette: [Getting Started with NNS: Partial Moments](https://CRAN.R-project.org/package=NNS/vignettes/NNSvignette_Partial_Moments.html)

## 3. Dependence & Association
- `NNS.dep(x, y)` — nonlinear dependence coefficient.
- `NNS.copula(X, target, continuous, plot, independence.overlay)` — dependence from co-partial moments.

See NNS Vignette: [Getting Started with NNS: Correlation and Dependence](https://CRAN.R-project.org/package=NNS/vignettes/NNSvignette_Correlation_and_Dependence.html)

## 4. Hypothesis Testing
- `NNS.ANOVA(control, treatment, ...)` — certainty of equality (distributions or means).

See NNS Vignette: [Getting Started with NNS: Comparing Distributions](https://CRAN.R-project.org/package=NNS/vignettes/NNSvignette_Comparing_Distributions.html)


## 5. Regression, Classification & Causality
- `NNS.part(x, y, ...)` — partition analysis for variable segmentation.
- `NNS.reg(x, y, ...)` — partition-based regression/classification (`$Fitted.xy`, `$Point.est`).
- `NNS.boost(IVs, DV, ...)`, `NNS.stack(IVs, DV, ...)` — ensembles using `NNS.reg` base learners.
- `NNS.caus(x, y)` — directional causality score.

See NNS Vignette: [Getting Started with NNS: Clustering and Regression](https://CRAN.R-project.org/package=NNS/vignettes/NNSvignette_Clustering_and_Regression.html)

\medskip

See NNS Vignette: [Getting Started with NNS: Classification](https://CRAN.R-project.org/package=NNS/vignettes/NNSvignette_Classification.html)

## 6. Differentiation & Slope Measures
- `dy.dx(x, y)` — numerical derivative of `y` with respect to `x` via `NNS.reg`.
- `dy.d_(x, Y, var)` — partial derivative of multivariate `Y` w.r.t. `var`.
- `NNS.diff(x, y)` — derivative via secant projections.

## 7. Time Series & Forecasting
- `NNS.ARMA(...)`, `NNS.ARMA.optim(...)` — nonlinear ARMA modeling.
- `NNS.seas(...)` — detect seasonality.
- `NNS.VAR(...)` — nonlinear VAR modeling.
- `NNS.nowcast(x, h, ...)` — near-term nonlinear forecast.

See NNS Vignette: [Getting Started with NNS: Forecasting](https://CRAN.R-project.org/package=NNS/vignettes/NNSvignette_Forecasting.html)

## 8. Simulation, Bootstrap & Rescaling
- `NNS.meboot(...)` — maximum entropy bootstrap.
- `NNS.MC(...)` — Monte Carlo over correlation space.
- `NNS.rescale(...)` — risk-neutral or min–max rescaling.

See NNS Vignette: [Getting Started with NNS: Sampling and Simulation](https://CRAN.R-project.org/package=NNS/vignettes/NNSvignette_Sampling.html)

## 9. Portfolio Analysis & Stochastic Dominance
- `NNS.FSD.uni(x, y)`, `NNS.SSD.uni(x, y)`, `NNS.TSD.uni(x, y)` — univariate stochastic dominance tests.
- `NNS.SD.cluster(R)`, `NNS.SD.efficient.set(R)` — dominance-based portfolio sets.


For complete references, please see the Vignettes linked above and their specific referenced materials.


