Geometrical Underpinnings
of Machine Learning

Lorien Hayden

Unsupervised Machine Learning Anaylsis of US Stock Markets

Archetypal Analysis

$ R_{ts} \sim E_{tf}W_{fs} $ where $E_{tf} = R_{ts'}C_{s'f}$ $$ C_{s'f} \geq 0, \Sigma_{s'}C_{s'f}=1 $$ $$ W_{fs} \geq 0, \Sigma_{f} W_{fs} = 1 $$

Dimensionality

Takeaways

$r^2$ values and PVE are comparable to the performance of the benchmark factor decomposition of Fama and French

Provides principled means to analyze company exposure and create sector specific indices

Neural Networks through the lens of Manifold Learning

Why does Science Work?

Hyper-ribbons

Hyper-ribbons

A Systems Biology Example

A Systems Biology Example

Jeffrey's Prior

$$ H\approx\sum_k\bigg{(}\frac{\partial y_k^\theta}{\partial\theta_i}\frac{\partial y_k^\theta}{\partial\theta_j}\bigg{)}=J^TJ$$

$$ p(\theta)=\sqrt{|g_{\alpha\beta}|}=\sqrt{|J^TJ|}=\prod_i \Sigma_i $$

emcee: the MCMC Hammer

Manifold Widths

Single Digit Network

Boundary Relationship

N-1 and N-dimensional descriptions have a boundary relationship: $M_{N−1} \approx \partial M_{N}$

Transtrum et. al. has explored the relationship between manifold boundaries and emergent model classes

Paradox

For a hyper-ball 95.8 % of the the volume is within 10% of the surface.

Conversely, for a 10-D sphere with radius 1 and cap height 0.1, the volume of the cap corresponds to a tiny 0.0014%

Jeffrey's Prior $\rightarrow$ Generative Model (e.g. VAE)

Curse of Dimensionality

InPCA: Katherine Quinn et. al.

Takeaways

The geometry of a model manifold can inform approaches to model simplification and reduction

Computational Neural Networks display a unique manifold geometry

Supplement: Canonical Sectors

Supplement: Model Manifold