TriadicGPT is a 40M-parameter GPT that learns algebraically verifiable prime-factor semantic representations end-to-end — as a side effect of language modeling.
Benchmark Results
Adding a triadic head to a GPT costs nothing in language quality — and produces algebraically verifiable semantic representations.
| Finding | Result |
|---|---|
| Language cost of triadic head | Zero (PPL 7.69 vs 7.56 ablation) |
| Semantic ordering emergence | Phase transition at 40M params |
| Optimal bit width | k = 32–64 bits |
| Analogy verification | 69.2% (random baseline 50%) |
| Semantic compression | 8× (64 bits = 512D probe accuracy) |
| Signature uniqueness | 100% across all concepts |
| GPT-2 transfer (InfoNCE) | Closes 72% of gap to Engine PCA |
Architecture
Standard next-token prediction plus a triadic projection head that produces discrete prime-factor signatures.
Bits fire ~50% of the time. No degenerate all-ones or all-zeros.
Different sequences produce different bit patterns.
No dead bits — every bit carries information.
Triadic similarity matches embedding similarity. The key innovation.
Experiment 10 — Transfer Learning
Same GPT-2 embeddings, 9× gap difference from changing only the alignment loss.
Semantic gap (intra − inter group similarity). Higher = better domain separation. Engine PCA is the post-hoc upper bound.
Quick Start
Standalone PyPI package. Triadic algebra + HuggingFace wrapper. Works with any causal LM.
~76 min on an RTX 5060 Ti. Produces a 40M-param model with 64-bit triadic signatures.
Desktop Explorer
A full PySide6 desktop application for encoding, comparing, and chatting with TriadicGPT. 3 backends, 7 tabs.
Click any prime factor to see all 280 probe words that share it. Discover what each learned prime “means” empirically.
Load native .pt checkpoints, GPT-2 Transfer (Exp10), or any HuggingFace model with post-hoc projection.
Converse with the model and see the prime signature of every turn in real time. Compare prompt vs response algebraically.
Research Paper
We present TriadicGPT, a 40M-parameter GPT augmented with a triadic projection head that produces discrete prime-factor signatures alongside standard next-token predictions. Unlike post-hoc approaches that project frozen embeddings into prime space, TriadicGPT learns algebraically verifiable semantic representations end-to-end as a side effect of language modeling. Through 29 training runs and 11 experiments, we demonstrate: (i) zero language cost from the triadic head, (ii) a phase transition in semantic ordering at 40M parameters, (iii) 8× semantic compression (64 bits match 512D embedding probes), and (iv) a loss–embedding interaction where InfoNCE closes 72% of the gap to post-hoc PCA when attached to GPT-2.
Ecosystem
End-to-end GPT + triadic head. 29 training runs, 11 experiments, full paper. MIT license.
GitHubStandalone package: triadic algebra + HuggingFace wrapper. Works with any causal LM out of the box.
PyPICommercial SaaS. Encode, search, audit, and stream via REST API. Free tier available.
Learn moreThe original library and paper. Post-hoc prime factorization for any embedding model. Published on Zenodo.
GitHub