Reptimeline¶
Tracking how discrete neural representations evolve during training.
The Blind Spot¶
Current training tools (WandB, TensorBoard) log scalar metrics -- codebook utilization, perplexity, loss curves. But they don't show what changed or why.
When codebook collapse happens at step 27,500:
- Before reptimeline: You see utilization drop. That's all.
- With reptimeline: You know which codes died, whether they died together, that a phase transition caused it, and exactly which concepts lost unique representations.
How It Works¶
reptimeline tracks lifecycle events in discrete representation systems:
- Births: When a code element first activates for a concept
- Deaths: When a code element permanently deactivates
- Connections: When two concepts first share a discrete feature
- Phase transitions: Automatic detection via statistical outlier analysis
Comparison with Existing Tools¶
| Feature | VQ-VAE tools | SAE-Track | CBM tools | RepE | reptimeline |
|---|---|---|---|---|---|
| Per-code lifecycle | -- | -- | -- | -- | Yes |
| Connection tracking | -- | -- | -- | -- | Yes |
| Phase transition detection | -- | -- | -- | -- | Automatic |
| Churn heatmaps | -- | -- | -- | -- | Yes |
| Hierarchical dependencies | -- | -- | -- | -- | Yes |
| 3-way AND-gate interactions | -- | -- | -- | -- | Yes (novel) |
| Backend-agnostic | -- | -- | -- | -- | Yes |
Works With Any Discrete Bottleneck¶
- Triadic bits (TriadicGPT)
- VQ-VAE codebooks
- FSQ (Finite Scalar Quantization) levels
- Sparse autoencoders (SAEs)
- Concept bottleneck models
- Binary codes
- Any system quantizing neural representations into discrete codes
Proof-of-Concept Results¶
Real data from TriadicGPT (40M params, 63 bits, 8 checkpoints):
| Metric | Value |
|---|---|
| Bit births | 2,632 |
| Bit deaths | 1,732 |
| Connections formed | 1,378 |
| Phase transitions detected | 3 |
| Training phases identified | 4 (Exploration, Consolidation, Crystallization, Stabilization) |
Phase Transitions Detected¶
| Step | Metric | Event |
|---|---|---|
| 7,500 | Churn rate | Increase (delta=1.000) |
| 27,500 | Entropy | Decrease (delta=0.263) |
| 27,500 | Utilization | Decrease (delta=0.189) |
Dual Coherence (Anti-correlation of Opposites)¶
| Pair | Coherence |
|---|---|
| placer / dolor | 1.00 (perfect) |
| libertad / control | 0.94 |
| vida / muerte | 0.88 |
| verdad / mentira | 0.50 |
What's Next¶
- Quickstart -- install and run your first analysis
- Architecture -- 4-layer design, adding new backends
- Discovery Pipeline -- bottom-up ontology discovery
- Visualization -- 4 visualization types with examples