ODIN / POCKETFrom-scratch causal language model
A little model.
A page of possibilities.
Give ODIN a beginning. See where a language model
small enough to run locally takes it.
Your continuation will appear here.
Generate a continuation using ODIN.
Under the words
After generation, explore the tokens and the probability the model assigned to each one.
Tokens appear here after a run
Which ending fits?
Compare how likely the model finds each continuation.
Its preference is a language score, not a fact check.
Nothing behind the curtain.
One small model, a visible training run, and results
bound to the checkpoint you're actually using.
Trainable parametersUnmeasured50,000,000 maximum
Training tokens seenUnmeasuredNo training record loaded
Training hardwareUnmeasuredRandomly initialized weights
Learning, measured
● Held-out development lossTraining measurements appear after a checkpoint is loaded.
Cross-entropy loss on a fixed development sample. Lower is better.
Built to fit
- Architecture
- Causal Transformer
- Layers / width
- Unmeasured
- Attention
- Unmeasured
- Feed-forward
- SwiGLU
- Tokenizer
- Unmeasured
- Weights
- Tied input / output
- Inference
- Checking…
The competition tests
Not yet measured| Benchmark | What it tests | Accuracy / perplexity | Coverage |
|---|
Zero-shot lm-evaluation-harness results. WikiText-103 token perplexity uses this model's tokenizer; it is not directly comparable across tokenizers.
Checkpoint identity and training provenance
No checkpoint loaded.