artifact
active
artifact:one-layer-transformer-1l-modelOne-layer transformer (1L model)
2.9M parameter decoder-only transformer studied, d_m=256, 4 heads, 4096 vocab.
Neighborhood — ranked by edge-count
Papers (1)
paper
Datasets (2)
dataset
- English and code training text for the transformer.
- 4096-token BPE vocabulary.