Humans asked to name a "random" number between 1 and 100 disproportionately choose 37 — a well-documented cognitive bias toward numbers that feel random. We ran the equivalent test on google/gemma-4-E4B-it, sampling 100,000 responses across 10 distinct prompt phrasings at temperature 1.0 with no nucleus truncation. The distribution is catastrophically non-uniform: 42 accounts for 35.2% of all responses, and numbers 88–100 are never produced. Chi-square against uniform yields p ≈ 0, KL divergence = 9.75. The bias is consistent across all prompt variants, ruling out prompt sensitivity as an explanation.
The result is consistent with training data encoding the cultural salience of 42 (the "answer to everything" from The Hitchhiker's Guide to the Galaxy), though the precise mechanism — whether RLHF, SFT corpus frequency, or tokenizer-level bias — is not isolatable without access to training internals.
Download notebook80, 88, 89, 90, 95, 96, 97, 98, 99, 100 — and most numbers above ~75 are near-zero.
The classic human "random" bias number appears at 2.18% (2,175 occurrences vs. 1,000 expected) — elevated, but completely overshadowed by 42.
Ten distinct phrasings of the same task were used. All 10 reject uniformity at p ≈ 0. Top-3 numbers shift slightly by phrasing, but the 42/73/47/57 cluster dominates throughout.
1. Model: google/gemma-4-E4B-it loaded in bfloat16 via HuggingFace Transformers 5.9.0.
2. Sampling: do_sample=True, temperature=1.0, top_p=1.0 — unmodified logits, stochastic output.
3. Prompts: 10 semantically equivalent phrasings, each queried 10,000 times (100,000 total). Prompts wrapped in Gemma instruct chat template.
4. Parsing: first integer in [1,100] extracted from each response via regex. 5 responses produced no valid integer.
5. Statistics: chi-square goodness-of-fit vs. uniform(1,100), KL divergence, per-prompt chi-square, top/bottom frequency ranking.
The notebook was designed for Colab T4 (16GB VRAM) with batch size 32 and an estimated 4–6 hour runtime. Halfway through setup I got impatient, switched to an NVIDIA RTX PRO 6000 Blackwell (102GB VRAM), bumped batch size to 256, and knocked out 100k samples in ~2.4 minutes. The T4 path — including the OOM handler that halves batch size on cuda out-of-memory — is still in the notebook and works fine if you actually want to run it on Colab. The stats don't care what GPU you used.
pip install transformers bitsandbytes accelerate scipy tqdm
Load google/gemma-4-E4B-it, run generate_batch() across PROMPT_TEMPLATES.
See notebook: gemma4_favorite_number.ipynb