Qwen3.8-Flash-Next (NVFP4) on a Google Colab G4, served by vLLM 0.32.0,
queried through term-llm. Companion files for
https://wasnotwas.com/writing/qwen38-flash-next-colab-g4-term-llm/

Everything here was run end to end on 2026-10-19/20 (UTC) by Jarvis.
No credentials, OAuth material, or Hugging Face tokens are included; none
were needed beyond the Colab CLI's own interactive login.

Model:   nvidia/Qwen3.8-Flash-Next-NVFP4
         revision fc694b54fb0174e0913e6adf86691ef85a4ead47
Runtime: Colab --gpu G4 (RTX PRO 6000 Blackwell Server Edition, 97,887 MiB,
         sm120, driver 580.82.07), 48 vCPU, 176 GB RAM, 236 GB overlay disk
Stack:   Ubuntu 24.04 image, Python 3.13, torch 2.13.0+cu130, CUDA 13.0,
         vLLM 0.32.0, flashinfer 0.7.0.post1 (+cubin, +jit-cache sm120f),
         term-llm v0.9.82, google-colab-cli 0.7.4

Files
  setup_vm.sh            run on the VM: vLLM, precompiled kernels, model
  start_vllm.sh          run on the VM: verified 131K serial server launch (original benchmark: 16K, max-num-seqs 8)
  tunnel_and_termllm.sh  run locally: SSH local forward + one term-llm request
  run_tl_bench.sh        run locally: tunnel + term-llm built-in benchmark
  term-llm-config.yaml   isolated provider config (install under a scratch
                         XDG_CONFIG_HOME as term-llm/config.yaml)
  bench_direct.py        direct OpenAI-API harness; run ON the VM against
                         http://127.0.0.1:8000/v1 so no network is measured
  benchmark_results.json consolidated results from both harnesses

Order of operations
  1. colab usage; colab sessions            (auth + balance check)
  2. colab new -s flashnext --gpu G4
  3. colab upload -s flashnext setup_vm.sh /content/setup_vm.sh  (same for
     start_vllm.sh and bench_direct.py)
  4. on the VM, detached:  nohup bash /content/setup_vm.sh > /content/setup.log 2>&1 &
     wait for SETUP_DONE in the log
  5. on the VM, detached:  nohup bash /content/start_vllm.sh > /content/vllm.log 2>&1 &
     wait for "Application startup complete."
  6. locally: ./tunnel_and_termllm.sh ; ./run_tl_bench.sh ...
  7. on the VM: BASE_URL=http://127.0.0.1:8000/v1 MODEL=qwen3.8-flash-next \
       OUT_TOKENS=256 REPEATS=3 INPUTS=128,512,2048,8192,12000 python3 -u bench_direct.py
  8. colab stop -s flashnext; colab sessions; colab usage  (must show 0 sessions,
     0.00/hr)

benchmark_results.json notes
  direct_localhost: medians of 3 measured repeats after 1 warmup per shape,
    unique prompts (random nonce), enable_thinking=false, ignore_eos with
    min_tokens=max_tokens=256, prefix_cache_hits delta verified 0.
    server_prefill_tps = delta(vllm:prompt_tokens_total) /
    delta(vllm:request_prefill_time_seconds_sum) from /metrics.
  term_llm: client-observed at the provider boundary through the SSH tunnel;
    the "fits" entry is term-llm's own median-pairwise-slope prefill fit
    (16,280 tok/s effective, 0.436 s fixed intercept). Compare slopes, not
    absolute TTFT, against direct_localhost.
  The per-session GPU UUID was removed from the hardware block; nothing else
  was edited.

Not included: Colab OAuth client config/token, the SSH private key, raw vLLM
logs, raw per-request JSONL (summarised in benchmark_results.json).

Updated default: --max-model-len 131072 --max-num-seqs 1; prompt plus output must fit.
Verified: 105.22K input via term-llm; 130,138 tokens direct, both retrieved planted code.
context_results.json contains the separate long-context measurements.
The 2/4-stream benchmark_results.json rows used the earlier 16K configuration.