Submit a recipe

OpenRecipes takes two kinds of contribution: a faster recipe for a model and GPU type, including a pair the catalog does not list yet, and a search procedure that finds recipes, such as a tuner, an agent or a set of rules. Submit a procedure through the recipes it finds; each is one recipe file, and its notes describe or link the procedure.

A recipe is one YAML file at recipes/<model-slug>/<gpu>/<handle>-<layout>.yaml: the layout, the command as run, the measured result, the stack it ran on, the correctness evidence and, where needed, notes. You run the commands below yourself and open a pull request with the file.

Steps

1. Pick the pair and the reference recipe. Recipes rank within a model and GPU type, on any number of GPUs. The reference recipe is the one you tune from: a public recipe, the pair's current best or one you wrote. You need not publish it. Apply the objective conventions to both.
2. Build the rows with data/ (its README gives the command per tokenizer) and check them with python3 data/verify_rows.py <rows.parquet> --recipe <a recipe of the same model>.
3. Measure. Run the recipe once under the protocol: 8,192-token rows, global batch 128, BF16, 20 steps with the profiler off, scored over steps 5–19. Record the score, the step-time CV, the GPU-hours and the framework versions.
4. Check correctness. Run a 20-step arm of the reference recipe on the same image and rows, compare its per-step loss with the measured run's using Megatron-Bridge's validate_convergence, and class every argument that differs between the two. Each same-math swap needs a passed component test at the model's shape. OpenRecipes lists a recipe once all of them pass.
5. Write the recipe from the template below. Count N with python3 tools/active_params.py <hf-repo>, take the rate from gpus.yaml, and add notes only for what is specific to the recipe: the steps every recipe shares are under Running a recipe, and those a model's recipes share go in the model's notes in models.yaml. Add the model to models.yaml if it is missing, and put any code beyond the stock stack under patches/.
6. Validate and build. python3 tools/validate.py checks the file against the schema and recomputes the cost, the MFU, the data-parallel size and the window; python3 tools/build.py rebuilds site/, and python3 tools/build.py --check and python3 -m pytest -q tests must pass. Commit the rebuilt site/ with the recipe.
7. Open a pull request with the recipe, any new patches and the rebuilt site.

Leave out internal paths, image registry addresses, account or cluster names, and datasets you may not cite.

The recipe file

The full schema is schema/recipe.schema.yaml; every recipe in recipes/ is a worked example.

schema_version: 1
model: Qwen3.5-122B-A10B          # as in models.yaml; add the model there if it is missing
gpu: {type: H200, memory_gb: 141, count: 32, per_node: 8, instance: AWS p5en.48xlarge}
author: {github: your-handle}     # or {name: <tool>, url: https://...}
status: verified
stack:                            # the versions the measured run used; "not pinned" when unknown
  framework: slime <version> (THUDM/slime <commit>)
  megatron_lm: <commit> (megatron-core <version>)
  transformer_engine: <version>
  pytorch: <version>
  cuda: <version>
  nccl: <version>
workload:
  row_tokens: 8192
  rows: SFT conversations
  global_batch: 128
  precision: bf16
  dataset:
    name: Tulu 3 SFT mixture
    source: allenai/tulu-3-sft-mixture
    revision: b14afda60f1bbebe55d5d2fa1e4df5042f97f8be
    license: ODC-By 1.0
    rows_sha256: <printed by data/verify_rows.py>
    build: {name: qwen35-tulu3-fixed8192-v1, tokenizer: <name and tokenizer.json sha256>, loss_mask: qwen3_5,
            rows: 6400, row_tokens: 8192, seed: 42, builder: data/generate_fixed_sft.py}
layout:
  label: TP1 PP4 EP8 DP8
  summary: <the full layout in words>
  highlights: <the settings that set the recipe apart>
  tp: 1
  pp: 4
  vpp: 1
  cp: 1
  ep: 8
  etp: 1
  dp: 8                           # gpu.count / (tp x pp x cp)
  sequence_parallel: false
  layer_split: null               # or {first: 11, last: 10}
  max_tokens_per_gpu: 8192
  rows_per_microbatch: 1
  recompute: {granularity: full, method: uniform, num_layers: 1}
  offload: null                   # or {optimizer_fraction: 0.9, overlap_d2h_h2d: true}
  dispatcher: alltoall
result:
  tokens_per_s_per_gpu: <score over steps 5-19>
  usd_per_million_tokens: <rate in gpus.yaml / (tokens/s/GPU x 3600) x 1e6, to the cent>
  step_time_cv_percent: <CV of the step time over the window>
  mfu: <6 x N x tokens/s/GPU / peak, or null>
  mfu_note: null                  # why, when mfu is null
  active_params: {count: <N from tools/active_params.py>, source: https://huggingface.co/<repo>, revision: <commit>}
  measured: 2026-10-14
  steps_per_run: 20
  window: [5, 19]
  window_tokens: 15728640         # 15 steps x 128 rows x 8,192 tokens
  runs: 1
  profiler: false
  gpu_hours: <GPUs x (admission to finish)>
  gpu_hours_note: GPUs x (scheduler admission to finish), including startup
correctness:
  result: pass
  summary: <one line>
  reference: <the reference recipe, or "Not published.">
  configuration: pass
  component_tests: []             # one entry per same-math swap: test, shape, gpu, date, result, detail
  convergence: {steps: 20, result: pass, correlation: <r>, final_loss_difference: <d>, pointwise_failures: 0}
  rules: <the rule lists the check used>
env: {CUDA_DEVICE_MAX_CONNECTIONS: "1"}
patches: []                       # files under patches/ the recipe needs
run: |
  <the training command as run, with paths replaced by variables>
notes: |                          # optional; plain text, blank lines separate paragraphs, "- " starts a list item
  <what a reader needs to run this recipe that the protocol and the model's notes do not say>

Fields

Generated from the schema. tools/validate.py also checks the cross-field rules listed at the top of the schema file.