ByteDance/Ouro-1.4B

Text generation with vllm-rlt on NVIDIA H20-3e.

Text generationBF16vllm-rlt1 × H20-3eunverified

INSTALLvllm-rlt · CUDAShell

Run in the prepared environment

git clone https://github.com/InferMatrixLab/vllm-rlt.git
cd vllm-rlt
git checkout 9d601ceaf9af63121c742df5f584467e23630a0b
uv venv --python 3.10 .venv
source .venv/bin/activate
uv pip install -e .
python - <<'PY'
from huggingface_hub import snapshot_download
snapshot_download(repo_id="ByteDance/Ouro-1.4B",
                  revision="574fa66cb8bf5abdc979642d01cf2b79b16bfab1",
                  local_dir="./artifacts/models/Ouro-1.4B",
                  allow_patterns=["*.json", "*.safetensors", "*.model", "*.txt"])
PY

CUDA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES-0}" python -m vllm_rlt.entrypoints.serve \
  --model ./artifacts/models/Ouro-1.4B --served-model-name ouro \
  --device cuda --dtype bfloat16 --attention-backend triton \
  --num-blocks 64 --max-num-seqs 1 --host 127.0.0.1 --port 8000

HARDWARE

NVIDIA

Huawei Ascend

Apple Silicon

VARIANT

bf16

ByteDance/Ouro-1.4B

ENGINE

BACKEND

Linux

TASK

Single device · recorded configuration

VERIFICATION

Deployment unverifiedAcceptance INCONCLUSIVE

Not revalidated for this catalog. See the original evidence for the exact scope.

ADVANCED Source & reproducibility

Deployment ref: 9d601ceaf9af63121c742df5f584467e23630a0b
Recipe source: InferMatrixLab/vllm-rlt/recipes/ouro-1-4b-rlt-h20-3e-bf16.yaml
Content commit: 326f7edb8580b6c729ba52ce5b15ed67e7ab649e
Updated: 2026-10-19
Maintainers: @bjf-frz

Edit the source recipe

Environment & prerequisites

Linux

One allocated NVIDIA H20-3e; Linux, Git, curl, uv and Python 3.10+.
CUDA-enabled PyTorch 2.5+ and a compatible NVIDIA driver; use the Triton attention backend.
BF16 weights need roughly 3 GB before KV and runtime buffers; this first-request configuration bounds the KV pool and active sequences.
Initial installation and model download require package-index, GitHub and Hugging Face access.

Limitations & verification scope

This exact deployment ref and first-request configuration have not been re-run on H20-3e during catalog work.
H20-3e BF16 hardware scope is taken from the historical README benchmark; those earlier runtime results are not qualification of this ref or request.
Source and historical protocol: https://github.com/InferMatrixLab/vllm-rlt/blob/9d601ceaf9af63121c742df5f584467e23630a0b/README.md
Uses default synchronous execution and exit_threshold 1.0 (fixed four loops). It does not enable asynchronous scheduling, preemption, self-speculation or disaggregation.
Async execution with preemption and async self-speculation have documented correctness/lifecycle limitations. The small KV pool is a first-request setting, not a throughput profile.
Linux distribution and resolved CUDA/PyTorch/Triton versions must be captured during remote validation. Preserve scheduler-provided CUDA_VISIBLE_DEVICES instead of overriding it.

Evidence & attribution

Source recipe Engine deployment revision

Imported reports retain their authorship and verification scope. Catalog checks do not establish inference quality or performance.