InferMatrix/Recipes

LiquidAI/LFM2.5-350M

Choice scoring with system1-omni on NVIDIA L40S.

View on Hugging Face
Choice scoringFP16system1-omni1 × L40Sunverified
INSTALLsystem1-omni · CUDAShell
Run in the prepared environment
git clone https://github.com/InferMatrixLab/system1-omni.git
cd system1-omni
git checkout e66ff34996760a84ad56690f0ae4cf434e669e65
python3.12 -m venv .venv-lfm
.venv-lfm/bin/python -m pip install -r src/models/lfm2/requirements.lock.txt
source .venv-lfm/bin/activate
cargo build --release --locked
Unverified on NVIDIA L40S
# First terminal, from the repository root with .venv-lfm activated:
(cd src/models/lfm2 && CUDA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES-0}" python worker.py \
  --candidate-batch-size 16 --device cuda --dtype float16 --host 127.0.0.1 --port 8000)
# Second terminal, from the same repository root after worker readiness:
OMNI_JEV_BIND=127.0.0.1:8080 OMNI_JEV_BACKEND_URL=http://127.0.0.1:8000 \
  ./target/release/omni-jev
HARDWARE
NVIDIA
Huawei Ascend
Apple Silicon
VARIANT
fp16
LiquidAI/LFM2.5-350M
ENGINE
BACKEND
Linux
TASK
Single device · recorded configuration
VERIFICATION
Deployment unverifiedAcceptance INCONCLUSIVE

Not revalidated for this catalog. See the original evidence for the exact scope.

ADVANCED Source & reproducibility
Deployment ref
e66ff34996760a84ad56690f0ae4cf434e669e65
Recipe source
InferMatrixLab/system1-omni/recipes/lfm2-5-350m-system1-l40s-fp16.yaml
Content commit
733be7a6a47bace63b8acfd8d455f72cbc781c9b
Updated
2026-10-19
Maintainers
@prettygirlisnotme
Edit the source recipe

Environment & prerequisites

Linux

  • One allocated NVIDIA L40S, Python 3.12, Git, curl and a stable Rust toolchain capable of Cargo resolver 3.
  • Install src/models/lfm2/requirements.lock.txt (recorded torch 2.14.0 and transformers 5.17.0); a compatible NVIDIA driver is required.
  • The worker loads LiquidAI/LFM2.5-350M at 9e6c6ccf47cd318696e137d381a7ded8fe4df09f.
  • Initial Rust/Python installation and checkpoint download require network access.

Limitations & verification scope

  • This deployment ref has not been re-run on L40S during catalog work; historical reports identify earlier Python source by hashes and an earlier Rust frontend commit.
  • This path is a Python Transformers FP16 worker behind a Rust frontend, not native Rust/CUDA model execution.
  • Only text state and independent choice questions are supported by this worker. Confidence is normalized entropy concentration, not calibrated correctness.
  • Historical exact hardware, evidence and boundaries: https://github.com/InferMatrixLab/system1-omni/blob/e66ff34996760a84ad56690f0ae4cf434e669e65/recipe/lfm2/README.md
  • The Linux distribution and current Rust/CUDA runtime must be captured in remote validation. Keep the worker and frontend running in separate terminals.
  • No repository-wide root license was found at this ref. Preserve component notices, the MIT RLCD reference notice and the model license; repository-wide licensing remains a maintainer follow-up.

Evidence & attribution

Source recipe Engine deployment revision

Imported reports retain their authorship and verification scope. Catalog checks do not establish inference quality or performance.