Luxi Inference · research program

Deterministic, energy-aware transformer execution.

Luxi Inference is LuxiEdge's research program for transformer workloads. It is secondary to LuxiQuant. It is not a full production serving replacement, and there is no universal vLLM win. Batch-specific prefill is not end-to-end inference.

Evidence register Prefill evaluation offer Older inference notes

What this program studies

Prefill

Reads the prompt. Measured in prefill positions, not generated tokens. This is the stage with independent and internal numbers.

Decode

Produces output tokens. The current story is reference agreement under a defined numerical audit, not a serving offer.

Receipts and determinism

Where a contract exists, what repeats is the declared unit: a stack fingerprint, a greedy token sequence, or a receipt. Those contracts are not interchangeable.

Energy measurement

GPU-board energy via NVML, on the stated workload. That is not wall-plug energy and not facility energy. PUE-adjusted figures are not claimed.

How to read this page. Independent TestFort results, later internal matched comparisons, and internal absolute results are three different evidence classes. Version tags such as 99, 100, and 101 are internal milestone tags, not product names. They are never mixed into one system-wide win.

Independent evidence · TestFort · July 23, 2026

Packed-prefill baseline on one H100

TestFort QA Lab measured Qwen2-7B-Instruct packed prefill, batch 16, sequence length 128, one NVIDIA H100 80GB. Work unit: prefill positions. Energy: GPU-board joules via NVML.

28,374.7

Prefill positions per second

Luxi path, TestFort median

0.018718

GPU-board J/position

Not facility energy

3.10%

Less energy than default vLLM

0.018718 vs 0.019316 J/position

9.15%

Less energy than batch-invariant vLLM

0.018718 vs 0.020604 J/position

Configuration	Throughput (positions/sec)	Board J/position	Throughput	Energy
LuxiEdge prefill build	28,374.7	0.018718	-	-
vLLM (default)	35,203.1	0.019316	Luxi 80.60% of vLLM	Luxi 3.10% lower
vLLM (batch-invariant)	30,914.3	0.020604	Luxi 91.78% of vLLM	Luxi 9.15% lower

Scope, kept exact. Packed prefill only. Lower throughput than both vLLM configurations. GPU-board energy is not facility energy. This is not a full serving or decode comparison, and not a universal vLLM win. The tested build carries the internal milestone tag Version 99.

Report: TestFort packed-prefill evaluation, July 2026 (PDF) · Evidence pack

Internal matched evidence · August 7, 2026

Internal matched prefill, batch 16

Internal, same-GPU, sequential comparison arms. Qwen2-7B-Instruct FP16, sequence length 128, batch 16, one NVIDIA H100 80GB-class GPU. Locked Flash, device-resident FP16 path. This is a different build and a later date than the TestFort baseline. The two are not combined.

~1.18x

Of the tested vLLM throughput at batch 16

~41,221 prefill positions per second on the Luxi arm

~12%

Lower board J/position than the tested vLLM at batch 16

~0.0169 J/position, NVML GPU-board energy

<1%

Throughput variation across measured runs

Stability of the comparison, not output-token identity

Batch-32 supporting detail: ~43,464 positions per second at ~0.0158 J/position, ~1.19x the tested vLLM at ~14% lower board energy. No matched batch-72 vLLM arm exists. Independent reproduction is the next step. Matched evidence pack

Internal absolute evidence · August 7, 2026

Internal absolute prefill at batch 72

Fastest measured prefill configuration to date, reported as an absolute internal result. No vLLM comparison arm. One NVIDIA H100 80GB HBM3; Qwen2-7B-Instruct-class weights; sequence length 128; batch 72; dual-GEMM; Flash attention; device-resident FP16 path; median of five 15-second runs.

~44,860 /s

Prefill positions per second

Median of five 15-second runs

~0.01532 J

Board joules per prefill position

NVML GPU-board energy

The ~1.18x batch-16 ratio is never applied to this result. The B72 throughput loop did not emit hashes. A separate Door-B receipt check passed for independent-process stack-fingerprint equality at batches 1, 16, and 32. That is not a Door-A AUDIT bit-exact B72 result.

Absolute evidence pack

Decode / generation

Reference agreement, not a serving offer

Decode produces output tokens. The GPU path matches the reference next-token result under the defined numerical audit. Serving readiness, end-to-end quality, throughput, and board energy on this path are still being locked.

Faithful Qwen2-7B CUDA path

Full 28-layer Qwen2-7B on NVIDIA H100. CPU/CUDA and Hugging Face/CUDA greedy-token match rate 1.0 under the current acceptance protocol. Logit cosine similarity approximately 1.0; top-10 overlap 10/10. Four-request independent CUDA generation smoke test passed. Token embedding row gathering and final greedy argmax remain host-side.

True-autoregressive decode

Internal prototype. Numerical audit complete as of August 6, 2026. Throughput and board-joules figures published before that date are superseded and are not current product proof.

Llama 3.1 internal milestone

Active-batch output matched the serial BF16 path on tested p32 and p128 gates. Cold process/model load about 190 to 210 seconds on the current development path. Independent validation of performance and energy is next.

Fixed-window verifier research

About 4,669 evaluated positions per second. Work unit is evaluated positions, never generated tokens. Internal research; public data pack in preparation.

Current limitations, stated directly

Not a full production serving replacement for vLLM or any other stack.
No universal vLLM win. The independent TestFort run used less board energy and less throughput.
Batch-specific prefill is not end-to-end inference.
Prefill positions are not generated tokens.
GPU-board energy is not facility energy.
Internal Version 100/101 and August 7 internal prefill results are separate from TestFort. They are labeled as such.
Decode is not a serving offer. Host-side embedding gather and argmax remain.

Full evidence register Request a prefill evaluation Luxi System architecture