Kernarc Request access

Kernarc: at least 100 tokens per second at every context depth.

Qwen3.8-27B, faster than 100 tokens per second at every context length.

Request access Watch it stream
Decode speed against context depth, one stream Kernarc: 117.8, 139.9, 154.3 and 113.0 tokens per second at 128, 8K, 32K and 64K tokens of context. Stock vLLM 0.31.0: 83.9, 83.8, 50.9 and 34.9. A datum line marks 100 tokens per second. 0 40 80 120 160 128 8K 32K 64K Context depth, tokens Decode tok/s, one stream 100 tok/s A ≥ 100 TOK/S A 83.983.850.934.9 117.8139.9154.3113.0 ×3.24 B C
View A — Decode speed vs context depth Kernarc Stock vLLM 0.31.0 Derived ratio
Dwg no.KA-001
RevA
Scale1 : 1
Contacthello@kernarc.com

Watch one stream decode.

Stock vLLM 0.31.011.93 ms/token

      
1 s

      

        
Tokens
0

        
Elapsed
0.00 s

        
Last 1 s
0 tok

      

    

    

      
Kernarc7.15 ms/token

      

      
1 s

      

        
Tokens
0

        
Elapsed
0.00 s

        
Last 1 s
0 tok

      

    

  

  
Detail B — One stream, replayed at measured speed

  
Replay at measured median TPOT. Fixed sample text, approximate token split.






  

    
Speed lives below the framework.

  


  

    
    
    
      
  1. 
        
    Draft + verify
    
        
    Speculative decoding. MTP drafts 3 tokens each step. The full model checks every one before it is accepted.
    
      
    2. 
      
  2. 
        
    Schedule
    
        
    CUDA graphs per verify width. Each step replays a captured graph, which keeps host launch time off the critical path.
    
      
    3. 
      
  3. 
        
    Attention
    
        
    Split-KV verify attention. Long contexts split across all 108 SMs, so a verify step at 32K does not wait on a few busy SMs.
    
      
    4. 
      
  4. 
        
    Linear attention
    
        
    Fused Gated DeltaNet kernels. The model's linear-attention layers decode and verify in fused kernels.
    
      
    5. 
      
  5. 
        
    GEMM
    
        
    FP8 weights. Marlin GEMMs on sm_80, checked at the row counts a verify step uses.
    
      
    6. 
      
  6. 
        
    Silicon
    
        
    One NVIDIA A100-SXM4-80GB. sm_80, 108 SMs, 2.04 TB/s HBM. No multi-GPU tricks in these figures.
    
      
    7. 
    

    
  

  
Section C–C — One decode step, top to silicon






  

    
Every figure has its conditions.

  


  

    

      Table 1 — Decode speed, one stream
      
        

          
          Stock vLLM 0.31.0 Measured 2026-10-11
          Kernarc Measured 2026-10-14
          
        

        

          Context
          ms/tokenStock tok/s
          ms/tokenKernarc tok/s
          Speed-up Derived
        

      
      
        
12811.9283.98.49117.8×1.40

        
8,19211.9383.87.15139.9×1.67

        
32,76819.6550.96.48154.3×3.03

        
65,53628.6934.98.85113.0×3.24

      
      
        
Geomean16.8359.47.68130.2×2.19

      
    

  


  

    
General notes

    
    
      
  1. Speed is decode tokens per second for one stream: 1000 ÷ median time per output token, or TPOT. TPOT is measured client side over the streamed response.
    2. 
      
  2. Context depth is the prompt length before decoding: 128, 8,192, 32,768 and 65,536 tokens. Greedy decoding, batch 1.
    3. 
      
  3. Model: Qwen3.8-27B with FP8 weights and MTP speculative decoding, 3 draft tokens per step.
    4. 
      
  4. Hardware: one NVIDIA A100-SXM4-80GB for both columns.
    5. 
      
  5. The two columns come from different sessions on the same GPU. Greedy text changes between sessions, and that moves ms/token by up to about 7 %. Speed-up is a ratio of the two columns, not a separate measurement.
    6. 
      
  6. A change ships only after a paired gate: the same requests on both builds, a bootstrap 95 % confidence interval on the timing, and an output check against the reference model.
    7.