__init__.py         :176  2026-09-06 09:49:17,722 Creating authenticated Supabase client
__init__.py         :68   2026-09-06 09:49:17,722 Exchanging API token for JWT
__init__.py         :84   2026-09-06 09:49:17,999 Successfully exchanged API token for JWT
__init__.py         :187  2026-09-06 09:49:18,604 Initialized OpenWeights client for organization: niels.warncke@gmail.com (ID: 8e10fb40-9487-40ae-9b48-f210c6f52a23)
utils.py            :74   2026-09-06 09:49:18,921 Grouping models and adapters: ['Qwen/Qwen3.8-27B']
utils.py            :113  2026-09-06 09:49:19,395 Assumed base model: Qwen/Qwen3.8-27B
Downloading (or re-using) model 'Qwen/Qwen3.8-27B' …

Fetching 32 files:   0%|          | 0/32 [00:00<?, ?it/s]
Fetching 32 files:   3%|▎         | 1/32 [00:00<00:11,  2.75it/s]
Fetching 32 files:   6%|▋         | 2/32 [00:00<00:07,  4.13it/s]
Fetching 32 files:  22%|██▏       | 7/32 [00:00<00:01, 14.27it/s]
Fetching 32 files:  28%|██▊       | 9/32 [00:19<01:10,  3.07s/it]
Fetching 32 files:  31%|███▏      | 10/32 [00:22<01:07,  3.06s/it]
Fetching 32 files:  38%|███▊      | 12/32 [00:25<00:49,  2.45s/it]
Fetching 32 files:  41%|████      | 13/32 [00:29<00:54,  2.87s/it]
Fetching 32 files:  44%|████▍     | 14/32 [00:31<00:44,  2.49s/it]
Fetching 32 files:  47%|████▋     | 15/32 [00:31<00:32,  1.93s/it]
Fetching 32 files:  50%|█████     | 16/32 [00:32<00:28,  1.79s/it]
Fetching 32 files:  53%|█████▎    | 17/32 [00:35<00:30,  2.03s/it]
Fetching 32 files:  56%|█████▋    | 18/32 [00:38<00:31,  2.27s/it]
Fetching 32 files:  59%|█████▉    | 19/32 [00:44<00:45,  3.52s/it]
Fetching 32 files:  62%|██████▎   | 20/32 [00:45<00:31,  2.64s/it]
Fetching 32 files:  69%|██████▉   | 22/32 [00:45<00:15,  1.53s/it]
Fetching 32 files:  72%|███████▏  | 23/32 [00:46<00:11,  1.25s/it]
Fetching 32 files:  75%|███████▌  | 24/32 [00:46<00:08,  1.03s/it]
Fetching 32 files:  78%|███████▊  | 25/32 [00:47<00:07,  1.04s/it]
Fetching 32 files:  81%|████████▏ | 26/32 [00:50<00:08,  1.45s/it]
Fetching 32 files:  84%|████████▍ | 27/32 [00:50<00:05,  1.16s/it]
Fetching 32 files:  88%|████████▊ | 28/32 [00:51<00:03,  1.03it/s]
Fetching 32 files:  91%|█████████ | 29/32 [00:51<00:02,  1.35it/s]
Fetching 32 files:  94%|█████████▍| 30/32 [00:53<00:02,  1.31s/it]
Fetching 32 files:  97%|█████████▋| 31/32 [00:54<00:01,  1.16s/it]
Fetching 32 files: 100%|██████████| 32/32 [00:56<00:00,  1.29s/it]
Fetching 32 files: 100%|██████████| 32/32 [00:56<00:00,  1.76s/it]
files.py            :207  2026-09-06 09:50:15,970 Downloading file: conversations:file-039938ff12e6
files.py            :210  2026-09-06 09:50:16,838 File downloaded: conversations:file-039938ff12e6 (207 bytes)
cli.py              :176  2026-09-06 09:50:16,838 Loading model with vLLM
cli.py              :177  2026-09-06 09:50:16,838 load_kwargs: {
  "model": "/workspace/hf_models/Qwen_Qwen3.8-27B",
  "enable_prefix_caching": true,
  "enable_lora": false,
  "tensor_parallel_size": 1,
  "max_num_seqs": 32,
  "gpu_memory_utilization": 0.95,
  "max_model_len": 1024
}
N GPUs =  1
INFO 09-06 09:50:17 [api_utils.py:272] non-default args: {'max_model_len': 1024, 'enable_prefix_caching': True, 'gpu_memory_utilization': 0.95, 'max_num_seqs': 32, 'disable_log_stats': True, 'model': '/workspace/hf_models/Qwen_Qwen3.8-27B'}
INFO 09-06 09:50:31 [model.py:672] Resolved architecture: Qwen3_5ForConditionalGeneration
INFO 09-06 09:50:31 [model.py:1965] Using max model len 1024
WARNING 09-06 09:50:31 [model.py:963] Model does not support mm_device_do_normalize, forcing mm_device_do_normalize = False.
INFO 09-06 09:50:32 [import_utils.py:478] DeepEP v2 requires NCCL >= 2.30.4 but found 2.29.7. deepep_v2 backend will not be available.
INFO 09-06 09:50:34 [scheduler.py:242] Chunked prefill is enabled with max_num_batched_tokens=16384.
INFO 09-06 09:50:34 [config.py:605] Mamba cache mode is set to 'align' for Qwen3_5ForConditionalGeneration by default when prefix caching is enabled
INFO 09-06 09:50:34 [kernel.py:308] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
[transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
WARNING 09-06 09:50:42 [system_utils.py:157] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
__init__.py         :176  2026-09-06 09:50:53,812 Creating authenticated Supabase client
__init__.py         :68   2026-09-06 09:50:53,812 Exchanging API token for JWT
__init__.py         :84   2026-09-06 09:50:54,090 Successfully exchanged API token for JWT
__init__.py         :187  2026-09-06 09:50:54,533 Initialized OpenWeights client for organization: niels.warncke@gmail.com (ID: 8e10fb40-9487-40ae-9b48-f210c6f52a23)
(EngineCore pid=1031) INFO 09-06 09:50:54 [core.py:122] Initializing a V1 LLM engine (v0.28.0) with config: model='/workspace/hf_models/Qwen_Qwen3.8-27B', speculative_config=None, tokenizer='/workspace/hf_models/Qwen_Qwen3.8-27B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=1024, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=/workspace/hf_models/Qwen_Qwen3.8-27B, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [16384], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 64, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']), enable_flashinfer_autotune=True, enable_cutedsl_warmup=True, enable_jit_warmup=True, enable_bf16x3_router_gemm=False, moe_backend='auto', linear_backend='auto')
(EngineCore pid=1031) INFO 09-06 09:50:55 [import_utils.py:478] DeepEP v2 requires NCCL >= 2.30.4 but found 2.29.7. deepep_v2 backend will not be available.
(EngineCore pid=1031) INFO 09-06 09:50:58 [parallel_state.py:1638] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_1e088dfef6a24b2ab61a29759d6b8b8a backend=nccl
(EngineCore pid=1031) INFO 09-06 09:50:58 [parallel_state.py:1982] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=1031) INFO 09-06 09:50:58 [topk_topp_sampler.py:62] Using FlashInfer for top-p & top-k sampling.
(EngineCore pid=1031) [transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
(EngineCore pid=1031) INFO 09-06 09:51:04 [gpu_model_runner.py:5419] Starting to load model /workspace/hf_models/Qwen_Qwen3.8-27B...
(EngineCore pid=1031) INFO 09-06 09:51:04 [cuda.py:545] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=1031) INFO 09-06 09:51:04 [mm_encoder_attention.py:375] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=1031) INFO 09-06 09:51:04 [qwen_gdn_linear_attn.py:158] Using FlashInfer GDN prefill kernel (requested=auto, head_k_dim=128).
(EngineCore pid=1031) WARNING 09-06 09:51:04 [qwen_gdn_linear_attn.py:165] FlashInfer GDN prefill is JIT-compiled; first run may take a while. Set --gdn-prefill-backend triton to skip JIT.
(EngineCore pid=1031) INFO 09-06 09:51:04 [qwen_gdn_linear_attn.py:505] GDN decode kernel: cuda
(EngineCore pid=1031) INFO 09-06 09:51:04 [cuda.py:486] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=1031) INFO 09-06 09:51:04 [flash_attn.py:866] Using FlashAttention version 3
(EngineCore pid=1031) INFO 09-06 09:51:05 [weight_utils.py:858] Filesystem type for checkpoints: FUSE. Checkpoint size: 51.75 GiB. Available RAM: 179.17 GiB.
(EngineCore pid=1031) INFO 09-06 09:51:05 [weight_utils.py:881] Auto-prefetch is disabled because the filesystem (FUSE) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.
(EngineCore pid=1031)
Loading safetensors checkpoint shards:   0% Completed | 0/18 [00:00<?, ?it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:   6% Completed | 1/18 [00:02<00:43,  2.54s/it]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  11% Completed | 2/18 [00:04<00:35,  2.20s/it]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  17% Completed | 3/18 [00:05<00:23,  1.59s/it]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  22% Completed | 4/18 [00:06<00:20,  1.46s/it]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  28% Completed | 5/18 [00:07<00:17,  1.36s/it]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  33% Completed | 6/18 [00:08<00:14,  1.17s/it]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  39% Completed | 7/18 [00:10<00:13,  1.26s/it]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  44% Completed | 8/18 [00:11<00:11,  1.19s/it]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  50% Completed | 9/18 [00:11<00:08,  1.04it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  56% Completed | 10/18 [00:12<00:07,  1.13it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  61% Completed | 11/18 [00:12<00:05,  1.36it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  67% Completed | 12/18 [00:13<00:04,  1.42it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  72% Completed | 13/18 [00:13<00:03,  1.63it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  78% Completed | 14/18 [00:14<00:02,  1.62it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  83% Completed | 15/18 [00:14<00:01,  1.80it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  89% Completed | 16/18 [00:15<00:01,  1.72it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards:  94% Completed | 17/18 [00:15<00:00,  1.87it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards: 100% Completed | 18/18 [00:16<00:00,  1.99it/s]
(EngineCore pid=1031)
Loading safetensors checkpoint shards: 100% Completed | 18/18 [00:16<00:00,  1.11it/s]
(EngineCore pid=1031)
(EngineCore pid=1031) INFO 09-06 09:51:21 [default_loader.py:430] Loading weights took 16.30 seconds
(EngineCore pid=1031) INFO 09-06 09:51:22 [gpu_model_runner.py:5515] Model loading took 51.1 GiB memory and 17.015850 seconds
(EngineCore pid=1031) INFO 09-06 09:51:22 [interface.py:911] Setting attention block size to 784 tokens to ensure that attention page size is >= mamba page size.
(EngineCore pid=1031) INFO 09-06 09:51:22 [interface.py:935] Padding mamba page size by 0.13% to ensure that mamba page size and attention page size are exactly equal.
(EngineCore pid=1031) INFO 09-06 09:51:22 [gpu_model_runner.py:6590] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
(EngineCore pid=1031) INFO 09-06 09:52:02 [backends.py:1094] Using cache directory: /root/.cache/vllm/torch_compile_cache/54c948c125/rank_0_0/backbone for vLLM's torch.compile
(EngineCore pid=1031) INFO 09-06 09:52:02 [backends.py:1155] Dynamo bytecode transform time: 8.46 s
(EngineCore pid=1031) INFO 09-06 09:52:16 [backends.py:393] Compiling a graph for compile range (1, 16384) takes 13.51 s
(EngineCore pid=1031) INFO 09-06 09:52:20 [backends.py:920] collected artifacts: 65 entries, 6 artifacts, 10723271 bytes total
(EngineCore pid=1031) INFO 09-06 09:52:20 [decorators.py:708] saved AOT compiled function to /root/.cache/vllm/torch_compile_cache/torch_aot_compile/766c9e92eb8002ef5d3ce9194578f468f5a193c33e28e32cbaf02c38aaec2361/rank_0_0/model
(EngineCore pid=1031) INFO 09-06 09:52:20 [monitor.py:53] torch.compile took 27.20 s in total
(EngineCore pid=1031) INFO 09-06 09:52:28 [monitor.py:81] Initial profiling/warmup run took 7.67 s
(EngineCore pid=1031) INFO 09-06 09:52:32 [gpu_model_runner.py:6816] Profiling CUDA graph memory: PIECEWISE=11 (largest=64), FULL=7 (largest=32)
(EngineCore pid=1031) INFO 09-06 09:52:35 [gpu_model_runner.py:6941] Estimated CUDA graph memory: 0.19 GiB total
(EngineCore pid=1031) INFO 09-06 09:52:35 [gpu_worker.py:578] Available KV cache memory: 78.23 GiB
(EngineCore pid=1031) INFO 09-06 09:52:35 [gpu_worker.py:593] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9500 is equivalent to --gpu-memory-utilization=0.9487 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9513. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
(EngineCore pid=1031) INFO 09-06 09:52:35 [kv_cache_utils.py:1869] GPU KV cache size: 209,152 tokens, Maximum concurrency for 1,024 tokens per request: 204.25x
(EngineCore pid=1031) INFO 09-06 09:52:35 [qwen_triton_warmup.py:270] Warming up Qwen Triton kernels for model_type=qwen3_5_text.
(EngineCore pid=1031) INFO 09-06 09:52:36 [kernel_warmup.py:260] Using FlashInfer autotune cache file: /root/.cache/vllm/flashinfer_autotune_cache/0.6.16.post3/90a/30be637023a94f15fe7528665772bfb6e86d0df0965c0a5800d39660cf126243/autotune_configs.json
(EngineCore pid=1031) 2026-09-06 09:52:36,528 - INFO - autotuner.py:829 - flashinfer.jit: [Autotuner]: Autotuning process starts ...
(EngineCore pid=1031) 2026-09-06 09:52:36,542 - INFO - autotuner.py:852 - flashinfer.jit: [Autotuner]: Autotuning process ends
(EngineCore pid=1031) 2026-09-06 09:52:36,549 - INFO - autotuner.py:2269 - flashinfer.jit: [Autotuner]: Saved 0 configs to /root/.cache/vllm/flashinfer_autotune_cache/0.6.16.post3/90a/30be637023a94f15fe7528665772bfb6e86d0df0965c0a5800d39660cf126243/autotune_configs.json (0 new, 0 from previous config)
(EngineCore pid=1031) INFO 09-06 09:52:36 [gpu_model_runner.py:7002] Rank 0: Torch profiler disabled for CUDA graph capture
(EngineCore pid=1031)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/11 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  27%|██▋       | 3/11 [00:00<00:00, 21.16it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  55%|█████▍    | 6/11 [00:00<00:00, 21.03it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  82%|████████▏ | 9/11 [00:00<00:00, 19.86it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 11/11 [00:00<00:00, 20.37it/s]
(EngineCore pid=1031)
Capturing CUDA graphs (decode, FULL):   0%|          | 0/7 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL):  29%|██▊       | 2/7 [00:00<00:00, 15.85it/s]
Capturing CUDA graphs (decode, FULL):  57%|█████▋    | 4/7 [00:00<00:00, 10.33it/s]
Capturing CUDA graphs (decode, FULL):  86%|████████▌ | 6/7 [00:00<00:00,  9.68it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 7/7 [00:01<00:00,  5.25it/s]
(EngineCore pid=1031) INFO 09-06 09:52:40 [gpu_model_runner.py:7048] Graph capturing finished in 4 secs, took 0.18 GiB
(EngineCore pid=1031) INFO 09-06 09:52:40 [gpu_worker.py:741] CUDA graph pool memory: 0.18 GiB (actual), 0.19 GiB (estimated), difference: 0.0 GiB (2.2%).
(EngineCore pid=1031) INFO 09-06 09:52:40 [gpu_worker.py:804] Free memory on device (138.6/139.81 GiB) on startup. Desired GPU memory utilization is (0.95, 132.82 GiB). Actual usage is 51.81 GiB for consumed memory (weights + non-torch), 2.78 GiB for peak activation, and 0.18 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=83646282548` (77.9 GiB) to fit into requested memory, or `--kv-cache-memory=89851627520` (83.68 GiB) to fully utilize gpu memory. Current kv cache memory in use is 78.23 GiB.
(EngineCore pid=1031) INFO 09-06 09:52:41 [jit_monitor.py:85] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=1031) INFO 09-06 09:52:41 [torch_utils.py:262] Reducing Torch threads from 96 to 1 for serving. Set OMP_NUM_THREADS in the external environment to override.
(EngineCore pid=1031) INFO 09-06 09:52:41 [core.py:347] init engine (profile, create kv cache, warmup model) took 79.67 s (compilation: 27.20 s)
(EngineCore pid=1031) INFO 09-06 09:52:41 [kernel.py:308] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
INFO 09-06 09:52:42 [hf.py:551] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this.
[transformers] Qwen3VL video processing does not apply the per-frame pixel cap the reference implementation (qwen-vl-utils) applies, so some videos cost far more tokens than they would there. In v5.22 the capped behavior will become the default and `cap_pixels_per_frame` will be removed. Pass `cap_pixels_per_frame=True` to adopt the reference behavior now, or `False` to keep the current behavior and silence this warning.
INFO 09-06 09:52:58 [base.py:235] Multi-modal warmup completed in 16.169s
INFO 09-06 09:53:00 [base.py:235] Readonly multi-modal warmup completed in 1.946s
cli.py              :187  2026-09-06 09:53:00,613 LLM initialized: Qwen3_5ForConditionalGeneration()(
  (visual): Qwen3_VisionTransformer()(
    (patch_embed): Qwen3_VisionPatchEmbed()(
      (proj): Conv3dLayer(in_channels=3, out_channels=1152, kernel_size=(2, 16, 16), stride=(2, 16, 16), padding=(0, 0, 0), bias=True)
    )
    (pos_embed): Embedding(2304, 1152)
    (rotary_pos_emb): RotaryEmbedding(head_size=72, rotary_dim=36, max_position_embeddings=8192, base=10000, is_neox_style=True)(
      (apply_rotary_emb): ApplyRotaryEmb(is_neox_style=True, enable_fp32_compute=False)
    )
    (merger): Qwen3_VisionPatchMerger()(
      (norm): LayerNorm((1152,), eps=1e-06, elementwise_affine=True, bias=True)
      (linear_fc1): ColumnParallelLinear(in_features=4608, output_features=4608, bias=True, tp_size=1, gather_output=False)
      (act_fn): GELU(approximate='none')
      (linear_fc2): RowParallelLinear(in_features=4608, output_features=5120, bias=True, tp_size=1, reduce_results=True)
    )
    (deepstack_merger_list): ModuleList()
    (blocks): ModuleList()(
      (0-26): 27 x Qwen3_VisionBlock()(
        (norm1): LayerNorm((1152,), eps=1e-06, elementwise_affine=True, bias=True)
        (norm2): LayerNorm((1152,), eps=1e-06, elementwise_affine=True, bias=True)
        (attn): Qwen2_5_VisionAttention()(
          (qkv): QKVParallelLinear(in_features=1152, output_features=3456, bias=True, tp_size=1, gather_output=False)
          (proj): RowParallelLinear(in_features=1152, output_features=1152, bias=True, tp_size=1, reduce_results=True)
          (attn): MMEncoderAttention()
          (apply_rotary_emb): ApplyRotaryEmb(is_neox_style=True, enable_fp32_compute=False)
        )
        (mlp): Qwen3_VisionMLP()(
          (linear_fc1): ColumnParallelLinear(in_features=1152, output_features=4304, bias=True, tp_size=1, gather_output=False)
          (linear_fc2): RowParallelLinear(in_features=4304, output_features=1152, bias=True, tp_size=1, reduce_results=True)
          (act_fn): GELU(approximate='tanh')
        )
      )
    )
  )
  (language_model): Qwen3_5ForCausalLM()(
    (model): Qwen3_5Model()(
      (embed_tokens): VocabParallelEmbedding(num_embeddings=248320, num_embeddings_per_partition=248320, embedding_dim=5120, org_vocab_size=248320, num_embeddings_padded=248320, tp_size=1)
      (layers): ModuleList()(
        (0-2, 4-6, 8-10, 12-14, 16-18, 20-22, 24-26, 28-30, 32-34, 36-38, 40-42, 44-46, 48-50, 52-54, 56-58, 60-62): 48 x Qwen3_5DecoderLayer()(
          (linear_attn): QwenGatedDeltaNetAttention()(
            (conv1d): ColumnParallelLinear(in_features=4, output_features=10240, bias=False, tp_size=1, gather_output=False)
            (in_proj_qkvz): MergedColumnParallelLinear(in_features=5120, output_features=16384, bias=False, tp_size=1, gather_output=False)
            (in_proj_ba): MergedColumnParallelLinear(in_features=5120, output_features=96, bias=False, tp_size=1, gather_output=False)
            (norm): RMSNormGated()
            (out_proj): RowParallelLinear(in_features=6144, output_features=5120, bias=False, tp_size=1, reduce_results=True)
            (chunk_gated_delta_rule): ChunkGatedDeltaRule()
          )
          (mlp): Qwen2MoeMLP()(
            (gate_up_proj): MergedColumnParallelLinear(in_features=5120, output_features=34816, bias=False, tp_size=1, gather_output=False)
            (down_proj): RowParallelLinear(in_features=17408, output_features=5120, bias=False, tp_size=1, reduce_results=True)
            (act_fn): SiluAndMul()
          )
          (input_layernorm): GemmaRMSNorm()
          (post_attention_layernorm): GemmaRMSNorm()
        )
        (3, 7, 11, 15, 19, 23, 27, 31, 35, 39, 43, 47, 51, 55, 59, 63): 16 x Qwen3_5DecoderLayer()(
          (self_attn): Qwen3NextAttention()(
            (qkv_proj): QKVParallelLinear(in_features=5120, output_features=14336, bias=False, tp_size=1, gather_output=False)
            (o_proj): RowParallelLinear(in_features=6144, output_features=5120, bias=False, tp_size=1, reduce_results=True)
            (rotary_emb): MRotaryEmbedding()(
              (apply_rotary_emb): ApplyRotaryEmb(is_neox_style=True, enable_fp32_compute=False)
            )
            (attn): Attention(head_size=256, num_heads=24, num_kv_heads=4, scale=0.0625, backend=FlashAttentionImpl)
            (q_norm): GemmaRMSNorm()
            (k_norm): GemmaRMSNorm()
          )
          (mlp): Qwen2MoeMLP()(
            (gate_up_proj): MergedColumnParallelLinear(in_features=5120, output_features=34816, bias=False, tp_size=1, gather_output=False)
            (down_proj): RowParallelLinear(in_features=17408, output_features=5120, bias=False, tp_size=1, reduce_results=True)
            (act_fn): SiluAndMul()
          )
          (input_layernorm): GemmaRMSNorm()
          (post_attention_layernorm): GemmaRMSNorm()
        )
      )
      (norm): GemmaRMSNorm()
    )
    (lm_head): ParallelLMHead(num_embeddings=248320, num_embeddings_per_partition=248320, embedding_dim=5120, org_vocab_size=248320, num_embeddings_padded=248320, tp_size=1)
    (logits_processor): LogitsProcessor(vocab_size=248320, org_vocab_size=248320, scale=1.0, logits_as_input=False)
  )
)
cli.py              :188  2026-09-06 09:53:00,621 Sampling 3 conversations
cli.py              :53   2026-09-06 09:53:00,621 Applying chat template to all conversations
cli.py              :69   2026-09-06 09:53:00,622 Generating completions through vllm

Rendering prompts:   0%|          | 0/3 [00:00<?, ?it/s]
Rendering prompts: 100%|██████████| 3/3 [00:00<00:00, 198.06it/s]

Processed prompts:   0%|          | 0/3 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s](EngineCore pid=1031) WARNING 09-06 09:53:00 [jit_monitor.py:141] CuTeDSL JIT compilation during inference: _FullyFusedDeltaRuleSm90. This causes a latency spike; consider extending warmup to cover this shape/config.

Processed prompts:  33%|███▎      | 1/3 [00:03<00:06,  3.39s/it, est. speed input: 17.70 toks/s, output: 4.13 toks/s]
Processed prompts:  67%|██████▋   | 2/3 [00:03<00:01,  1.48s/it, est. speed input: 33.14 toks/s, output: 10.20 toks/s]
Processed prompts: 100%|██████████| 3/3 [00:03<00:00,  1.48s/it, est. speed input: 48.30 toks/s, output: 17.94 toks/s]
Processed prompts: 100%|██████████| 3/3 [00:03<00:00,  1.21s/it, est. speed input: 48.30 toks/s, output: 17.94 toks/s]
cli.py              :208  2026-09-06 09:53:04,262 Sampled 3 answers (counting each prompt once)
cli.py              :219  2026-09-06 09:53:04,262 Uploading /tmp/output.jsonl to OpenWeights
files.py            :156  2026-09-06 09:53:04,501 Uploading file: /tmp/output.jsonl (purpose: result, size: 581 bytes)
files.py            :193  2026-09-06 09:53:05,131 File uploaded successfully: result:file-b3baa37126a4
cli.py              :223  2026-09-06 09:53:05,131 Logging file result:file-b3baa37126a4
run.py              :29   2026-09-06 09:53:05,131 Initializing existing run: 70686
INFO 09-06 09:53:05 [utils.py:615] [shutdown] Process manager: send sigterm to process EngineCore
(EngineCore pid=1031) INFO 09-06 09:53:05 [core.py:1329] [shutdown] EngineCore: trigger received signal=SIGTERM
(EngineCore pid=1031) INFO 09-06 09:53:05 [core.py:1465] [shutdown] EngineCore: start mode=abort timeout=0s
(EngineCore pid=1031) INFO 09-06 09:53:05 [core.py:1496] [shutdown] EngineCore: request processing complete; starting resource teardown
(EngineCore pid=1031) INFO 09-06 09:53:05 [core.py:1342] [shutdown] EngineCore: exiting busy loop
