# tgi 3.3.7 -> llamacpp b12323  (input read as: env)

LLAMA_ARG_HF_REPO=meta-llama/Llama-3.1-8B-Instruct
LLAMA_ARG_CTX_SIZE=8192
LLAMA_ARG_PORT=8080

## Report: 1 exact, 2 approximate, 4 no equivalent, 0 unknown

APPROXIMATE    MODEL_ID=meta-llama/Llama-3.1-8B-Instruct  ->  --hf-repo meta-llama/Llama-3.1-8B-Instruct
               tgi: Hugging Face model id or a local directory saved with save_pretrained. llamacpp:
               Hugging Face repository that contains GGUF files, optionally with a ":QUANT" suffix;
               other weight formats are not loaded.
               source: tgi@v3.3.7:launcher/src/main.rs:589
               target: llamacpp@b12323:common/arg.cpp:3062
NO EQUIVALENT  REVISION=main
               llama-server has no revision option; the GGUF file or repository tag selects the
               weights.
               source: tgi@v3.3.7:launcher/src/main.rs:594
NO EQUIVALENT  NUM_SHARD=2
               llama-server has no tensor-parallel degree; multi-GPU placement is controlled by
               --device and --split-mode.
               source: tgi@v3.3.7:launcher/src/main.rs:612
APPROXIMATE    MAX_TOTAL_TOKENS=8192  ->  --ctx-size 8192
               tgi: Per-request budget of input tokens plus max_new_tokens. llamacpp: Size of the
               prompt context of the server (0 = loaded from model); how it is shared between
               parallel slots depends on other options.
               source: tgi@v3.3.7:launcher/src/main.rs:695
               target: llamacpp@b12323:common/arg.cpp:1609
NO EQUIVALENT  MAX_CONCURRENT_REQUESTS=64
               No option of llamacpp b12323 is mapped to 'admission limit'.
               source: tgi@v3.3.7:launcher/src/main.rs:649
NO EQUIVALENT  TRUST_REMOTE_CODE=true
               llama-server never runs model-repository code, so there is nothing to switch.
               source: tgi@v3.3.7:launcher/src/main.rs:643
EXACT          PORT=8080  ->  --port 8080
               source: tgi@v3.3.7:launcher/src/main.rs:774
               target: llamacpp@b12323:common/arg.cpp:3339

## Also check

- Prometheus metrics: llamacpp b12323 has a switch for this (--metrics); tgi has none, so decide
  whether to add it.
