marin · quick serve
{{ statusText }}
{{ info.model || model || 'model' }}
tp {{ info.tensor_parallel_size ?? '?' }}
max_model_len {{ info.max_model_len ?? 'auto' }}
dtype {{ info.dtype || '?' }}
chat template {{ info.has_chat_template ? 'yes' : 'no' }}
{{ m.role }}
{{ m.content }}
Cmd/Ctrl+Enter to run. Streams tokens as they arrive. In chat mode the
conversation is kept; clear it to start over.
{{ error }}
{{ output }}