ChatTalk to the model on your chip. Every setting on the right goes with each message.
Conversation
No messages. Start the server, then send a message.
Sampling
Temperature 0.00
Top p 1.00
Thinking
Temperature 0 is greedy: the same message gives the same reply. Qwen suggests 0.7, top p 0.8 and top k 20 for sampled replies.
APIThe same server speaks the OpenAI chat format, so any client or coding agent can use it.
Call it
curl http://127.0.0.1:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "qwen3-8-27b", "messages": [{"role": "user", "content": "Hello"}], "max_tokens": 128}'