How to enable auto-tool-choice and tool-call-parser for Gemma 3 (12B/27B)

Troubleshooting and help
1

Hello there,

We are attempting to serve Gemma 3 (google/gemma-3-12b-it) via vLLM and want to enable native OpenAI-compatible tool calling.

When sending a chat completion request that includes:

"tool_choice": "auto"

the API consistently returns the following error:

Error: status_code: 400 “auto” tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set

We have configured the following environment variables in the Koyeb service:

  • VLLM_ENABLE_AUTO_TOOL_CHOICE=true

  • VLLM_TOOL_CALL_PARSER=pythonic

After redeploying the service, the same error is still returned.

2

If you’re using our one-click deploy version of gemma-3-12b-it, the envars you need are ENABLE_AUTO_TOOL_CHOICEandTOOL_CALL_PARSER.