Qwen模型使用openai_api.py部署报错,RuntimeError: probability tensor contains either inf, nan or element < 0

使用Qwen仓库的openai_api.py脚本部署时会出现错误

RuntimeError: probability tensor contains either inf, nan or element < 0

这个问题之前也是百思不得其解,经过查阅仓库与相关仓库的issues之后的到了解决方案

解决方法

predict 函数内490行左右

# 原代码
delay_token_num = max([len(x) for x in stop_words]) if stop_words_ids else 0
# 修改为
if stop_words:
    delay_token_num = max([len(x) for x in stop_words])
else:
    delay_token_num = 0

create_chat_completion 函数,也就是/v1/chat/completions路由上,398行左右

#原代码
gen_kwargs['temperature'] = request.temperature
# 改为
gen_kwargs['temperature'] = request.temperature
if gen_kwargs['temperature'] < 0.51:
    gen_kwargs['temperature'] = 0.51

相关链接

https://github.com/QwenLM/Qwen1.5/issues/145

https://github.com/QwenLM/Qwen/issues/1121