专门为98k做的网站,杭州网站制作公司排名,湖南网络推广公司,微信商城怎么开店最近在做大模型对话相关功能#xff0c;需要将对话内容流式返回给前端页面#xff08;类似GPT的效果#xff09;。下面直接说下如何实现#xff1a;
1.首先导入fastapi和sse流式返回所需要的包
from fastapi import APIRouter, Response, status
from sse_starlette.sse …最近在做大模型对话相关功能需要将对话内容流式返回给前端页面类似GPT的效果。下面直接说下如何实现
1.首先导入fastapi和sse流式返回所需要的包
from fastapi import APIRouter, Response, status
from sse_starlette.sse import EventSourceResponse
2.用EventSourceResponse来调用生成方法在对话方法用yield推送对话内容
api_router.post(/stream-chat)
def stream_chat(ask_form: ChatParam, response: Response):# 设置响应头部信息response.headers[Content-Type] text/event-streamresponse.headers[Cache-Control] no-cachereturn EventSourceResponse(stream_generate_text(ask_form, streamTrue))
def stream_generate_text(ask_form: ChatParam, stream: bool False):for res in model.chat(queryask_form.prompt, streamstream, historyshistory, temperaturetemperature):yield json.dumps({answer: res},ensure_asciiFalse,)
3.返回效果
