This repository was archived by the owner on Jun 5, 2025. It is now read-only.
Repository navigation
This repository was archived by the owner on Jun 5, 2025. It is now read-only.
Server-sent events fired 1000s times per conversation #936
Description
Describe the issue
Found it hard to export the logs from the server sent events, but here is a video recording of the network tab. This is for a single chat message that produced a handful of alerts.
Screen.Recording.2025-02-05.at.2.21.19.PM.mov
Steps to Reproduce
- Open a connection with
/api/v1/alerts_notification - chat with an LLM
- notice how many events are sent over the stream
Operating System
MacOS (Arm)
IDE and Version
Visual Studio Code version: 1.96.4
Extension and Version
Identifier github.copilot-chat Version 0.23.2
Provider
GitHub Copilot
Model
GPT 4o mini
Codegate version
CodeGate version: v0.1.16-5abdad6-dev
Logs
No response
Additional Context
No response
Activity
lukehinds commentedon Feb 6, 2025
@aponcedeleonch might have some insight
aponcedeleonch commentedon Feb 6, 2025
Hypothesis: What I think is going on is that we switched from doing simple inserts to doing upserts for Copilot because of how we receive messages. Before, we weren't storing everything (hence the switch), but now a single chat request can involve multiple upserts. Each time we do an upsert, we're sending an SSE. The alerts and everything else should stay the same, and they’re not actually being updated in the database—but we're still sending a bunch of SSEs.
This needs to be confirmed but if that's the case this is a Copilot specific problem. I think the table that needs upserting is
outputstable. Right now, when upserting we upsert all 3 tables (prompts,alerts,outputs). If the hypothesis is confirmed we could change the logic to only upsert theoutputstable.yrobla commentedon Feb 18, 2025
We changed this logic because with the caching, we were not recording the complete output of the message but just the initial fragment. Then the conversations in the UI didn't make sense because only the first chunk of the conversation was seen. I actually saw the problem in prompts table as well, that's why i modified.
But is true that now with multiple requests, there are multiple calls to DB. That has 2 problems :
For the SSE calls we could implement some kind of debounce, but for the DB calls we should find another solution
fix: only record db content if it is the last chunk in stream
fix: only record db content if it is the last chunk in stream (#1091)