Repository navigation
[MEDI] Establish pattern for handling LLM failures #6983
Description
Currently, a failure in one of the requests send to the LLM would tear down entire pipeline.
We should figure out a pattern for handling such failures and expose it to the end user.
The user should be allowed to:
- retry
- ignore the failure
- provide default value
Since all our enrichers accept an IChatClient client and ChatOptions? chatOptions = null, we should consider creating an option bag that would take mandatory client and let the users specify chat options and error handling strategy.
Context: #6957 (comment)
Activity
stephentoub commentedon Oct 30, 2025
This has to be addressed, and not just "failure" in terms of HTTP responses, but even minor hallucinations. With how the enrichers are currently written, if for example a classification enricher had classes "stephen" and "adam" and it sent back "steven" for just a single piece of data, the entire pipeline will fail.