Complete the Langfuse integration migration to JS SDK v5 #2283

Hi Refly team 👋 — I'm Hassieb from the Langfuse team.

I reviewed the current setup and this is more than an @langfuse/otel version bump: apps/api uses @langfuse/otel@^4.4.2, while packages/observability also carries @langfuse/langchain@^4.4.2 and the legacy langfuse@^3.30.0 stateful client.

The concrete v5 migration is:

• Bump @langfuse/otel and @langfuse/langchain to v5.4+, and remove the unscoped langfuse dependency.
• Rewrite LangfuseClientManager/TraceManager from .trace()/.span()/.generation() clients to v5 observations, exported through the existing LangfuseSpanProcessor.
• Use propagateAttributes() for user/session/tags/trace metadata so the fields reach every observation.
• Review the current shouldExportSpan predicate: in v5 it is a full override. Keep it if exporting every non-excluded span is intentional; otherwise combine the excluded-scope check with isDefaultExportSpan.
• Compile and test the custom FilteredLangfuseCallbackHandler overrides against v5, including masking, nesting, errors, flush, and shutdown.

The relevant guide is JS SDK v4→v5.

If you need support with the migration, we're happy to jump on a call or help review a PR.