Repository navigation
Activity failure, workflow worker, and replay-safe logs lose trace correlation #1819
Description
Expected Behavior
Workflow/activity logs emitted while processing a workflow work item should keep the same trace correlation when trace context is available.
This should include:
- activity activation failure logs
- activity not-found logs
- activity execution failure logs
- workflow worker completion/failure logs
- replay-safe workflow logs emitted during live, non-replay execution
In OpenTelemetry structured logs, these entries should show the same workflow trace ID as the surrounding workflow/activity logs.
Actual Behavior
Some logs emitted by the .NET SDK workflow worker are not correlated with the workflow trace and appear with no trace ID in Aspire/OpenTelemetry structured logs.
For example, after an activity throws, the activity/user logs can have the expected trace ID, while related SDK worker or workflow logs have no assigned Trace ID.
Activity trace propagation was improved in #1795 and #1808, but some failure and orchestration-side log paths still appear to run without an ambient Activity.Current.
Steps to Reproduce the Problem
-
Create a Dapr Workflow .NET app with OpenTelemetry/Aspire structured logging enabled.
-
Add a workflow that calls an activity and handles activity failure, for example:
try
{
await context.CallActivityAsync<object>("UpdateInventoryActivity", input);
}
catch
{
var logger = context.CreateReplaySafeLogger("OrderProcessingWorkflow");
logger.LogInformation("Order {OrderId} failed due to insufficient inventory - processing refund", orderId);
}- Add an activity that throws:
public override Task<object?> RunAsync(WorkflowActivityContext context, object? input)
{
throw new InvalidOperationException("Simulated activity failure");
}-
Run the workflow and inspect Aspire/OpenTelemetry structured logs.
-
Compare the trace column for:
- activity/user logs before the exception
- SDK worker activity failure log
- replay-safe workflow log in the catch block
- workflow execution completed/failed log
Some of these logs can appear without a trace ID even though they belong to the same workflow execution.
The likely cause is that activities receive trace context through ActivityRequest.ParentTraceContext, but orchestration-side logs need to recover workflow trace context from ExecutionStartedEvent.ParentTraceContext in workflow history and set it as the ambient Activity.Current while processing the workflow turn.
Release Note
RELEASE NOTE:
Activity
WhitWaldo commentedon May 18, 2026
This is probably already fixed in #1808 , but certainly worth validating once I get an RC out.
WhitWaldo commentedon May 21, 2026
@mtaghavi2005 This should be fixed in any of the 1.18 RCs. Please give it another look and confirm!
mtaghavi2005 commentedon May 21, 2026
Thanks, I’m checking it.
mtaghavi2005 commentedon May 21, 2026
I checked it with 1.18.0-rc.4, but the issue still seems to be reproducible.
I've just updated following repo and simulated an exception inside an activity.
https://github.com/mtaghavi2005/DaprWorkflowExploration/tree/missing-tracing-id
As you can see here the failed activity in the trace has no any sign of exception. (repeated 3 times - retry policy)
Additionally in the structured logs:
So it looks like the trace/log correlation problem is still present in the latest RC (1.18.0-rc.4).
WhitWaldo commentedon May 21, 2026
@mtaghavi2005 I'll keep poking at it then. Thank you for the follow-up!
mtaghavi2005 commentedon May 25, 2026
I took another look at this and prepared PR #1829 to demonstrate one possible fix for the trace correlation issue.
The approach currently restores correlation for workflow/activity execution and replay-safe logging scenarios by propagating the workflow trace context through the worker pipeline.
With this fix, all structured logs related to activity exceptions, workflow replay logging, and workflow completion contain the correct trace ID, as shown here.
In addition, failed activities can now be marked with an error status, as shown below.
If the overall direction looks reasonable to you, I’d be happy to address any review comments, adjust the implementation details, or help refine the approach further.
WhitWaldo commentedon May 27, 2026
@mtaghavi2005 Honestly, the PR looked perfectly fine to me. It'll go in the next RC (likely today). I appreciate you taking the time to both report and respond on this issue and then actually put a PR together. It's a really big assist, so thank you for the effort.
mtaghavi2005 commentedon May 28, 2026
Thanks a lot for the review and the kind words , really appreciate it.
I’m glad the fix direction made sense. Thanks as well for the quick feedback and for getting it into the next RC so quickly.