Security report: request for a private disclosure channel #2303

refly-reproduction.md

Describe the bug
I would like to report a security-related issue affecting Refly. This issue tracker appears to be public, so I am withholding the technical description here. Please advise me on a private disclosure channel.

To Reproduce

Test environment and scope

• Application: Refly v1.1.0 vulnerable build (reflyai/refly-api:v1.1.0 and reflyai/refly-web:v1.1.0).

• Local URL: http://127.0.0.1:4461/login.

• Account: an ordinary customer account provisioned by the local test environment. Record the balance shown for that account before the test; do not assume a particular starting balance.

• Model: Cage Chat 128K, an operator-provisioned model configured with a 128,000-token context limit.

• Provider boundary: the isolated cage uses a local OpenAI-compatible stand-in. Its billed: true flag means that a successful simulated call is considered billable by the fixture. It is not a real provider invoice or charge.

Manual reproduction

1. Record the starting customer ledger. Sign in to the local URL with the test customer account. Open the account balance, then go to Settings → Subscription. Record Total Available Credits and the current Credit Usage Details rows so they can be compared after the run.

2. Import the four input files. In the workspace, choose New Workflow → Files → Import Files and import notes-1.txt, notes-2.txt, notes-3.txt, and notes-4.txt. Use these supplied files without shortening them; their size is needed to make a later model request exceed the configured context limit.

3. Configure the Agent. Add an Agent node and use the following prompt, including the run marker used to identify the matching stand-in log entries:

   Please read all four files I uploaded in order. After reading each one, save the key points as a note document. Then provide an overall summary. [run manual-refly-20261021]

   Attach all four imported files through @ Add tools and context → Files. Under Tools, enable Read File and Generate Document; remove the default Send Email tool if present. Select Cage Chat 128K as the model.

4. Run the Agent. Click Run in the Agent panel and inspect Output. The run should complete some model calls and file/tool work, then fail when a later request exceeds the model's 128,000-token context limit. Record the displayed run status, error, and any notes saved before the failure.

5. Check Refly's ledger for this same run. Return to Settings → Subscription and compare the available balance and Credit Usage Details with Step 1. The reported behavior is reproduced if the run failed after earlier successful model calls but Refly recorded no credit deduction for the run. If Refly charged the recorded usage, this run did not reproduce the finding; preserve the run ID and logs for diagnosis.

6. Corroborate the model-call sequence. In the local stand-in's call log, filter on the exact marker manual-refly-20261021. The observed rows supplied for this run were:

   Call | HTTP status | Stand-in billed flag | Estimated prompt tokens -- | -- | -- | -- 1 | 200 | true | 3,527 2 | 200 | true | 43,747 3 | 200 | true | 44,235 4 | 200 | true | 84,455 5 | 200 | true | 84,943 6 | 200 | true | 125,163 7 | 200 | true | 125,651 8 | 400 | false | 165,871

   The first seven requests succeeded and are marked billable by the stand-in. The eighth request was rejected with HTTP 400 after its estimated prompt size exceeded the configured 128,000-token context limit; that failed request itself is marked non-billable. These prompt counts are stand-in estimates, not provider-reported token usage.

Result and evidence boundary

The stand-in log establishes that the run reached seven successful, billable simulated model calls before a later context-length error. The customer-side ledger check in Step 5 determines whether Refly failed to charge the usage recorded for those completed calls. Use the balance and usage rows from that exact run as the application-side evidence.

This reproduction uses Refly's local vulnerable v1.1.0 cage and a provider stand-in. It does not demonstrate a real third-party provider invoice, production financial loss, or impact on another tenant. The billed flags describe the fixture's simulated provider-side billing boundary only.

• Application: Refly v1.1.0 vulnerable build (reflyai/refly-api:v1.1.0 and reflyai/refly-web:v1.1.0).
• Local URL: http://127.0.0.1:4461/login.
• Account: an ordinary customer account provisioned by the local test environment. Record the balance shown for that account before the test; do not assume a particular starting balance.
• Model: Cage Chat 128K, an operator-provisioned model configured with a 128,000-token context limit.
• Provider boundary: the isolated cage uses a local OpenAI-compatible stand-in. Its billed: true flag means that a successful simulated call is considered billable by the fixture. It is not a real provider invoice or charge.

Manual reproduction

The markdown file on the top.

Result and evidence boundary

The stand-in log establishes that the run reached seven successful, billable simulated model calls before a later context-length error. The customer-side ledger check in Step 5 determines whether Refly failed to charge the usage recorded for those completed calls. Use the balance and usage rows from that exact run as the application-side evidence.

This reproduction uses Refly's local vulnerable v1.1.0 cage and a provider stand-in. It does not demonstrate a real third-party provider invoice, production financial loss, or impact on another tenant. The billed flags describe the fixture's simulated provider-side billing boundary only.

Desktop (please complete the following information):

• OS: macOS (exact version not recorded)
• Browser: Safari (exact version not recorded)
• Version: Refly build/version not recorded

Smartphone (please complete the following information):

• Not applicable; testing was performed on desktop.

Additional context
I have prepared a proposed patch for maintainer review:

• Pull request: fix(api): bill recorded model usage after agent errors #2302
• Patch commit: tomatomptt@ae83857

Please let me know how to share the technical details and reproduction evidence privately.