Skip to content

Preserve exact BIGINT values in pagination cursors - #3863

Open
Joel Town Road (hippi345) wants to merge 2 commits into
Azure:mainfrom
hippi345:fix/3856-bigint-cursor
Open

Joel Town Road (hippi345) wants to merge 2 commits into
Azure:mainfrom
hippi345:fix/3856-bigint-cursor

Conversation

@hippi345

Copy link
Copy Markdown

Why make this change?

What is this change?

  • SqlPaginationUtil.TryResolveJsonElementToScalarVariable (used by MakeCursorFromJsonElement) now resolves JSON numbers as Int64 first, then as an integral decimal when the value is outside Int64, and only then as double.
    • That keeps BIGINT cursor values exact so BaseSqlQueryStructure.ParseParamAsSystemType can parse them as Int64.
    • Ordinary ints and fractional numbers still resolve to values that parse as int/double unchanged.

How was this tested?

  • Integration Tests
  • Unit Tests

Before (unmodified TryResolveJsonElementToScalarVariable, new tests only):

Failed!  - Failed:     2, Passed:     0, Skipped:     0, Total:     2
Assert.AreEqual failed. Expected:<5123692796229827916 (System.Int64)>. Actual:<5.123692796229828E+18 (System.Double)>.
Assert.AreEqual failed. Expected:<9007199254740993 (System.Int64)>. Actual:<9007199254740992 (System.Double)>.

After the fix:

Passed!  - Failed:     0, Passed:    28, Skipped:     0, Total:    28
  (SqlPaginationUtilUnitTests)

Passed!  - Failed:     0, Passed:    57, Skipped:     0, Total:    57
  (SqlPaginationUtilUnitTests + BaseSqlQueryStructureHelperTests + SqlResponseHelpersUnitTests)

dotnet format src/Azure.DataApiBuilder.sln --verify-no-changes completed with no formatting changes.

Sample Request(s)

  • REST first page (entity with a 19-digit BIGINT PK):
GET /api/JotFormSubmission?$first=3&$select=JTF_SUB_ID
  • Follow the returned nextLink (or pass $after from that response). The cursor FieldValue stays 5123692796229827916 and page 2 returns 200 with the next rows instead of 400.
GET /api/JotFormSubmission?$first=3&$select=JTF_SUB_ID&$after=<cursor-from-page-1>
  • GraphQL equivalent: page with first / after on the same BIGINT key. MCP read_records using that after cursor follows the same path.

This change was prepared with AI assistance and reviewed by the author.

TryResolveJsonElementToScalarVariable stored every JSON number as a
double, so 19-digit primary keys lost precision and the next-page
$after token failed ParseParamAsSystemType with HTTP 400.
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The decimal fallback can round tiny nonzero fractions to zero, corrupting pagination boundaries.

1 open issue
What changed in this PR

Addresses #3856 by preserving exact BIGINT values in DAB pagination cursors.

Changes:

  • Resolves JSON numbers as Int64 first, then integral decimal, before falling back to double.
  • Adds regression tests for large integer cursors and ordinary numeric values.
File Description
src/​Service.Tests/​UnitTests/​SqlPaginationUtilUnitTests.cs Tests numeric precision and cursor-value parsing.
src/​Core/​Resolvers/​SqlPaginationUtil.cs Prioritizes exact integer types during scalar conversion.

Review effort: Balanced

Comment on lines +531 to +532
else if (element.TryGetDecimal(out decimal decimalValue) &&
decimalValue == decimal.Truncate(decimalValue))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Int64-first approach looks right for fixing #3856. Before merging, could we address the decimal fallback identified above? TryGetDecimal can successfully round 1e-30 to zero, which then passes the integrality check. Since this helper also processes ORDER BY values, that changes the pagination boundary and can cause repeated or skipped rows.

For this BIGINT-focused fix, could we keep TryGetInt64 followed by the existing double fallback, without the intermediate decimal conversion? Please also add regression cases for positive/negative tiny fractions and long.MinValue/long.MaxValue to confirm we preserve floating-point behavior while fixing BIGINT cursors.

@gmohit21 Mohit G (gmohit21) self-assigned this Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Pagination fails on a table with a 19-digit bigint primary key.

3 participants