Skip to content

feat: add Python 3.15 runtime - #950

Open
jhengle wants to merge 4 commits into
aws:developfrom
jhengle:python315-support
Open

jhengle wants to merge 4 commits into
aws:developfrom
jhengle:python315-support

Conversation

@jhengle

@jhengle jhengle commented Nov 3, 2026 •

Copy link
Copy Markdown

Issue #, if available:

Description of changes

feat: Add python3.15 runtime support.

Description of how you validated changes

Running manual testing locally

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jhengle
jhengle requested a review from a team as a code owner November 3, 2026 20:09
@aws-sam-tooling-bot

aws-sam-tooling-bot Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

🧪 Hands-on test: No problems observed in 6 scenarios

Commit bda0e46 · build LambdaToolingHandsOn-NoDocker:a0c4aeac-5c07-4d1f-9e07-29985b1e6b61. The plan's scenarios ran on an isolated build host; a problem is reported only if it reproduced at least twice. Passing scenarios show only that these checks behaved as expected, not that the change is correct.

Scenarios

Scenario Planned check Result Runner's note
S1 The PR's own unit tests for the changed code pass on the sandbox's default Python: decode/_locale_encoding, _is_within_… ✅ pass On Python 3.11.18 (default venv), tests/unit/test_utils.py, tests/unit/workflows/dotnet_clipackage/test_utils.py and tests/unit/workflows/python_pip/test_packager.py: 112 passed, 5 subtests passed, e…
S2 Main path: a user builds a python3.15 function with a native dependency (numpy, from the PR's requirements-numpy.txt) f… ✅ pass Head: python3.15 x86_64 build from a source dir whose path contains a space succeeded (exit 0). ARTIFACTS ['app.py', 'numpy', 'numpy-2.5.4.dist-info', 'numpy.libs', 'requirements.txt']; WHEEL tags cp…
S3 Cross-architecture build for python3.15 on arm64: the PR's cp315 and glibc 2.34 mapping must make pip select an aarch64… ✅ pass Head: arm64 cross-build on the x86_64 host succeeded (exit 0) with numpy, numpy.libs and numpy-2.5.4.dist-info. WHEEL tags were cp315-cp315-manylinux_2_27_aarch64 and cp315-cp315-manylinux_2_28_aarch…
S4 The repo's python_pip integration tests and the whole unit suite run under Python 3.15 itself, the interpreter the new … ✅ pass Under Python 3.15.2, the selected python_pip integration tests gave 12 passed, 2 skipped, exit 0. The 2 tests skipped themselves via their own 'Skip if not running on ARM64' decorator (test_python_pi…
S5 The tar path-traversal fix: a member that escapes into a sibling directory sharing the unpack dir's name prefix (pkg vs… ✅ pass Python 3.11.18. Head: the member '../pkgevil/escaped.txt' was blocked with 'ExtractError Attempted Path Traversal in Tar File' and ESCAPED_FILE_EXISTS False. Base: 'EVIL_RESULT extracted' and ESCAPED…
S6 decode() behavior across locales on default Python and 3.15: a dotnet zip symlink with a UTF-8 target (cafe with acute … ✅ pass Symlink target check: on 3.11 and 3.15, LC_ALL=C and LC_ALL=C.UTF-8 both gave LINK 'caf\xe9.txt' EXISTS True, on head and on base. LC_ALL=C PYTHONUTF8=0 crashed the same way on head and base, on both…
Runner's notes in full
  • S1: On Python 3.11.18 (default venv), tests/unit/test_utils.py, tests/unit/workflows/dotnet_clipackage/test_utils.py and tests/unit/workflows/python_pip/test_packager.py: 112 passed, 5 subtests passed, exit 0.
  • S2: Head: python3.15 x86_64 build from a source dir whose path contains a space succeeded (exit 0). ARTIFACTS ['app.py', 'numpy', 'numpy-2.5.4.dist-info', 'numpy.libs', 'requirements.txt']; WHEEL tags cp315-cp315-manylinux_2_27_x86_64 and cp315-cp315-manylinux_2_28_x86_64, both within glibc 2.34. Base: exit 1, 'BUILD_FAILED WorkflowFailedError PythonPipBuilder:Validation - Runtime python3.15 is not supported'. Python 3.15.2 was installed with uv. Whether the wheels import on the real Lambda python3.15 runtime needs cloud and was not tested.
  • S3: Head: arm64 cross-build on the x86_64 host succeeded (exit 0) with numpy, numpy.libs and numpy-2.5.4.dist-info. WHEEL tags were cp315-cp315-manylinux_2_27_aarch64 and cp315-cp315-manylinux_2_28_aarch64, with no x86_64 tag. Base: exit 1, 'Runtime python3.15 is not supported'.
  • S4: Under Python 3.15.2, the selected python_pip integration tests gave 12 passed, 2 skipped, exit 0. The 2 tests skipped themselves via their own 'Skip if not running on ARM64' decorator (test_python_pip.py:166). The full unit suite gave 921 passed, 11 subtests passed, exit 0.
  • S5: Python 3.11.18. Head: the member '../pkgevil/escaped.txt' was blocked with 'ExtractError Attempted Path Traversal in Tar File' and ESCAPED_FILE_EXISTS False. Base: 'EVIL_RESULT extracted' and ESCAPED_FILE_EXISTS True, so base's commonprefix check let the file escape into the sibling directory 'pkgevil'. On both trees the normal tar extracted to ['a/b/c.txt', 'pkg-1.0/setup.py', 'top.txt']. Exit 0 on both.
  • S6: Symlink target check: on 3.11 and 3.15, LC_ALL=C and LC_ALL=C.UTF-8 both gave LINK 'caf\xe9.txt' EXISTS True, on head and on base. LC_ALL=C PYTHONUTF8=0 crashed the same way on head and base, on both Pythons, in 2 of 2 runs: UnicodeEncodeError in stdlib zipfile while creating the regular member file 'café.txt', because the filesystem encoding is ascii. That crash happens before the symlink is processed, is not caused by this PR, and is not a finding. Latin-1 check: localedef failed, but the system en_US ISO-8859-1 locale exists and LOCALE_ENC was ISO-8859-1. Head gave DECODED 'stra\xdfe' on 3.11 and 3.15. Base gave 'stra\xdfe' on 3.11 and 'stra\ufffde' on 3.15, which confirms the PEP 686 regression that the PR fixes.

🤖 Automated hands-on test — not a review approval · model claude-opus-5.5 · Kiro kiro-cli 2.33.1 · build LambdaToolingHandsOn-NoDocker:a0c4aeac-5c07-4d1f-9e07-29985b1e6b61 · commit bda0e46

History (3)
Commit Result When (UTC) Build
bda0e46 No problems observed 2026-11-05 21:55 LambdaToolingHandsOn-NoDocker:a0c4aeac-5c07-4d1f-9e07-29985b1e6b61
1798900 Problems reproduced: decode() now uses locale.getencoding(), so UTF-8 tool output becomes U+FFFD rep…; dotnet OSUtils.unzip creates broken symlinks for non-ASCII targets under LC_ALL… 2026-11-04 19:51 LambdaToolingHandsOn-NoDocker:e893e37a-812e-412a-9a35-5d3ffde10bc7
12ac870 No problems observed 2026-11-03 20:18 LambdaToolingHandsOn-NoDocker:da1ed379-e3a1-45e5-9156-7e1ef166d9ed

@github-actions github-actions Bot added pr/external stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at. area/workflow/python_pip labels Nov 3, 2026
reedham-aws
reedham-aws previously approved these changes Nov 3, 2026

@reedham-aws reedham-aws left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the unit tests are failing, I think you'll need to check that.

- decode(): Python 3.15 enables UTF-8 mode by default (PEP 686), so
  locale.getpreferredencoding() returns "utf-8" even when the locale (and the
  output of external tools such as pip, npm and maven) uses another encoding.
  Prefer locale.getencoding() (3.11+), falling back to getpreferredencoding().
  Fixes TestDecode::test_does_not_crash_non_utf8_encoding on Windows/3.15.
- _is_within_directory(): use os.path.commonpath instead of the character-based
  (and, on 3.15, deprecated) os.path.commonprefix, which treated sibling paths
  sharing a name prefix (e.g. /tmp/ab/x vs /tmp/a) as being inside the directory.

@aws-sam-tooling-bot aws-sam-tooling-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧪 Hands-on test reproduced 2 new problems on commit 1798900. Commands and evidence are in the hands-on comment.

  1. [REGRESSION] decode() now uses locale.getencoding(), so UTF-8 tool output becomes U+FFFD replacement characters under a C/POSIX locale · medium · aws_lambda_builders/utils.py:255
  2. [REGRESSION] dotnet OSUtils.unzip creates broken symlinks for non-ASCII targets under LC_ALL=C (target decoded via the new decode() encoding) · low · aws_lambda_builders/workflows/dotnet_clipackage/utils.py:100

🤖 Automated hands-on test — not a review approval

Comment thread aws_lambda_builders/utils.py Outdated
UTF-8 mode.
"""
getencoding = getattr(locale, "getencoding", None)
return getencoding() if getencoding else locale.getpreferredencoding()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧪 Hands-on: [REGRESSION] decode() now uses locale.getencoding(), so UTF-8 tool output becomes U+FFFD replacement characters under a C/POSIX locale (reproduced 2 times in S5). Commands and evidence: hands-on comment.

…-8 mode

locale.getencoding() returns ASCII for the C/POSIX locale, where Python
already enables UTF-8 mode automatically (PEP 540), and ignores an explicit
PYTHONUTF8=1 / -X utf8. Using it unconditionally turned UTF-8 tool output into
U+FFFD in those environments (regression from the previous commit).

_locale_encoding() now:
- returns utf-8 when UTF-8 mode is explicitly requested
- otherwise returns the locale encoding, except ASCII (or unknown) locale
  encodings, which are decoded as utf-8 (a superset of ASCII)
- keeps locale.getpreferredencoding() on Python < 3.11

This matches the pre-3.15 behaviour in all these cases, while still decoding
non-UTF-8 locale output (e.g. Windows ANSI code pages) correctly on 3.15.
Regression test for OSUtils.unzip decoding UTF-8 symlink targets when the
locale encoding is ASCII (C/POSIX). Fixed by the previous commit.

@reedham-aws reedham-aws left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the changes you made are alright, but I'd rather keep this PR separate to just adding the Python 3.15 runtime. Do you think you could split some of the decoding changes out and just have a test fix in this one to make the CI pass?

Comment on lines +223 to +224
# commonpath compares whole path components; commonprefix compares characters,
# which would wrongly treat "/tmp/ab/x" as being inside "/tmp/a".

@reedham-aws reedham-aws Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need this comment especially because we don't use commonprefix anymore, so it's essentially referencing nothing.

It's fine, not worth another round of review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/workflow/python_pip pr/external stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants