Skip to content

MAINT: Generate Symbol and ZapfDingbats encodings from Adobe sources - #4333

Open
plenaerts wants to merge 3 commits into
py-pdf:mainfrom
plenaerts:main
Open

plenaerts wants to merge 3 commits into
py-pdf:mainfrom
plenaerts:main

Conversation

@plenaerts

Copy link
Copy Markdown

Some files include material under an old, non-free license from
Unicode Inc. Therefore, it is not possible to ship this in main or
contrib.

The problematic license contains the text "Unicode, Inc. hereby grants the
right to freely use the information supplied in this file in the creation
of products supporting the Unicode Standard", which is non-free because it
prohibits the use of the code in products that do not support the Unicode
standard.

Unicode relicensed their files in 2004 with a DFSG license, but some
programs that incorporated the code prior to that never updated their
license text. There is information in the associated bug report about how
projects have appropriately updated the licence.

Ref Lintian tag: license-problem-old-unicode

Ref Lintian tag: license-problem-old-unicode

Assisted-By: https://inference.debian.net
@codecov

codecov Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.65%. Comparing base (0397a6f) to head (5219cde).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4333   +/-   ##
=======================================
  Coverage   98.65%   98.65%           
=======================================
  Files          59       59           
  Lines       11974    11974           
  Branches     2273     2273           
=======================================
  Hits        11813    11813           
  Misses         84       84           
  Partials       77       77           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@plenaerts plenaerts changed the title Use the current Unicode Data Files and Software License DOC: Use the current Unicode Data Files and Software License Nov 1, 2026
@stefan6419846

Copy link
Copy Markdown
Collaborator

Please provide evidence why reproducing the official license headers from https://www.unicode.org/Public/MAPPINGS/VENDORS/ADOBE/zdingbat.txt is wrong here and the Unicode-3.0 license can be used instead.

@stefan6419846 stefan6419846 added the needs-discussion The PR/issue needs more discussion before we can continue label Nov 1, 2026
@plenaerts

Copy link
Copy Markdown
Author

Thanks for your fast response. I'm checking with the Debian Free Software Guidelines team to make sure I'm sending you a correct answer.

@plenaerts

Copy link
Copy Markdown
Author

I consulted the DFSG team. You were right to push back. Thanks.

  1. https://www.unicode.org/copyright.html states in 3.c that all is licensed under unicode license v3 unless stated otherwise.
  2. https://www.unicode.org/Public/MAPPINGS/VENDORS/ADOBE/symbol.txt and https://www.unicode.org/Public/MAPPINGS/VENDORS/ADOBE/zdingbat.txt contain the old notice. So they state otherwise and the license is non-free.
  3. So those files can not be in the Debian archives in the main section, where they are now.

Claude suggested to me that a solution could be to regenerate equivalent or better mapping tables from byte to glyph name using Adobe's core-14 Symbol.afm and ZapfDingbats.afm and next from glyph name to unicode using Adobe Glyph List (glyphlist.txt) for Symbol and zapfdingbats.txt for Zapf Dingbats, from adobe-type-tools/agl-aglfn

As a result, those non-free files could be removed from pypdf.

Would you consider this a viable option? If so, I can rework this PR with the generator code and the new mapping table.

@stefan6419846

Copy link
Copy Markdown
Collaborator

Under these conditions, it seems like pypdf would never have been allowed in the regular Debian repositories. The offending patch from baa62ed, available since 6.15.0, only added the missing copyright notices, which where missing since version 2.2.0 which added these mappings: 72fcaae

Claude suggested to me that a solution could be to regenerate equivalent or better mapping tables from byte to glyph name using Adobe's core-14 Symbol.afm and ZapfDingbats.afm and next from glyph name to unicode using Adobe Glyph List (glyphlist.txt) for Symbol and zapfdingbats.txt for Zapf Dingbats, from adobe-type-tools/agl-aglfn

As a result, those non-free files could be removed from pypdf.

Would you consider this a viable option? If so, I can rework this PR with the generator code and the new mapping table.

Under the condition that the correct copyright and license notices are reproduced and there is a clean documented way on how to regenerate this data from the given sources, I see no problem.

For reference, these existing files might help:

The built-in encodings of the Symbol and ZapfDingbats fonts were copied
from Unicode's MAPPINGS/VENDORS/ADOBE files, which carry an old Unicode
license that only permits use "in the creation of products supporting
the Unicode Standard".

Generate them instead with resources/afm_to_dataclass.py from the
character codes and glyph names in Adobe's core 14 AFM files and the
Unicode values in the Adobe Glyph List, the same sources already used
for core_font_metrics.py and adobe_glyphs.py.

This changes a few mappings: the space is U+0020 instead of U+00A0,
Symbol's fraction is U+2044 FRACTION SLASH instead of U+2215 DIVISION
SLASH, and ZapfDingbats 0x80-0x8D map to the ornament brackets
U+2768-U+2775 instead of Private Use Area code points.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@plenaerts plenaerts changed the title DOC: Use the current Unicode Data Files and Software License MAINT: Generate Symbol and ZapfDingbats encodings from Adobe sources Nov 1, 2026
@plenaerts

Copy link
Copy Markdown
Author

it seems like pypdf would never have been allowed in the regular Debian repositories

Yes. I agree. Thanks for clarifying with the relevant commits. They explain why I happen to be the guy who bumps into this.

Under the condition that the correct copyright and license notices are reproduced and there is a clean documented way on how to regenerate this data from the given sources, I see no problem.

I think these conditions are met. I've used Claude Opus 5.5 to follow the patterns already in the repo to generate the needed files, tests pass fine, I had an additional specific test generated for the changes in the mapping.

Ref the commit message 5219cde:

...
This changes a few mappings: the space is U+0020 instead of U+00A0,
Symbol's fraction is U+2044 FRACTION SLASH instead of U+2215 DIVISION
SLASH, and ZapfDingbats 0x80-0x8D map to the ornament brackets
U+2768-U+2775 instead of Private Use Area code points.
...

I hope this fixes the situation.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-discussion The PR/issue needs more discussion before we can continue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants