Exceptions when using cz from ISO8859-1 Terminal #956

Activity

1. github-actions

   added

   issue-status: needs-triage

   on Jan 9, 2024
2. Lee-W commented on May 20, 2024

   

   Lee-W

   on May 20, 2024

   Member

   More actions

   Hi @keenonkites , thanks for filing this issue. I just tested with the following commands without encountering issues.

   export LANG=de_CH.ISO8859-1
   cz info

   I also tried to install 3.13.0 and change terminal encoding. Could you please check if it still happens? If so, could you please provider another way to reproduce? Thanks!
3. Lee-W

   added

   type: bug

   issue-status: wait-for-response

   and removed

   issue-status: needs-triage

   on May 20, 2024
4. keenonkites commented on May 21, 2024

   

   keenonkites

   on May 21, 2024

   Author

   More actions

   I've just tested it on my mac (where cz is installed via brew) as well as on a AlmaLinux VM (where cz is installed via pip) with the newest version 3.26.0 and it still happens on both systems.

   to be mentioned facts:

   • both systems have ISO8559-1 as system setting
   • it does throw an exception when I start a terminal with LANG=de_CH.ISO8859-1
   • it works if I start a terminal with LANG=de_CH.UTF-8
   • if does throw an exception when I start a UTF8 Terminal and change to export LANG=de_CH.ISO8859-1 before issuing cz info

   Below you see result of the third version (UTF8 Term, cz info, changing Lang, czinfo:

   [xx@yyyyy:~] $ echo $LANG
   de_CH.UTF-8
   [xx@yyyyy:~] $ cz info
   The commit contains the following structural elements, to communicate
   intent to the consumers of your library:
   
   fix: a commit of the type fix patches a bug in your codebase
   (this correlates with PATCH in semantic versioning).
   
   feat: a commit of the type feat introduces a new feature to the codebase
   (this correlates with MINOR in semantic versioning).
   
   BREAKING CHANGE: a commit that has the text BREAKING CHANGE: at the beginning of
   its optional body or footer section introduces a breaking API change
   (correlating with MAJOR in semantic versioning).
   A BREAKING CHANGE can be part of commits of any type.
   
   Others: commit types other than fix: and feat: are allowed,
   like chore:, docs:, style:, refactor:, perf:, test:, and others.
   
   We also recommend improvement for commits that improve a current
   implementation without adding a new feature or fixing a bug.
   
   Notice these types are not mandated by the conventional commits specification,
   and have no implicit effect in semantic versioning (unless they include a BREAKING CHANGE).
   
   A scope may be provided to a commit’s type, to provide additional contextual
   information and is contained within parenthesis, e.g., feat(parser): add ability to parse arrays.
   
   <type>[optional scope]: <description>
   
   [optional body]
   
   [optional footer]
   
   [xx@yyyyy:~] $ export LANG=de_CH.ISO8859-1
   [xx@yyyyy:~] $ cz info
   Traceback (most recent call last):
     File "/home/user/pb/venv_commitizen/bin/cz", line 8, in <module>
       sys.exit(main())
                ^^^^^^
     File "/home/user/pb/venv_commitizen/lib/python3.11/site-packages/commitizen/cli.py", line 607, in main
       args.func(conf, arguments)()
     File "/home/user/pb/venv_commitizen/lib/python3.11/site-packages/commitizen/commands/info.py", line 13, in __call__
       out.write(self.cz.info())
     File "/home/user/pb/venv_commitizen/lib/python3.11/site-packages/commitizen/out.py", line 13, in write
       print(value, *args)
   UnicodeEncodeError: 'latin-1' codec can't encode character '\u2019' in position 1068: ordinal not in range(256)[xx@yyyyy:~] $
   

   I can't think of anything else how to produce it as for me it's more an 'avoiding' the problem, but question from my end:
   Do you have de_CH.ISO8859-1 installed on your test system ? If the local is not installed you probably fall back automatically to UTF8 anyway.
5. keenonkites commented on May 21, 2024

   

   keenonkites

   on May 21, 2024

   Author

   More actions

   In the case of issuing the command cz info the crash is caused by the single quote on line 24 from "commit’s" in the file commitizen/cz/conventional_commits/conventional_commits_info.txt.

   Taking this out from the source code prevents cz from crashing for 'cz info' with non-utf terminals. But there are other non-ascii characters in other sections also that crashes cz with other commands (cz init, e.g., as mentioned in the original posting).

   To make the application non-utf save I think the functions for producing the output in the file commitizen/out.py have to be written in a way that allows masking/rewriting characters that are not safe for the actual encoding... or at least prevent crashes and print out proper error messages.
6. Lee-W

   added

   issue-status: needs-triage

   and removed

   issue-status: wait-for-response

   on May 21, 2024
7. Lee-W commented on May 21, 2024

   

   Lee-W

   on May 21, 2024

   Member

   More actions

   Thanks for updating! Will take a deeper look after I come back.
8. Lee-W

   self-assigned this

   on May 21, 2024
9. bearomorphism commented on May 9, 2026

   

   bearomorphism

   on May 9, 2026

   Collaborator

   More actions

   Triage from #1964: Confirmed still valid. commitizen/out.py (lines 7-9) only reconfigures stdout encoding on win32. On Linux/macOS with LANG=de_CH.ISO8859-1, print() raises UnicodeEncodeError for emojis and other non-Latin1 characters. Fix: extend the reconfigure to all platforms, or wrap print calls in an encoding-tolerant helper that falls back to escape sequences.
10. bearomorphism commented on May 9, 2026

    

    bearomorphism

    on May 9, 2026

    Collaborator

    More actions

    Verification update (re #1964)

    I simulated the failure mode against current master (4.15.1) by wrapping sys.stdout in a Latin-1 TextIOWrapper (since I'm on Windows and can't set LANG=de_CH.ISO8859-1 in the OS sense):

    • cz.info() → UnicodeEncodeError: 'latin-1' codec can't encode character '\u2019'
    • init success message — out.success("Configuration complete \U0001f680") — → UnicodeEncodeError: 'latin-1' codec can't encode character '\U0001f680'

    The relevant code, unchanged since the original report:

    # commitizen/out.py:7-9
    if sys.platform == "win32":
        if isinstance(sys.stdout, io.TextIOWrapper) and sys.version_info >= (3, 7):
            sys.stdout.reconfigure(encoding="utf-8")

    Only Windows triggers the stdout reconfigure; non-Windows users with non-UTF-8 LANG still hit the original error.

    Verdict: STILL VALID.

    Suggested fix: extend the reconfigure to all platforms when the current encoding can't represent the characters commitizen produces. Something like:

    import sys, io
    if isinstance(sys.stdout, io.TextIOWrapper):
        encoding = (sys.stdout.encoding or "").lower()
        if encoding not in ("utf-8", "utf8", "utf_8"):
            try:
                sys.stdout.reconfigure(encoding="utf-8", errors="replace")
            except (AttributeError, ValueError):
                pass

    (Using errors="replace" guards against terminals that genuinely can't render the bytes — better than crashing mid-flow.)
11. bearomorphism

    added

    issue-status: pr-created

    on May 9, 2026