Skip to content

asyncpg.connect: ValueError: invalid literal for int() with base 10 #1151

Closed

Description

@dss010101
  • asyncpg version: asyncpg==0.29.0
  • PostgreSQL version: Postgres 16 (Azure Postgres Flexible Server)
  • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
    the issue with a local PostgreSQL install?
    :
  • Python version: 3.12.3
  • Platform: Debian 12.5
  • Do you use pgbouncer?:
  • Did you install asyncpg with pip?: pip install asyncpg
  • If you built asyncpg locally, which version of Cython did you use?:
  • Can the issue be reproduced under both asyncio and
    uvloop?
    :

My connection string looks like this:

asyncpg_connect_str = f'postgres://{db_user}:{db_pwd}@{db_host}/mydb'

Im connecting as follows:

   async def read_sql_async_asyncpg(self, sql, params=None):
      t1 = timeit.default_timer()
      try:
         conn = await asyncpg.connect(self.cfg.asyncpg_connect_str)
         stmt = await conn.prepare(sql)
         columns = [a.name for a in stmt.get_attributes()]
         data = await conn.fetch(sql, *params)
         if not len(data): return None

         df = pd.DataFrame(data, columns=columns)
         self.log.debug(f'read_sql_async sql: {sql}, params:{params}')
         self.log.info(f'perf: {timeit.default_timer() - t1}')
         return df   
      except Exception as exc:
         self.log.exception(exc)         
      finally:
         await conn.close()

The error i am getting is complaining about my password (that is what its highlighting anyway)

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/jtson/.local/lib/python3.12/site-packages/asyncpg/connection.py", line 2329, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jtson/.local/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 983, in _connect
    addrs, params, config = _parse_connect_arguments(**kwargs)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jtson/.local/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 692, in _parse_connect_arguments
    addrs, params = _parse_connect_dsn_and_args(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jtson/.local/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 294, in _parse_connect_dsn_and_args
    host, port = _parse_hostlist(dsn_hostspec, port, unquote=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jtson/.local/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 229, in _parse_hostlist
    hostlist_ports.append(int(hostspec_port))
                          ^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: '<mypwd>'

My password has '!' and another non-ascii character in it in a few places - i wonder if that may be causing it or if its how i have created the connection string above?
Alternatively, the stack trace seems to indicate it wants a port - but I'm using an Azure Postgres Flexible Server - im not sure there is a port available for that...(with psycopg driver, it wasnt necessary to specify a port.)

Any help would be apricated

Activity

  1. elprans commented on May 9, 2024

    @elprans
    Member

    The DSN is a URL and so all parts must be explicitly quoted with urllib.parse.quote

  2. ranchodeluxe commented on Jun 7, 2024

    @ranchodeluxe

    came here via #1159 with the same problem, even if quoted some special characters are not allowed it seems

  3. elprans commented on Jun 7, 2024

    @elprans
    Member

    came here via #1159 with the same problem, even if quoted some special characters are not allowed it seems

    Quoted does not mean surrounded by double quotes but rather percent encoded. DSN is a URL, so you must use urllib.parse.quote to encode special characters properly. This is also prominently documented:

    image

  4. ranchodeluxe commented on Jun 7, 2024

    @ranchodeluxe

    thanks @elprans, should've known that 😉

    using urllib.parse.quote still produces the same problem and #1159 is updated to reflect that

  5. elprans commented on Jun 7, 2024

    @elprans
    Member

    Oh, right, looks like plain quote() isn't quoting enough, use urllib.parse.quote_plus(). I'll amend the docs.

  6. added a commit that references this issue on Oct 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions