Repository navigation
I get a cryptic KeyError when I tryto use asyncpg's executemany method #807
Description
- asyncpg version: 0.24.0
- PostgreSQL version: PostgreSQL 11.9 on x86_64-pc-linux-gnu, compiled by x86_64-pc-linux-gnu-gcc (GCC) 7.4.0, 64-bit
- Python version: 3.7.11
- Platform: Windows
- Do you use pgbouncer?: No
- Did you install asyncpg with pip?: Yes
I'm trying to execute this code:
result = await conn.executemany(command=query, args=records)where query is:
INSERT INTO elexeon_time_series (subject, flow, timestamp, value, update_time, environment) VALUES ($1, $2, $3, $4, $5, $6);and records consists of the following data:
[{'environment': 'test',
'flow': 'test_flow',
'subject': 'test_subject_K4oESG2YRrnUhld',
'timestamp': Timestamp('2021-08-11 10:34:19.458810'),
'update_time': Timestamp('2021-08-11 12:34:19.458810'),
'value': 0.4},
{'environment': 'test',
'flow': 'test_flow',
'subject': 'test_subject_K4oESG2YRrnUhld',
'timestamp': Timestamp('2021-08-11 11:34:19.458810'),
'update_time': Timestamp('2021-08-11 12:34:19.458810'),
'value': 0.5}]When I try to execute this code, I get a rather cryptic traceback. can anybody explain what it means? What am I doing wrong?
test_postgres_aio.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\..\eunrg_utils\database\postgres_async.py:33: in async_insert_postgres_data
result = await conn.executemany(command=query, args=records)
C:\installs\anaconda\envs\eunrg_utils\lib\site-packages\asyncpg\connection.py:355: in executemany
return await self._executemany(command, args, timeout)
C:\installs\anaconda\envs\eunrg_utils\lib\site-packages\asyncpg\connection.py:1677: in _executemany
result, _ = await self._do_execute(query, executor, timeout)
C:\installs\anaconda\envs\eunrg_utils\lib\site-packages\asyncpg\connection.py:1711: in _do_execute
result = await executor(stmt, None)
asyncpg\protocol\protocol.pyx:254: in bind_execute_many
???
asyncpg\protocol\coreproto.pyx:945: in asyncpg.protocol.protocol.CoreProtocol._bind_execute_many_more
???
asyncpg\protocol\protocol.pyx:220: in genexpr
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E KeyError: 0
asyncpg\protocol\prepared_stmt.pyx:149: KeyError
Activity
elprans commentedon Aug 11, 2021
executemanyexpects a sequence of tuples not mappings. We ought to handle incorrect input data better, of course.salimfadhley commentedon Aug 13, 2021
Thanks, it looks like I misunderstood the argument types. Yes, I think list of dicts, list of lists, and list of tuples should all be supported, but I know what I did wrong now.
elprans commentedon Aug 13, 2021
Supporting mappings would be confusing, because the iteration is done over keys in Python. It would also be inconsistent with the handling of query arguments in Postgres/asyncpg in general as they're inherently positional.
salimfadhley commentedon Aug 18, 2021
yes, perhaps all that's needed is a clearer error message in case the user passes in the wrong type of record?
elprans commentedon Aug 18, 2021
Definitely.
Improve diagnostics of invalid executemany() input
Improve diagnostics of invalid executemany() input