Synapse 1.26.0 should depend on psycopg2>=2.8

[callahad]

@turt2live reported that his /sync broke:

[synchrotron_1] 2021-01-21 06:41:20,454 - synapse.http.server - 83 - ERROR - GET-6- Failed handle request via 'SyncRestServlet': <XForwardedForRequest at 0x7fe0a7364e10 method='GET' uri='/_matrix/client/r0/sync?filter=76&timeout=0&since=s30055085_169441467_1384_26059860_26058150_2158_574317_12846128_152' clientproto='HTTP/1.0' site=8050>
Traceback (most recent call last):
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/handlers/sync.py", line 313, in _wait_for_sync_for_user
    sync_config, since_token, full_state=full_state
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/handlers/sync.py", line 344, in current_sync_for_user
    return await self.generate_sync_result(sync_config, since_token, full_state)
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/handlers/sync.py", line 1000, in generate_sync_result
    sync_result_builder, account_data_by_room
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/handlers/sync.py", line 1439, in _generate_sync_entry_for_rooms
    await concurrently_execute(handle_room_entries, room_entries, 10)
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/util/async_helpers.py", line 174, in _concurrently_execute_inner
    await maybe_awaitable(func(next(it)))
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/handlers/sync.py", line 1434, in handle_room_entries
    always_include=sync_result_builder.full_state,
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/handlers/sync.py", line 1831, in _generate_room_entry
    newly_joined_room=newly_joined,
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/handlers/sync.py", line 452, in _load_filtered_recents
    room_id
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/state/__init__.py", line 211, in get_current_state_ids
    ret = await self.resolve_state_groups_for_events(room_id, latest_event_ids)
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/util/metrics.py", line 92, in measured_func
    r = await func(self, *args, **kwargs)
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/state/__init__.py", line 436, in resolve_state_groups_for_events
    state_res_store=StateResolutionStore(self.store),
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/state/__init__.py", line 578, in resolve_state_groups
    state_res_store=state_res_store,
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/state/__init__.py", line 637, in resolve_events_with_store
    state_res_store,
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/state/v2.py", line 101, in resolve_events_with_store
    room_id, state_sets, event_map, state_res_store
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/state/v2.py", line 339, in _get_auth_chain_difference
    room_id, state_sets_ids
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/storage/databases/main/event_federation.py", line 170, in get_auth_chain_difference
    state_sets,
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/storage/database.py", line 664, in runInteraction
    **kwargs,
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/storage/database.py", line 740, in runWithConnection
    self._db_pool.runWithConnection(inner_func, *args, **kwargs)
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/enterprise/adbapi.py", line 306, in _runWithConnection
    compat.reraise(excValue, excTraceback)
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/python/compat.py", line 464, in reraise
    raise exception.with_traceback(traceback)
  File "/home/matrix/synapse/lib/python3.6/site-packages/twisted/enterprise/adbapi.py", line 297, in _runWithConnection
    result = func(conn, *args, **kw)
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/storage/database.py", line 734, in inner_func
    return func(db_conn, *args, **kwargs)
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/storage/database.py", line 534, in new_transaction
    r = func(cursor, *args, **kwargs)
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/storage/databases/main/event_federation.py", line 327, in _get_auth_chain_difference_using_cover_index_txn
    rows = txn.execute_values(sql, args)
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/storage/database.py", line 284, in execute_values
    lambda *x: execute_values(self.txn, *x, fetch=True), sql, *args
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/storage/database.py", line 314, in _do_execute
    return func(sql, *args)
  File "/home/matrix/synapse/lib/python3.6/site-packages/synapse/storage/database.py", line 284, in <lambda>
    lambda *x: execute_values(self.txn, *x, fetch=True), sql, *args
TypeError: execute_values() got an unexpected keyword argument 'fetch'

According to the psycopg2 release notes, the fetch parameter was added to execute_values() function in version 2.8.

However, our dependencies list "postgres": ["psycopg2>=2.7"], at https://github.com/matrix-org/synapse/blob/release-v1.26.0/synapse/python_dependencies.py#L89-L90

[callahad]

This was introduced in 1315a2e

It was not caught by our CI since we do not test an old deps variant against postgresql.

[callahad]

(Our docker images are fine; it's just if you're installing from source and have an old psycopg2)

[erikjohnston]

We can also gate usage of execute_values behind a version check, and fall back to the slower SQLite query.

[callahad]

Decision is to bump the minimum psycopg2 version.

Debian 10 (buster) and Ubuntu 18.05 (bionic) do not provide a suitable version of psycopg2 in their own repositories, which may complicate installation for users who rely on Python modules which are globally installed by their distribution's package manager.

That is, however, an uncommon configuration. Users who install synapse in an isolated virtual environment, as recommended, are unaffected. Similarly, the .deb packages and Docker images produced by Matrix.org bundle all dependencies, are similarly unaffected.