Use the right config section when starting the module by babolivier · Pull Request #62 · matrix-org/synapse-s3-storage-provider

babolivier · 2021-10-12T10:10:00Z

No description provided.

squahtx

lgtm

erikjohnston · 2021-10-12T10:12:09Z

s3_storage_provider.py


    def __init__(self, hs, config):
-        self.cache_directory = hs.config.media_store_path
+        self.cache_directory = hs.config.media.media_store_path


I think we might be able to delete this entirely, its only used in store_file as part of os.path.join, but the path store_file gets appears to be absolute already, so its pointless. I think.

store_file seems to be called by Synapse

store_file's path appears to be relative to me, or am I mistaken?
https://github.com/matrix-org/synapse/blob/develop/synapse/rest/media/v1/media_storage.py#L159

Oh right, yes, I misread that fname is the absolute path. Annoying.

patrickstump · 2021-10-20T12:50:13Z

@babolivier , I know this is after the fact, but is there value to adding a version check since this popped up in v1.45.0? That is, if someone is running < v1.45.0 would it be required to use the previous call of self.cache_directory = hs.config.media_store_path ?

babolivier · 2021-10-20T13:04:59Z

@babolivier , I know this is after the fact, but is there value to adding a version check since this popped up in v1.45.0? That is, if someone is running < v1.45.0 would it be required to use the previous call of self.cache_directory = hs.config.media_store_path ?

No, with Synapse < v1.45.0, reading hs.config.media.media_store_path still works.

anoadragon453 · 2021-10-20T15:51:34Z

@patrickstump For context we had a magic method which translated hs.config.media_store_path to hs.config.media.media_store_path. The method has now been removed, as it was a noticeable performance loss for minor developer convenience. But using the latter always worked 🙂

richvdh · 2021-10-21T09:09:55Z

To be clear, is this fixing something that makes s3-storage-provider not work with newer versions of synapse? If so, what were the symptoms of the failure? Some sort of exception?

babolivier · 2021-10-21T09:26:39Z

To be clear, is this fixing something that makes s3-storage-provider not work with newer versions of synapse? If so, what were the symptoms of the failure? Some sort of exception?

Yes, because media storage providers apparently get passed the Synapse config directly, and v1.45.0 removed the magic that translated hs.config.media_store_path into hs.config.media.media_store_path. This module was reading from hs.config.media_store_path, which raised an AttributeError and would prevent either "just" the media repository workers or the whole homeserver (if not using workers) from starting.

We should have caught this in matrix-org/synapse#10985 (comment), but forgot about the particular case of media storage providers.

richvdh · 2021-10-21T10:49:24Z

ok, thanks Brendan.

As a reminder, it's very helpful to include an example of the error message, so that other people seeing the same error can search for it and find the fix.

babolivier · 2021-10-21T11:39:47Z

Ah good point. Here's the full stacktrace:

Traceback (most recent call last):
  File "/home/synapse/src/synapse/app/_base.py", line 196, in wrapper
    await cb(*args, **kwargs)
  File "/home/synapse/src/synapse/app/_base.py", line 387, in start
    hs.start_listening()
  File "/home/synapse/src/synapse/app/generic_worker.py", line 389, in start_listening
    self._listen_http(listener)
  File "/home/synapse/src/synapse/app/generic_worker.py", line 325, in _listen_http
    media_repo = self.get_media_repository_resource()
  File "/home/synapse/src/synapse/server.py", line 185, in _get
    dep = builder(self)
  File "/home/synapse/src/synapse/server.py", line 615, in get_media_repository_resource
    return MediaRepositoryResource(self)
  File "/home/synapse/src/synapse/rest/media/v1/media_repository.py", line 986, in __init__
    media_repo = hs.get_media_repository()
  File "/home/synapse/src/synapse/server.py", line 185, in _get
    dep = builder(self)
  File "/home/synapse/src/synapse/server.py", line 619, in get_media_repository
    return MediaRepository(self)
  File "/home/synapse/src/synapse/rest/media/v1/media_repository.py", line 108, in __init__
    backend = clz(hs, provider_config)
  File "/home/synapse/synapse-s3-storage-provider/s3_storage_provider.py", line 63, in __init__
    self.cache_directory = hs.config.media_store_path
AttributeError: 'HomeServerConfig' object has no attribute 'media_store_path'

HarHarLinks · 2021-10-21T12:28:34Z

@babolivier , I know this is after the fact, but is there value to adding a version check since this popped up in v1.45.0? That is, if someone is running < v1.45.0 would it be required to use the previous call of self.cache_directory = hs.config.media_store_path ?

No, with Synapse < v1.45.0, reading hs.config.media.media_store_path still works.

I upgraded synapse-s3-storage-provider together with synapse 1.45.1 using pip install --force-reinstall, but had to downgrade back to 1.43 (matrix-org/synapse#11049 (comment)). Now can't upload media, error log:

2021-10-21 12:17:51,904 - synapse.http.server - 93 - ERROR - POST-7488- Failed handle request via 'UploadResource': <XForwardedForRequest at 0x7f3bfbc62e20 method='POST' uri='/_matrix/media/r0/upload' clientproto='HTTP/1.0' site='8008'>
Traceback (most recent call last):
  File "site-packages/synapse/http/server.py", line 258, in _async_render_wrapper
    callback_return = await self._async_render(request)
  File "site-packages/synapse/http/server.py", line 286, in _async_render
    callback_return = await raw_callback_return
  File "site-packages/synapse/rest/media/v1/upload_resource.py", line 93, in _async_render_POST
    content_uri = await self.media_repo.create_content(
  File "site-packages/synapse/rest/media/v1/media_repository.py", line 169, in create_content
    fname = await self.media_storage.store_file(content, file_info)
  File "site-packages/synapse/rest/media/v1/media_storage.py", line 81, in store_file
    await self.write_to_file(source, f)
  File "site-packages/synapse/rest/media/v1/media_storage.py", line 88, in write_to_file
    await defer_to_thread(self.reactor, _write_file_synchronously, source, output)
  File "site-packages/twisted/python/threadpool.py", line 238, in inContext
    result = inContext.theWork()  # type: ignore[attr-defined]
  File "site-packages/twisted/python/threadpool.py", line 254, in <lambda>
    inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
  File "site-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "site-packages/twisted/python/context.py", line 83, in callWithContext
    return func(*args, **kw)
  File "site-packages/synapse/logging/context.py", line 902, in g
    return f(*args, **kwargs)
  File "site-packages/synapse/rest/media/v1/media_storage.py", line 298, in _write_file_synchronously
    source.seek(0)  # Ensure we read from the start of the file
ValueError: seek of closed file

So apparently this is not backwards compatible?

Also synapse.rest.media.v1._base - 256 - WARNING - GET-40201- Failed to write to consumer: <class 'Exception'> Consumer asked us to stop producing happens.

callahad · 2021-10-21T14:43:45Z

@HarHarLinks Can you please open a new issue, also if you could add the output of pip freeze and more log context around the error?

Use the right config section

103e98b

babolivier requested a review from a team October 12, 2021 10:10

squahtx approved these changes Oct 12, 2021

View reviewed changes

erikjohnston approved these changes Oct 12, 2021

View reviewed changes

babolivier merged commit fd1e123 into main Oct 12, 2021

babolivier deleted the babolivier/fix_config branch October 12, 2021 10:14

HarHarLinks mentioned this pull request Oct 21, 2021

ValueError: seek of closed file when client aborts media upload at exactly the right time matrix-org/synapse#11154

Closed

Conversation

babolivier commented Oct 12, 2021

Uh oh!

squahtx left a comment

Choose a reason for hiding this comment

Uh oh!

erikjohnston Oct 12, 2021

Choose a reason for hiding this comment

Uh oh!

babolivier Oct 12, 2021

Choose a reason for hiding this comment

Uh oh!

squahtx Oct 12, 2021

Choose a reason for hiding this comment

Uh oh!

erikjohnston Oct 12, 2021

Choose a reason for hiding this comment

Uh oh!

patrickstump commented Oct 20, 2021

Uh oh!

babolivier commented Oct 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anoadragon453 commented Oct 20, 2021

Uh oh!

richvdh commented Oct 21, 2021

Uh oh!

babolivier commented Oct 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

richvdh commented Oct 21, 2021

Uh oh!

babolivier commented Oct 21, 2021

Uh oh!

HarHarLinks commented Oct 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

callahad commented Oct 21, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

babolivier commented Oct 20, 2021 •

edited

Loading

babolivier commented Oct 21, 2021 •

edited

Loading

HarHarLinks commented Oct 21, 2021 •

edited

Loading