Generate an mp3 file for any audio attachments for browser-compatible rendering

[nicopace]

Current Buggy or Suspicious Behavior

Steps to Reproduce

1. load explorer
2. find an m4a file
3. will not load, neither show length

Expected Behavior

for audio to play

[nicopace]

In the context of visualizing KoboToolbox submissions attachments in GC Explorer, In the case of having audio files attached, we need to be able to see the audio files, whether they are being looked through a Chrome browser or a Firefox browser or a Safari browser.

Each of these browsers have different compatibility issues with different audio formats, so there's a need to support multiple formats.

Based on https://developer.mozilla.org/en-US/docs/Web/Media/Guides/Formats/Audio_codecs:

For general music playback, the three most likely candidates are MP3, AAC, and Vorbis.

• AAC in an MP4 container is supported by all major browsers, which makes this a great choice.
• Vorbis is almost always used in Ogg files, but Ogg containers are not universally supported. Even Microsoft Edge, which supports both Vorbis, does not yet support Ogg containers.
• MP3 (MPEG-1 Audio Layer III) is supported by all major browsers. These files are MPEG-1 files that contain an Audio Layer III track.

[rudokemper]

I just tried to play an m4a audio on Chrome and had no issue.

But indeed we may have to solve this problem in the gc-scripts-hub KoboToolbox script, by converting all of the possible audio files to a standard format that will play well in all browsers. We do something similar for GeoTIFFs to be able to load them on the Alerts Dashboard here.

[rudokemper]

I think this is a good idea to pursue upstream. We can take a similar approach as the GeoTIFFs in the alerts_gcs script (e.g. save the original, but also create a browser-friendly format file).

Based on some preliminary research, MP3 is the most widely supported format.

I guess this should apply to all of our fetch scripts that deal with attachments - KoboToolbox, ODK, CoMapeo. So we can create a module for conversion and call it for attachments in all of these scripts.

[IamJeffG]

Just a couple observations:

The GC stack more and more requires files that are "implementation detail" for the stack: caches, format normalization... These are duplicative, and maybe a good rule of thumb is: "Does this need to go into the exit plan dump?" There might actually be a few tiers of this, for example a middle ground is derived data that does add value in the dump; but this issue is not that. How do we want to manage "implementation-detail-tier" data in the warehouse?

I don't know about browser support for FLAC, but please do consider FLAC instead of MP3:

1. FLAC is an open & free format, which is more aligned with our philosophy (update: MP3's patents are expired now so maybe this is not such a big deal)
2. Consider what these browser downloads are used for. If it's 100% to be heard by a human, MP3 is fine. If it might ever get input into a machine learning signal classifier (for example) then please consider FLAC, which is lossless.

[nicopace]

I agree with @IamJeffG 100% on the format.
Something to note is that files from KoboToolbox come from highly compressed audio formats... so turning them into FLAC would only make them bigger.

I would keep the original file, and then turn it to whatever standarized format works for most browsers.

[rudokemper]

The GC stack more and more requires files that are "implementation detail" for the stack: caches, format normalization... These are duplicative, and maybe a good rule of thumb is: "Does this need to go into the exit plan dump?" There might actually be a few tiers of this, for example a middle ground is derived data that does add value in the dump; but this issue is not that. How do we want to manage "implementation-detail-tier" data in the warehouse?

Yeah, interesting. In this repo, common_logic/ is growing quite a lot recently! I think we've actually been pretty good about avoiding being duplicative (or making an effort to avoid DRY the second we notice a duplication pattern), but it does raise a question of where the proposed code change for something like this issue should go.

For something like "Convert a GeoTIFF to a JPG", we added that as an implementation detail for the "Download and Transform Alerts from GCS" script way back, because we assumed no other code would need it. But for something like this, which seems to just be an implementation detail for KoboToolbox right now, we might benefit from already thinking ahead of where we store this code. At least, that's what I'm getting out of your comment, which is also touching on a somewhat different concern (how do we transform data for everyday GC usage vs. how do we do it for exit plan).

[IamJeffG]

@rudokemper

• Actually I was referring to the stored data, not the code.
• And one more thought is you could reformat at request time, i.e. on the webapp view from the server. Doubling storage of multimedia, especially "just in case we might need to access it from a browser" could be an unnecessary cloud expense.

@nicopace I am deeply skeptical that a FLAC of a compressed format would be bigger, but I accept your point that if it was previously lossy compressed then there's no point in preferring a lossless compression algo at this point.

[rudokemper]

Ahh, I understand your observations now @IamJeffG. Yes, interesting. I think we can benefit from deriving an explicit set of rules / questions about when we do file transformations, e.g.

1. Is the derived file needed by one of our front ends, and it is not feasible to generate it on demand client-side?
2. Is the derived file useful to store and make accessible via FileBrowser for the user to download as needed? (And it is not feasible to build frontend UI to generate on demand?)

If yes to either, then generate during ETL.

(And additionally, we do store the original file on principle, to keep a high fidelity original for purposes of data provenance / permanent safekeeping).

[nicopace]

2. And it is not feasible to build frontend UI to generate on demand?)

If yes to either, then generate during ETL.

There is also the possibility to call a windmill script over HTTP, and run it synchronously, so the result of the execution will be returned.

https://www.windmill.dev/docs/core_concepts/http_routing

We could have the audio converted on the fly and made available for that execution... and be removed later if needed.

That way we can have a cache of failes only for facilitating the access to them, and be cleared once they are not being accessed.

That way we only keep a copy of the original file, while getting the benefits of having them available in other formats.