Skip to content

Add VideoFrameDecoder to support decoding video frames from any source.#689

Merged
colinrtwhite merged 8 commits into
masterfrom
colin/video_frame_decoder
Mar 4, 2021
Merged

Add VideoFrameDecoder to support decoding video frames from any source.#689
colinrtwhite merged 8 commits into
masterfrom
colin/video_frame_decoder

Conversation

@colinrtwhite
Copy link
Copy Markdown
Member

Fixes #687.

This finally adds support for decoding video frame from any source (including the network!) with an important caveat. VideoFrameDecoder creates a short lived file on disk to buffer the source into. This is less performant than decoding the file in memory and it requires enough space on disk to copy the video which could be a problem for very large video files.

VideoFrameFileFetcher and VideoFrameUriFetcher don't have this caveat which is why they're not deprecated. Long term (possibly in the next major version) we'll hopefully be able to remove this limitation on VideoFrameDecoder and deprecate the fetchers.

@colinrtwhite colinrtwhite requested a review from Jawnnypoo March 4, 2021 08:36
import coil.size.Size
import kotlin.math.roundToInt

internal class VideoFrameDecoderDelegate(private val context: Context) {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class is basically pulled out entirely from VideoFrameFetcher without changes.

size: Size,
options: Options
): DecodeResult {
val tempFile = File.createTempFile("tmp", null, context.cacheDir.apply { mkdirs() })
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need write permissions to do this, right?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep! The cacheDir is free to write to.

Copy link
Copy Markdown
Member

@Jawnnypoo Jawnnypoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Great work, I know this is much appreciated from those who need it.

@colinrtwhite colinrtwhite merged commit 18fa89a into master Mar 4, 2021
@colinrtwhite colinrtwhite deleted the colin/video_frame_decoder branch March 4, 2021 19:09
@ultraon
Copy link
Copy Markdown

ultraon commented Apr 21, 2021

It doesn't look like a workable solution in conjunction with streaming content urls (mpd - MPEG DASH, m3u8 - HLS), right?

@colinrtwhite
Copy link
Copy Markdown
Member Author

colinrtwhite commented Apr 21, 2021

@ultraon It should work with any format supported by Android as long as the stream is not live and has completed. If you need video frames while the stream is in process I'd use ExoPlayer.

@yehia2030
Copy link
Copy Markdown

yehia2030 commented Feb 8, 2022

it still didn't work with me in compose and this is my example

        val context = LocalContext.current
        Image(
            modifier = Modifier
                .fillMaxWidth()
                .height(200.dp)
                .padding(horizontal = 10.dp),
            painter = rememberAsyncImagePainter(ImageRequest.Builder(LocalContext.current)
                .data(data = "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4")
                .apply(block = {
                 decoderFactory(VideoFrameDecoder.Factory())
                }).build()),
            contentDescription = "test",
        )

or this code

        AsyncImage(
            modifier = Modifier
                .fillMaxWidth()
                .height(200.dp)
                .padding(horizontal = 10.dp),
            model = ImageRequest.Builder(context)
                .data("http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4")
                .apply {  decoderFactory(VideoFrameDecoder.Factory()) }
                .build(),
            contentDescription = null
        )

if there is any solution please let me know

@yehia2030
Copy link
Copy Markdown

yehia2030 commented Feb 8, 2022

after a lot of pain the solution is

        val imageLoader = ImageLoader.Builder(context).build()
        val model = ImageRequest.Builder(LocalContext.current)
            .data(data = "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4")
            .apply(block = {
                decoderFactory(VideoFrameDecoder.Factory())
                videoFrameMillis(50)
                listener(
                    onStart = {println("Start")},
                    onError = { _, result -> println("Error ${result.throwable.message}")},
                    onSuccess = { _, _ ->  println("Success")},
                    onCancel = { println("Cancel") }
                )
                size(100)
            }).build()
        Image(
            modifier = Modifier
                .fillMaxWidth()
                .height(200.dp)
                .padding(horizontal = 10.dp),
            painter = rememberAsyncImagePainter(model, imageLoader = imageLoader,
                onError = {
                println("Error 2 ${it.result.throwable.message}")
            }, onLoading = {
                    println("Loading 2 ")
                }, onSuccess = {
                    println("Success 2${it.result.isSampled} ")

                }),
            contentDescription = "test"
        )
        LaunchedEffect(key1 = true) {
            imageLoader.execute(model)
        }

@yehia2030
Copy link
Copy Markdown

could we make a buffer of something like that as if the video is long it takes too much time to load the image ? as i think it loaded the fully video then get its thumbnail

colinrtwhite added a commit that referenced this pull request Oct 5, 2022
…e. (#689)

* Add VideoFrameDecoder.

* Ensure directory exists.

* Updates.

* Catch IOException.

* Docs.

* Add a test.

* Update Application.

* Docs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support loading video frames from the network.

4 participants