HLS on Safari / IOS now not working

[danrossi]

I apologise if there is nothing you can do. I am very certain in my tests HLS has been working and suddenly is now not.

I am using the CORS proxy hack here so CORS isn't a problem. I don't need to use the CORS proxy in WebView IOS apps and even there rendering is an issue.

Is there a WebGL fix that can be applied to get HLS rendering in WebGL ? I will try the canvas renderer to see if it helps. I know there is a requirement for double drawing of the canvas to get a frame up when using drawImage but this isn't an issue with Mp4 files.

The example is here

http://dev.electroteque.org/threejs/

[danrossi]

It is drawing to canvas and rendering with the canvas renderer albeit dropping frames. Something is required with webgl.

[danrossi]

This is a beauty. I have modified a kpano demo a little with the proxied HLS stream.

It is working now on Safari OSX but not on IOS 9. Any ideas what the difference is ? I am not an expert with the webgl flags sadly or what they even do. On IOS9 it's still a black frame.

http://dev.electroteque.org/threejs/webgl.html

[makc]

this last link is black rect / sound only in safari 9.1.3 / mac os x 10.11.6

[mrdoob]

What's HLS?

[danrossi]

@makc really ? I have OSX 10.10 and Safari 10. Well that explains the recent sabotage change by Apple. Safari 10. So if we're Safari 10 we're all good.

@mrdoob HLS = Apple Streaming. ie for live streams but also VOD. Live streaming is going to become a norm with the Terradeck gear. ie http://dev.electroteque.org/video/360/hls/ultra_light_flight.m3u8

Any ideas what the differences could be there ? I tried changing some things in the first example. The second example is raw webgl functions.

so texImage2D will partially render HLS but not on IOS yet maybe because it's not Safari 10 ?

[danrossi]

Please forgive me I have reported on a few webkit bug tickets.

They've gone and sabotaged the CORS proxy hack on an IOS update. macOS also is reported to not have fixed the CORS issue in Safari either.

I have to now figure out how to work around their sabotage. I have to get that working to further test HLS working on IOS. Apple is seriously making me work up a sweat to put out their fires hahah.

If I can get OSX rendering that would be a start. I just need to figure out what is different in three.js ?

[danrossi]

I'm still trying to do reduction methods to compare what is different. Once I figure that out I may get IOS working too.

Perhaps the extra shader programs within three.js is causing the rendering problems ? I'm not across it, but trying to replicate in the raw webgl example what three.js is doing in that equirectangular example. Is there example shader code for this ?

I found some raw functions here which hopefully I can replicate fully what three.js is doing. I can't just copy the shader programs directly because some variables are set externally like world position.

https://bl.ocks.org/mbostock/5446416

[dustinkerstein]

I may have just run across this or possibly a related issue. I have h264 mp4 files located on an S3 bucket and I am loading them to a video.src as a blob.

If I upload to my S3 bucket using the Amazon web-based uploader, the videos play fine when directly loaded in Mobile Safari (iOS 10.0.1), but if I try to play them using my three.js viewer then I just get a black screen. However, if I upload to the S3 bucket with Cyberduck, they play fine directly in Mobile Safari and in three.js.

It looks like Cyberduck is setting the meta-data 'Content-Type=video/mp4' and the AWS uploader is 'Content-Type=application/octet-stream'. When I manually set this meta-data to video/mp4 everything works correctly.

Strange issue, and one that caused me a lot of head scratching. Let me know if this is the same underlying issue as this ticket, or if I should create a new one.

Thanks,
Dustin

[danrossi]

This is for HLS mate as in mpegts fragments or fragmented files not mp4. Mp4 is fine on both Safari and IOS. I checked the mimetype in cyberduck and it says the mpegts is correct.

The same stream is working for me in the raw webgl example just not on IOS of course. Your issue seems similar to mine though as in a black frame. No rendering at all.

[dustinkerstein]

@danrossi ok cool. I'll create a new ticket specifically for my issue. Thanks.

[phaseOne]

I think this is the issue: www.krpano.com/ios/bugs/ios10-webgl-video-texture-crash/
I'm currently trying to implement the same workaround for three.js, but not getting very far.

[danrossi]

@phaseOne. It is a safari issue as both a problem on OSX and IOS. Mp4 is working fine for IOS and OSX. I'll tinker with antialias .

Still can't work out the differences between the two. The raw webgl example still won't work on IOS though but getting it working on OSX is a start.

I'm wondering if I can find a working equirectangle shader program that I can drop in.

[phaseOne]

More info here: http://stackoverflow.com/questions/39123109/rendering-a-video-with-webgl-in-ios-10-beta-7-safari-shows-weird-purplish-co.

I realize that this is a browser bug, but implementing a workaround might be necessary given how long these WebGL bugs take to get fixed.