Unify OS X and iOS AV Foundation Player

[ofTheo]

With the merging of #3200 we now have an AVFoundation player/grabber for OS X.
We also have a separate one for iOS.

I think the goal of AVFoundation was to have a shared API for video on both iOS and OS X, so it would be worth looking at consolidation.

It could result in more consistent behavior and less bugs to fix.

ping @julapy @admsyn @prisonerjohn @openframeworks/video @openframeworks/ios

[julapy]

yo, ive been bringing this up over the last couple months and no one has been listening! :)

ofxAVFoundationVideoPlayer runs on iOS and OSX.
https://github.com/julapy/ofxAVFoundationVideoPlayer

ive taken the ofxiOSVideoPlayer and reworked it slightly so it works exactly the same on iOS and OSX.
AVFoundation code across the two OS'es is essentially identical.

i think it might actually be safer to go with ofxAVFoundationVideoPlayer in the core as the OSX default because the original ofxiOSVideoPlayer has been tried and tested for a few years now and the code is solid. ive used it in a number of commercial AVFoundation intensive projects where single frame precision was required and its stood up to the task...

theres both iOS and OSX examples in there... give it a go.

[ofTheo]

Ahh! Okay, can we just do that then?
Sorry I missed that!!

I'm down with whatever works best.
I we would defer to you and @adamsyn on this

[prisonerjohn]

That's pretty cool! Let me know if you need help bringing it into the core, I'd be happy to help.

[admsyn]

@ofTheo Yeah I'd be deferring to @julapy in turn on this one :)

[julapy]

hi guys, I'll spend some time this weekend comparing the av player merged into the core with my implementation... will be sure to give you a shout @prisonerjohn if any questions arise, thx.

[kylemcdonald]

i'm going to push this one back to 0.9.1, meaning: we would love to have it in 0.9.0 but aren't making it a requirement. having separate osx/ios implementations for one release is not a huge issue.

[julapy]

ive been working on ofxAVFoundationVideoPlayer and i think its in a very good place now.
https://github.com/julapy/ofxAVFoundationVideoPlayer

ive been comparing it with ofAVFoundationPlayer (the current OSX video player) and making sure the functionality is the same and also borrowing some code from there to combine the best of the two video players.

@prisonerjohn @openframeworks/video would be really good if you could have a look at the code, run the examples and see if theres anything missing or that needs to change.

[julapy]

hey, just wanted to give this a little push since OF activity is ramping up for 0.9.0
have @prisonerjohn @openframeworks/video folks had a chance to look at this yet?

for now it might be an idea to replace the current iOS video player with ofxAVFoundationVideoPlayer (https://github.com/julapy/ofxAVFoundationVideoPlayer). ive been concentrating my dev efforts on the new ofxAVFoundationVideoPlayer and leaving the OF iOS player stagnant... so to take advantage of the new features and bug fixes, its probably worth doing the swap now for 0.9.0. keep in mind that ofxAVFoundationVideoPlayer stems from the OF iOS player so its essentially the same, but now also works on OSX.

@ofTheo @admsyn - you guys ok with this?

[ofTheo]

@julapy I am super happy to do this.
We're actually talking about doing a bunch of video stuff now anyway with #3543 so I think its the perfect time.

Please do a PR for it! It will be great to have a single video player for both.
Not sure if #3543 should be done first @pizthewiz @bakercp ? thoughts?

[prisonerjohn]

Sorry I haven't had a chance to dig through yet, I only took a quick glance. I agree that a unified video player for iOS and OS X is def the way to go! I could test it in my recent and current projects if it gets pulled in the main repo.

[admsyn]

👍

[julapy]

spent a couple days merging the OSX and iOS video player code (PR is here - #3590)

its working well across the two platforms in pretty much the same way.

one difference worth noting is that the current OSX video player in master uses the AVPlayerItemVideoOutput class which is only available in iOS6+ and currently ofxiOS supports iOS5 and up. so in this version, AVAssetReader is used for reading video sample data instead AVPlayerItemVideoOutput. this is ok i think, as they both do the same thing and when we bump up the minimum iOS version to iOS6+, then we can swap out AVAssetReader for AVPlayerItemVideoOutput.

the new version has a couple new features on top of the current implementation,

• video player can render to a ofTexture or display natively in a UIView or NSView.
• not only video sample data can be retrieved, but also audio sample data which can be used further down the chain for FFT analysis or mixing with other audio channels. as far as im aware, this is only possible with AVAssetReader and not AVPlayerItemVideoOutput.

@prisonerjohn @ofTheo @admsyn, let me know what you guys think.

[pizthewiz]

Should be good to close this yeah?

[kylemcdonald]

yes!