smartfilming

Exploring the possibilities of video production with smartphones

#48 Is ProRes video recording coming to the next iPhone and is it a big deal? — 30. August 2021

#48 Is ProRes video recording coming to the next iPhone and is it a big deal?

ProRes logo and iPhone12 Pro Max image: Apple.

One of the things that always surprised me about Apple’s mobile operating system iOS (and now also iPadOS) was the fact that it wasn’t able to work with Apple’s very own professional video codec ProRes. ProRes is a high-quality video codec that gives a lot of flexibility for grading in post and is easy on the hardware while editing. Years ago I purchased the original Blackmagic Design Pocket Cinema Camera which can record in ProRes and I was really looking forward to having a very compact mobile video production combo with the BMPCC (that, unlike the later BMPCC 4K/6K was actually pocketable) and an iPad running LumaFusion for editing. But no, iOS/iPadOS didn’t support ProRes on a system level so LumaFusion couldn’t either. What a bummer.

Most of us will be familiar with video codecs like H.264 (AVC) and the more recent H.265 (HEVC) but while these have now become ubiquitous “all-in-one” codecs for capturing, editing and delivery of video content, this wasn’t always so. Initially, H.264 was primarily meant to be a delivery codec for a finished edit. It was not supposed to be the common editing codec – and for good reason: The high compression rate required powerful hardware to decode the footage when editing. I can still remember how the legacy Final Cut Pro on my old Mac was struggling with H.264 footage while having no problems with other, less compressed codecs. The huge advantage of H.264 as a capturing codec however is exactly the high compression because it means that you can record in high resolution and for a long time while still having relatively small file sizes which was and still is crucial for mobile devices where storage is precious. ProRes is basically the opposite: You get huge file sizes for the same recording but it’s less taxing on the editing hardware because it’s not as heavily compressed as H.264. From a quality standpoint, it’s capturing more and better color information and is therefore more robust and flexible when you apply grading in post production.

Very recently, Marc Gurman published a Bloomberg article that claims (based on info from inside sources) that the next flagship iPhone will have the ability to capture video with the ProRes codec. This took me quite by surprise given the aforementioned fact that iOS/iPadOS doesn’t even “passively” support ProRes at this point but if it turns out to be true, this is quite a big deal – at least for a certain tribe among the mobile video creators crowd, namely the mobile filmmakers. 

I’m not sure so-called “MoJos” (mobile journalists) producing short current news reports on smartphones would necessarily have to embrace ProRes as their new capture codec since their workflow usually involves a fast turn-around without spending significant time on extensive color grading, something that ProRes is made for. The lighter compression of ProRes might also not be such a big deal for them since recent iPhones and iPads can easily handle 4K multi-track editing of H.264/H.265 encoded footage. On the other hand, the downside of ProRes, very big file sizes, might actually play a role for MoJos since iPhones don’t support the use of SD cards as exchangeable and cheap external storage. Mobile filmmakers however might see this as a game-changer for their line of work, as they usually offload and back-up their dailies externally before going back on set and also spend a significant amount of time in post with grading later on.

Sure, if you are currently shooting with an app like Filmic Pro and use their “Filmic Extreme” bitrate, ProRes bitrates might not even shock you that much but the difference to standard mobile video bitrates is quite extreme nonetheless. To be more precise, the ProRes codec is not a single standard but comes in different flavors (with increasing bitrate): ProRes Proxy, ProRes LT, ProRes 422 (the “422” indicates its chroma subsampling), ProRes 422 HQ, ProRes 4444, ProRes 4444 XQ. ProRes 422 can probably be regarded as the “standard” ProRes. If we look at target bitrates for 1080p FHD in this case, it’s 122 Mbit/s for 25fps and 245Mbit/s for 50fps. Moving on to UHD/4K things are really getting enormous with 492Mbit/s for 25fps and 983Mbit/s for 50fps. A 1-minute clip of ProRes 422 UHD 25fps footage would be 3.69GB, A 1-minute clip of ProRes 422 UHD 50fps would be 7.37GB. It’s easy to see why limited internal storage can easily and quickly become a problem here if you shoot lots of video. So I personally would definitely consider it a great option to have but not exactly a must for every job and situation. Of course I would expect ProRes also to be supported for editing within the system from then on. For more info on the ProRes codec and its bitrates, check here.

At this point the whole thing is however NOT officially confirmed by Apple but only (informed) speculation and until recently I would have heavily doubted the probability of this actually happening. But the fact that Apple totally out of the blue introduced the option to record with a PAL frame rate in the native camera app earlier this year, something that by and large only video pros really care about, gives me the confidence that Apple might actually pull this off for real, maybe in the hope of luring in well-known filmmakers that boost the iPhone’s reputation as a serious filmmaking tool. What do you guys think? Will it really happen and would it be a big deal for you?

As always, if you have questions or comments, drop them here or hit me up on the Twitter @smartfilming. If you like this article, also consider subscribing to my free Telegram channel (t.me/smartfilming) to get notified about new blog posts and receive the monthly Ten Telegram Takeaways newsletter featuring a personal selection of interesting things that happened in the world of mobile video in the last four weeks.

For an overview of all my blog posts click here.

I am investing a lot of time and work in this blog and I’m even paying to keep it ad-free for an undistracted reading experience. If you find any of the content useful, please consider making a small donation via PayPal (click on the PayPal button below). It’s very much appreciated. Thank you! 🙂

#42 Camera2 API Update 2021 – Android Pro Videography & Filmmaking — 15. April 2021

#42 Camera2 API Update 2021 – Android Pro Videography & Filmmaking

I’ve already written about Camera2 API in two previous blog posts (#6 & #10) but a couple of years have passed since and I felt like taking another look at the topic now that we’re in 2021. 

Just in case you don’t have a clue what I’m talking about here: Camera2 API is a software component of Google’s mobile operating system Android (which basically runs on every smartphone today expect Apple’s iPhones) that enables 3rd party camera apps (camera apps other than the one that’s already on your phone) to access more advanced functionality/controls of the camera, for instance the setting of a precise shutter speed value for correct exposure. Android phone makers need to implement Camera2 API into their version of Android and not all do it fully. There are four different implementation levels: “Legacy”, “Limited”, “Full” and “Level 3”. “Legacy” basically means Camera2 API hasn’t been implemented at all and the phone uses the old, way more primitive Android Camera API, “Limited” signifies that some components of the Camera2 API have been implemented but not all, “Full” and “Level 3” indicate complete implementation in terms of video-related functionality. “Level 3” only has the additional benefit for photography that you can shoot in RAW format. Android 3rd party camera apps like Filmic Pro, Protake, mcpro24fps, ProShot, Footej Camera 2 or Open Camera can only unleash their full potential if the phone has adequate Camera2 API support, Filmic Pro doesn’t even let you install the app in the first place if the phone doesn’t have proper implementation. “adequate”/”proper” can already be “Limited” for certain phones but you can only be sure with “Full” and “Level 3” devices. With some other apps like Open Camera, Camera2 API is deactivated by default and you need to go into the settings to enable it to access things like shutter speed and ISO control.

How do you know what Camera2 API support level a phone has? If you already own the phone, you can use an app like Camera2 Probe to check but if you want to consider this before buying a new phone of course this isn’t possible. Luckily, the developer of Camera2 Probe has set up a crowd sourced list (users can provide the test results via the app which are automatically entered into the list) with Camera2 API support levels of a massive amount of different Android devices, currently over 3500! The list can be accessed here and it’s great that you even get to sort the list by different parameters like the phone brand or type a device name into a search bar.

It’s important to understand that there’s a Camera2 API support level for each camera on the phone. So there could be a different one for the rear camera than for the selfie camera. The support level also doesn’t say anything about how many of the phone’s camera have been made accessible to 3rd party apps. Auxiliary ultra wide-angle or telephoto lenses have become a common standard in many of today’s phones but not all phone makers allow 3rd party camera apps to access the auxiliary camera(s). So when we talk about the Camera2 API support level of a device, most of the time we are referring to its main rear camera. 

Camera2 API was introduced with Android version 5 aka “Lollipop” in 2014 and it took phone makers a bit of time to implement it into their devices so one could roughly say that only Android devices running at least Android 6 Marshmallow are actually in the position to have proper support. In the beginning, most phone makers only provided full Camera2 API support for their high-end flagship phones but over the last years, the feature has trickled down to the mid-range segment and now even to a considerable amount of entry-level devices (Nokia and Motorola are two companies that have been good with this if you’re on a tight budget).

I actually took the time to go through the Camera2 Probe list to provide some numbers on this development. Of course these are not 100% representative since not every single Android device on the planet has been included in the list but I think 3533 entries (as of 21 March 2021) make for a solid sample size.

Phone models running Android 6

Level 3: 0

Full: 30

Limited: 18

Legacy: 444

Full/Level 3 %: 6.1

———-

Phone models running Android 7

Level 3: 82

Full: 121

Limited: 113

Legacy: 559

Full/Level 3 %: 23.2

———-

Phone models running Android 8

Level 3: 147

Full: 131

Limited: 160

Legacy: 350

Full/Level 3 %: 35.3

———-

Phone models running Android 9

Level 3: 145

Full: 163

Limited: 139

Legacy: 69

Full/Level 3 %: 59.7

———-

Phone models running Android 10

Level 3: 319

Full: 199

Limited: 169

Legacy: 50

Full/Level 3 %: 70.3

———-

Phone models running Android 11

Level 3: 72

Full: 28

Limited: 8

Legacy: 2

Full/Level 3 %: 90.9

I think it’s pretty obvious that the implementation of proper Camera2 API support in Android devices has been taking massive steps forward with each iteration of the OS and a 100% coverage on new devices is just within reach – maybe the upcoming Android 12 can already accomplish this mission?

As always, if you have questions or comments, drop them here or hit me up on the Twitter @smartfilming. If you like this article, also consider subscribing to my free Telegram channel (t.me/smartfilming) to get notified about new blog posts and receive the monthly Ten Telegram Takeaways newsletter featuring a personal selection of interesting things that happened in the world of mobile video in the last four weeks.

For an overview of all my blog posts click here.

I am investing a lot of time and work in this blog and I’m even paying to keep it ad-free for an undistracted reading experience. If you find any of the content useful, please consider making a small donation via PayPal (click on the PayPal button below). It’s very much appreciated. Thank you! 🙂