smartfilming

Exploring the possibilities of video production with smartphones

#20 Closing the Window of Opportunity – Farewell to Windows Phone — 8. January 2020

#20 Closing the Window of Opportunity – Farewell to Windows Phone

My personal Windows Phone / Windows 10 Mobile device collection: Microsoft Lumia 550, Nokia Lumia 630, Nokia Lumia 920, Microsoft Lumia 950.

If you are reading this, there’s a chance you might at least have heard about a mobile operating system called Windows Phone even though you never came close to owning a device running it. But there’s also a chance you never knew such a thing existed so let me just very briefly recapitulate.

Ever since Microsoft had built its quasi-monopoly in terms of an operating system for personal desktop computers and laptops with Windows, many assumed that they would have a good chance of utilizing this might in the emerging field of increasingly potent mobile phones with more complex operating systems. But that didn’t really happen. While Microsoft did introduce the Windows CE-based „Windows Mobile“ for pocketable computing devices in the early days of the 21st century, the real revolution in this market would only happen years later with the arrival of Apple’s iPhone. In a now infamous statement during an interview with USA Today in 2007, Microsoft’s then-CEO Steve Ballmer was extremely skeptical about Apple’s first phone: „There’s no chance that the iPhone is going to get any significant market share. No chance.“ Things turned out slightly different as we all know today. While Apple with iOS and soon after Google with Android went all-in on mobile touch-screen operating systems for smartphones, Microsoft hesitated for a long time – too long as history would tell. When they finally came out with Windows Phone 7 (the „7“ picking up from earlier OS versions under the „Windows Mobile“ moniker) in 2010 they were three years late to the party and never managed to catch up again despite occasional glimpses of hope. One, maybe THE crucial factor for failure was the fact that many popular 3rd party apps were not available for Microsoft’s platform or only in versions inferior to those on Apple’s iOS and Google’s Android. The acquisition of Nokia’s mobile phone division in 2014, a tighter integration with its desktop OS Windows 10 in the form of Windows 10 Mobile in 2015 and the app porting projects “Astoria” (Android) and “Islandwood” (iOS) were last desperate attempts to turn the tide but to no avail. Satya Nadella taking over from Steve Ballmer as CEO in 2014 also meant Microsoft reevaluated its business strategy and philosophy to concentrate on platform-independent services instead of pushing their own platform. In October 2017 a Microsoft executive revealed that they would cease development of new features and hardware for the platform, in January 2019 the company announced that software support for Windows 10 Mobile will end December 10, eventually postponing the date to January 14 2020 which will be next week.

End of life, end of story?

So why would I now bother to write about Windows Phone / Windows 10 Mobile when it’s being lowered into the virtual „cybergrave“ right now? Does anyone care at all? While I do assume that many will not have exactly waited for someone to do this, I consider Microsoft’s passing mobile OS to be worth a look in retrospect because not only was it a really refreshing alternative to Apple’s iOS and Google’s Android in terms of design but it actually had quite a bit of potential as a platform for mobile content creation in my opinion – potential that was wasted or got lost along the way, that is.

Mobile photographers livin’ the Lumia dream

Nokia Lumia 1020.
Nokia Lumia 1020 in bright yellow. (Photo Credit: Kārlis Dambrāns)

The Windows Phone device that single-handedly put the platform on the map for mobile content creation was without a doubt the Nokia Lumia 1020 which launched in the summer of 2013. Sporting a six-element lens array by Carl Zeiss with a whopping 41 (!) MP sensor to go along, optical image stabilization, a full set of manual controls for photography, a dedicated shutter button and (added via software update a bit later) the ability to shoot in RAW/DNG, the 1020 was a dream come true for mobile photographers, outshining pretty much everything the iPhone or Android handsets had to offer at that time. There weren’t any advanced photo editing apps around in the Windows Phone Store but as long as your focus was on the capturing experience this didn’t really matter much.

What’s with the video?

Things weren’t quite as impressive in the videography department however. Sure, the spectacular 41MP sensor was a very early example of how a photo camera megapixel „overkill“ could be utilized to provide a quasi-lossless digital zoom for video which only needed a small fraction of the 41MP prowess when recording in 1080p resolution. But while you had a pro mode with advanced manual controls for photography at hand, there wasn’t an equivalent for video – the fact that this asymmetrical feature distribution is still the standard with 99% of smartphones today isn’t really an excuse. The common simplicity and bare-bones functionality of most native camera apps doesn’t mean you have to throw in the towel for more ambitious videography work with your phone as long as 3rd party apps fill the gap but that’s exactly the point where we touch upon a sore spot with Windows Phone.

Video mode UI of ProShot on a Lumia 550. In addition to white balance presets and manual focus, an audio level meter and shutter speed / ISO control is available.

Up until very late in the game, when ProShot added video functionality to its advanced camera app for Windows 10 Mobile in the second half of 2016 there was not a single 3rd party video camera app in the Windows Phone store that provided a robust set of manual controls like ISO and shutter speed to set a precise exposure. You couldn’t even lock the exposure for video in any of the camera apps with video functionality. It wasn’t all bad though, there were some lights in the dark. I never owned a Lumia 1020 but had a 920 for over a year as my daily driver and I was pleased to find out that the native camera app supported the use of external mics via the headphone jack which is often an indispensable feature when using the phone for professional video work.

Video mode UI of the native camera app on the Lumia 950. White balance presets, manual focus and exposure compensation are available – but no exposure lock or precise shutter speed / ISO adjustment.

Another beacon (at least for folks residing in regions which use the PAL broadcast standard) was the ability of Lumia Camera (which was a separate app from the stock camera but could be considered an advanced native camera app) to shoot video in 25fps and not only the standard 30fps that is common for the native camera app of basically every single smartphone on the planet. Unfortunately, Lumia Camera did not support the use of external mics while the stock camera app did not support shooting in 25fps so if you needed both (and many did) you were out of luck. There could have been a happy ending to this when the Windows Phone native camera and Lumia Camera were merged into one app (I think they dubbed it „Windows Camera“) for the new Windows 10 Mobile. You now had white balance presets, manual focus and basic exposure adjustment (but only EV, not precise shutter speed / ISO values, and still no exposure lock!) at hand, plus PAL frame rate support. Devices running Windows 10 Mobile (at least the Lumias) were the only smartphones able to shoot in 25fps natively without the help of 3rd party apps. Up until this very day this has not been possible on any phone running Android or iOS! But incomprehensibly, the fusion of the two camera apps dropped the support for external mics (at least that’s what I found on the Lumia 950 and 550) despite the fact that it was still working in a Windows 10 Mobile preview version I ran on a Lumia 630 before the final release apparently killed it off. As mentioned before, ProShot (which had been a photo camera only for a long time) eventually added a video mode with a lot of pro features like precise shutter speed / ISO exposure control, an audio level meter and support for external mics. But it basically did so at a time when the MS Windows 10 Mobile had already hit the iceberg.

Windows Phone = Lumia?

If you are wondering at this moment why I have been only talking about Lumia devices in the context of Windows Phone and if Lumia phones basically equal Windows Phones you are both right and wrong. Yes, there were other companies than Nokia (and later Microsoft itself): Samsung, HTC, Acer, Blu and HP were among those launching phones running Microsoft’s mobile OS but their support for Windows Phone was very sparse and short-lived so it’s pretty hard to imagine that when someone talks about a „Windows Phone“ device now they are not referring to a Lumia phone.

It’s also undisputed that it were the Lumia flagships following the pioneering work of the Nokia Lumia 1020 which gave Windows Phone its reputation with mobile camera enthusiasts: photos and video footage from devices like the Lumia 920, 930, 950 and 950 XL were able to go head-to-head with iPhones and high-end Android handsets in terms of image quality. So why did Windows Phone fail to establish itself as a viable alternative for mobile content creation with smartphones?

Minding the app gap

Yes, we’re back talking about apps again. While the hardware was competitive (at least in certain cases), the software – or to be more precise: the software eco system wasn’t really compared to the other two dominant mobile platforms. I already pointed out that there was a serious lack of advanced 3rd party camera apps for video (FilmicPro actually ran a crowd funding campaign for a Windows Phone version once, it failed miserably) but the problems were not only confined to the capturing experience, they were just as imminent for post production.

UI of Movie Maker 8.1
UI of Videoshop

While Windows Phone 8.1 can actually be considered a huge step forward for the platform in terms of having any kind of video editing app available at all (apparently up until WP 8 the OS had prevented 3rd party apps from integrating even basic video editing tasks into their apps), seeing the launch of apps like Movie Maker 8.1, Videoshop (which turned out to become the only video editor available across all three major mobile platforms), MovieJax and the frustratingly short-lived Movie Creator Beta, they were all relatively basic – none of them offered a second video track for instance. That being said, they did allow you to create and produce a simple video story by adding several clips to a timeline, trim off unwanted parts, add audio like voice-over and very basic titles. But those who had hoped that this first wave of usable video editors would mature or bring about even more advanced ones over time eventually had to admit that their optimism wasn’t justified. The problems of a thin and often quality-lacking eco system in general which got caught up in a vicious circle involving poor sales figures and lacklustre involvement of app developers had a direct impact on the special case of using Windows Phone as a platform for mobile content creation.

You can still get things done!

That didn’t prevent some daring creators from using Windows Phone devices for actual professional videography however, tapping into its strengths while working around the shortcomings. I particularly want to highlight the work of Dutch/Frisian mobile journalist Wytse Vellinga who for some time used a Lumia 930 to produce news reports for the regional broadcaster Omrop Fryslan. Here are some fine examples:

Another example is by Croatian videographer Darko Flajpan who’s working for the national broadcaster HRT and also used a Lumia 930 as a main camera for a whole documentary, here’s the trailer:

To learn something about their personal experience working with Windows Phone in a professional context, I asked them a couple of short questions for the purpose of this article.

Q1: Why did you consider working with a Windows Phone at all?

Wytse Vellinga: “Windows Phone had the best camera quality on any smartphone in those days. And the fact that it could shoot 25fps with the native camera app was extremely helpful for me as a broadcast journalist.”

Darko Flajpan: “Windows Phone (Lumia 1020 in particular) got my attention at the time because of camera capabilities. It was a photo/video beast at that moment and still respectful even today. And with a battery grip which had a tripod thread and a shutter button it was the perfect tool for early MoJos. Also, it had 25fps which was very helpful for later editing and broadcasting. I’ve switched to Lumia 930 as soon it was released and with usable 4K@25fps it was unmatched in the smartphone world. So hardware was top notch and OS was quite polished and user friendly. The price of those smartphones was also on a fair side. I still have couple of 930s…”

Q2: What were the main challenges?

Wytse Vellinga: “The main challenges were the lack of good third party apps. There was no good editor, nor was there an app for making radio reports. And as the years progressed it didn’t get better but it got worse.”

Darko Flajpan: “Main challenges were on the software side. There were just two apps for video editing and those were buggy and not user friendly, so for any editing you had to transfer footage to a laptop. Lack of any support from Google (e.g. no official YouTube app) was quite irritating. On the hardware side, Microsoft never made a tiny bit of code to allow mic input to be used by a camera – for me that was huge.”

Q3: Why did Windows Phone fail in the end in your opinion?

Wytse Vellinga: “Because of what I just said. The lack of apps and the lack of support for those that wanted to build apps. It almost looked like Microsoft did not believe in their own platform.”

Darko Flajpan: “Third party developers were focused on higher value markets (iOS, Android), and Microsoft was not making an effort to attract them. From my point of view Microsoft had no clear strategy for smartphones. Brilliant engineers from Nokia (I’ve met a couple of them) were frustrated how Microsoft treated them – they’ve created great hardware and had just frustrations and lack of support from Microsoft’s side. Now, they are working for Apple and Chinese manufacturers.”

Q4: Do you think there’s any space left between Android and iOS for a third major mobile OS?

Wytse Vellinga: “There is always room for something new. iOS and Android are not perfect so there is room for improvement. But you will have to invest a lot of time and energy in getting app builders on board.”

Darko Flajpan: “This is very complex. If someone creates anything better and more attractive to consumers it will be bought, obstructed or destroyed by the two major players in its very early stage, that is my, not optimistic opinion.”

Finally, I would also like to share a short proof-of-concept video story I created using only the low-budget Lumia 550 (cost me about 50 Euro in used condition), shooting with its native camera app and editing with Videoshop. So it was completely produced on the mobile device. Notice: The Lumia 550 is only able to shoot 720p HD (not 1080p FHD) so the video has the same resolution.

“Slide down to power off”

It’s hard to imagine a more serious focus on mobile content creators alone could have made Windows Phone / Windows 10 Mobile a viable success story but it’s frustrating nonetheless looking back at the undeniable but underdeveloped potential of the platform and the devices that came with it. Fairly recently, Microsoft announced its return to the smartphone market: The long-lasting rumors about a supposed „Surface Phone“ will apparently materialize in the form of the Surface Duo smartphone in 2020. This will however only be a comeback in terms of hardware – the device is supposed to run on Android. So what was your experience with Windows Phone? Did you even know it existed? Did you ever use it and if so did you like it? Let me know in the comments or hit me up on the Twitter @smartfilming.

#19 Stabilizing shaky video footage on your smartphone (Android/iOS/WindowsMobile) — 5. October 2019

#19 Stabilizing shaky video footage on your smartphone (Android/iOS/WindowsMobile)

The fact that nowadays pretty much everyone owns a smartphone and shoots video with it has brought a gigantic wave of shaky handheld footage along. While some folks are actually allergic to any kind of shakiness in video, I personally think that depending on the amount and context it can work just fine – but definitely not all the time and under any circumstances. So there is a need to stabilize shaky handheld footage. Now the best thing to get smooth n’ stable footage is to avoid shakiness in the first place while shooting. While there are techniques for shooting (more) stable video handheld, the most common thing would be putting the phone on a tripod (using any kind of rig or clamp for mounting it). But maybe you want to move around a bit? More and more smartphones do have internal stabilization, be it on the hardware side with OIS (optical image stabilization) or on the software side with EIS (electronic image stabilzation). Over the last years there has also been a considerable and increasingly affordable influx of (motorized) gimbals that allow smooth camera movements. But let’s be honest: Unless you’re going to a planned shoot, you probably won’t carry around a tripod or gimbal (if you have one) – as compact as they have become over time, they are still too big and clunky to just put in your pocket. So it’s likely that you will find yourself in situations where you shoot video handheld and want to smooth out some distracting jitter afterwards. While most desktop video editing software has a built-in stabilizer function these days, things don’t look quite as bright on mobile but there are still a few (good) options.

Android

Google Photos
The easiest way to stabilize a pre-recorded video clip on your mobile phone is probably to use a little known feature of an immensely popular (and completely free!) app: Google Photos. Select any video clip and open the edit panel (sliders icon in the middle), then choose „Stabilise“. When I first used it I was really surprised how well it worked! The stabilization process doesn’t alter the resolution and frame rate but you will have to live with a lower video bitrate (sample clip: 17 to 11 Mbit/s) while the audio bitrate remains the same. Google Photos is basically available for all Android devices which is great. I have however found that very ancient pre-Android 5 devices (I tested it with two devices running Android 4.4) do not have the stabilization feature baked into the app. “What about iOS?” you may ask as Google Photos is also available on the Apple Appstore. Unfortunately, just like with the ancient Androids, the stabilization feature is not available in the iOS version of the app. Maybe at some point in the future.

PowerDirector
If you are looking for a stabilization feature already built into an advanced mobile video editing app with which you can produce your final edit, then Cyberlink’s PowerDirector is currently your only choice across platforms. Select the clip in your timeline, open the editing panel by tapping the pen icon on the left side bar and choose „Stabilizer“. Unlike with Google Photos where the stabilization is basically a one-button operation, PD does give you a 0-100 slider to increase or decrease the level of correction (default value is 50). The higher the level of correction, the more the image will be cropped. PD can keep the footage’s fps as long as it is a frame rate that is supported for export within the app. That means 24, 30 and 60fps – no PAL frame rates unfortunately. Resolution on the other hand shouldn’t be a problem at all, PD supports export up to UHD/4K resolution. You also get to choose between three bitrate options (Smaller Size/Standard/Better Quality), the actual bitrate will be depending on your export resolution. In the case of the sample clip used here the bitrate of 17Mbit/s remained unaltered when using „Better Quality“ but that seems to be the maximum for projects with FHD resolution. If you use a clip recorded in a higher bitrate it will be compressed upon export. The audio bitrate is reduced (sample clip 320 to128 Kbit/s). PD is free to download with watermark and some restrictions regarding certain features – watermark-free export and the complete feature set are only available with a paid subscription. KineMaster, which I generally regard as the best video editing app on Android, is still missing this feature by the way.

Microsoft Hyperlapse Mobile
There’s a third option on Android. Microsoft Hyperlapse Mobile shouldn’t be confused with Instagram’s „Hyperlapse“ app (which is only available on iOS so far). They are actually somewhat similar in that their main purpose is to speed up and stabilize video but while Instagram’s app can only do this for footage shot „live“ within the app, Microsoft’s version allows importing pre-recorded clips. By default, the result you get will be a 4x sped up clip but if you want the original speed, you can move the speed slider to „1x“ instead. As for the resolution, Microsoft Hyperlapse only supports import of clips with up to FHD resolution and you have to activate FHD export in the settings as the default setting is HD (720p). The frame rate remains the same, the video bitrate is seriously crunched (sample clip: 17 to 8 Mbit/s), the audio bitrate is kept intact. The stabilization result isn’t as good as Google’s Photos and while the app is free, you do get a Microsoft Hyperlapse branded bumper screen. There are no in-app purchases to get rid of this so you will probably have to trim it off using another app.

iOS (iPhone/iPad)

Emulsio
As mentioned above, while Google Photos is available for iOS, the stabilization feature from the Android version is not. And neither is PowerDirector or Microsoft Hyperlapse Mobile. Also, none of iOS’s best video editing apps including the likes of Luma Fusion, KineMaster, Adobe Rush, Videoleap or Splice feature a stabilization tool at this point. The only (fairly good) option to stabilize pre-recorded video that I was able to find was an app called Emulsio. The interesting thing about Emulsio is that unlike all other apps for stabilizing mentioned here, there’s a whole bunch of controls over the stabilization process at your fingertips. Just like PD it gives you a 0-100% scale for the strength of the correction, cropping more of the frame the higher the % is. But on top of that, you get control over which axes (X,Y,XY) are corrected, you can switch rotation compensation and wobble removal on or off and even reduce rolling shutter. Emulsio does keep resolution and frame rate intact but reduces both video bitrate (sample clip 17 to 15Mbit/s) and audio bitrate (320 to 256Kbit/s). It’s free with watermark, you can get rid of the watermark by purchasing a 8.99€ pro upgrade.

Windows Mobile

Microsoft Hyperlapse Mobile
Before I wrap this up let me tell you that while Microsoft Hyperlapse Mobile isn’t available for iOS, it (still) is for their own (now quasi-dead) mobile platform Windows Mobile. So just for the highly unlikely case that you are a die-hard Windows Phone enthusiast still holding on to your Lumia: You can join the stabilization fun! It basically works like the Android version described above but only supports import/export of HD (720p) clips, higher resolution clips will be transcoded to 720p. So when the resolution is reduced it shouldn’t come as a surprise that video bitrate (17 to 7 Mbit/s) and audio bitrate (320 to 192 Kbit/s) are as well. The frame rate remains the same as the original source clip.

And here’s a video presenting the deliberately shaky sample clip (shot on a Motorola Moto Z in 1080p 30fps handheld) in stabilized versions by each mentioned app (in the case of Power Director and Emulsio the default settings were used):

Have I missed something important? Did a new app or new feature for an established app just come out? Let me know in the comments or hit me up on the Twitter (@smartfilming).