smartfilming

Exploring the possibilities of video production with smartphones

#27 No, you don’t need a second video track for storytelling! (… and why it really doesn’t matter that much anymore) — 25. June 2020

#27 No, you don’t need a second video track for storytelling! (… and why it really doesn’t matter that much anymore)

As I pointed out in one of my very first blog posts here (in German), smartphone videography still comes with a whole bunch of limitations (although some of them are slowly but surely going away or have at least been mitigated). Yet one central aspect of the fascinating philosophy behind phoneography (that’s the term I now prefer for referring to content creation with smartphones in general) has always been one of “can do” instead of “can’t do” despite the shortcomings. The spirit of overcoming obvious obstacles, going the extra mile to get something done, trailblazing new forms of storytelling despite not having all the bells and whistles of a whole multi-device or multi-person production environment seems to be a key factor. With this in mind I always found it a bit irritating and slightly “treacherous” to this philosophy when people proclaimed that video editing apps without the ability to have a second video track in the editing timeline are not suitable for storytelling. “YOU HAVE TO HAVE A VIDEO EDITOR WITH AT LEAST TWO VIDEO TRACKS!” Bam! If you are just starting out creating your first videos you might easily be discouraged if you hear such a statement from a seasoned video producer. Now let me just make one thing clear before digging a little deeper: I’m not saying having two (or multiple) video tracks in a video editing app as opposed to just one isn’t useful. It most definitely is. It enables you to do things you can’t or can’t easily do otherwise. However, and I can’t stress this enough, it is by no means a prerequisite for phoneography storytelling – in my very humble opinion, that is. 

I can see why someone would support the idea of having two video tracks as being a must for creating certain types of videography work. For instance it could be based on the traditional concept of a news report or documentary featuring one or more persons talking (most often as part of an interview) and you don’t want to have the person talking occupying the frame all the time but still keep the statement going. This can help in many ways: On a very basic level, it can work as a means for visual variety to reduce the amount of “talking heads” air time. It might also help to cover up some unwanted visual distraction like when another person stops to look at the interviewee or the camera. But it can also exemplify something that the person is talking about, creating a meaningful connection. If you are interviewing the director of a theater piece who talks about the upcoming premiere you could insert a short clip showing the theater building from the outside, a clip of a poster announcing the premiere or a clip of actors playing a scene during the rehearsal while the director is still talking. The way you do it is by adding the so-called “b-roll” clip as a layer to the primary clip in the timeline of the editing app (usually muting the audio of the b-roll or at least reducing the volume). Without a second video track it can be difficult or even impossible to pull off this mix of video from one clip with the audio from another. But let’s stop here for a moment: Is this really the ONLY legitimate way to tell a story? Sure, as I just pointed out, it does have merit and can be a helpful tool – but I strongly believe that it’s also possible to tell a good story without this “trick” – and therefore without the need for a second video track. Here are some ideas:

WYSIWYH Style

Most of us have probably come across the strange acronym WYSIWYG: “What you see is what you get” – it’s a concept from computational UI design where it means that the preview you are getting in a (text/website/CMS) editor will very much resemble the way things actually look after creating/publishing. If you want a word to appear bold in your text and it’s bold after marking it in the editor, this is WYSIWYG. If you have to punch in code like <b>bold</b>  into your text editing interface to make the published end result bold, that’s not WYSIWYG. So I dare to steal this bizarre acronym in a slightly altered version and context: WYSIWYH – “What you see is what you hear” – meaning that your video clips always have the original sound. So in the case of an interview like described before, using a video editing app with only one video track, you would either present the interview in one piece (if it’s not very long) or cut it into smaller chunks with “b-roll” footage in between rather than overlaid (if you don’t want the questions included). Sure, it will look or feel a bit different, not “traditional”, but is that bad? Can’t it still be a good video story? One fairly technical problem we might encounter here is getting smooth audio transitions between clips when the audio levels of the two clips are very different. Video editing apps usually don’t have audio-only cross-fades (WHY is that, I ask!) and a cross-fade involving both audio AND video might not be the preferred transition of choice as most of the time you want to use a plain cut. There are ways to work around this however or just accept it as a stylistic choice for this way of storytelling. 

One-Shot Method

Another very interesting way that results in a much easier edit without the need for a second video track (if any at all) but includes more pre-planning in advance for a shoot is the one-shot approach. In contrast to what many one-man-band video journalists do (using a tripod with a static camera), this means you need to be an active camera operator at the same time to catch different visual aspects of the scene. This probably also calls for some sort of stabilization solution like phone-internal OIS/EIS, a rig, a gimbal or at least a steady hand and some practice. Journalist Kai Rüsberg has been an advocate of this style and collected some good tips here (blog post is in German but Google Translate should help you getting the gist). As a matter of fact, there’s even a small selection of noticeable feature films created in such a (risky) manner, among them “Russian Ark” (2002) and “Viktoria” (2015). One other thing we need to take into consideration is that if there’s any kind of asking questions involved, the interviewer’s voice will be “on air” so the audio should be good enough for this as well. I personally think that this style can be (if done right!) quite fascinating and more visually immersive than an edited package with static separate shots but it poses some challenges and might not be suited for everybody and every job/situation. Still, doing something like that might just expand your storytelling capabilities by trying something different. A one-track video editing app will suffice to add some text, titles, narration, fade in/out etc.

Shediting

A unique almagam of a traditional multi-clip approach and the one-shot method is a technique I called “shediting” in an earlier blog post. This involves a certain feature that is present in many native and some 3rd party camera apps: By pausing the recording instead of stopping it in between shots, you can cram a whole bunch of different shots into a single clip. Just like with one-shot, this can save you lots of time in the edit (sometimes things need to go really fast!) but requires more elaborate planning and comes with a certain risk. It also usually means that everything needs to be filmed within a very compact time frame and one location/area because in most cases you can’t close the app or have the phone go to sleep without actually stopping the recording. Nonetheless, I find this to be an extremely underrated and widely unknown “hack” to piece together a package on the go! Do yourself a favor and try to tell a short video story that way!

Voice-Over

A way to tackle rough audio transitions (or bad/challenging sound in general) while also creating a sense of continuity between clips is to use a voice-over narration in post production, most mobile editors offer this option directly within the app and even if you happen to come across one that doesn’t (or like Videoshop, hides it behind a paywall) you can easily record a voice-over in a separate audio recording app and import the audio to your video editor although it’s a bit more of a hassle if you need to redo it when the timing isn’t quite right. One example could be splicing your interview into several clips in the timeline and add “b-roll” footage with a voice-over in between. Of course you should see to it that the voice-over is somewhat meaningful and not just redundant information or is giving away the gist / key argument of an upcoming statement of the interviewee. You could however build/rephrase an actual question into the voice-over. Instead of having the original question “What challenges did you experience during the rehearsal process?” in the footage, you record a voice-over saying “During the rehearsal process director XY faced several challenges both on and off the stage…” for the insert clip followed by the director’s answer to the question. It might also help in such a situation to let the voice-over already begin at the end of the previous clip and flow into the subsequent one to cover up an obvious change in the ambient sound of the different clips. Of course, depending on the footage, the story and situation, this might not always work perfectly.

Text/Titles

Finally, with more and more media content being consumed muted on smartphones “on the go” in public, one can also think about having text and titles as an important narrative tool, particularly if there’s no interview involved (of course a subtitled interview would also be just fine!). This only works however if your editing app has an adequate title tool, nothing too fancy but at least covering the basics like control over fonts, size, position, color etc. (looking at you, iMovie for iOS!). Unlike adding a second video track, titles don’t tax the processor very much so even ultra-budget phones will be able to handle it.

Now, do you still remember the second part of this article’s title, the one in parentheses? I have just gone into lengths to explain why I think it’s not always necessary to use a video editing app with at least two video tracks to create a video story with your phone, so why would I now be saying that after all it doesn’t really matter that much anymore? Well, if you look back a whole bunch of years (say around 2013/2014) when the phoneography movement really started to gather momentum, the idea of having two video tracks in a video editing app was not only a theoretical question for app developers, thinking about how advanced they WANTED their app to be. It was also very much a plain technical consideration, particularly for Android where the processing power of devices ranged from quite weak to quite powerful. Processing multiple video streams in HD resolution simultaneously was no small feat at the time for a mobile processor, to a small degree this might even still be true today. This meant that not only was there a (very) limited selection of video editing apps with the ability to handle more than just one video track at the same time, but even when an app like KineMaster or PowerDirector generally supported the use of multiple video tracks, this feature was only available for certain devices, excluding phones and tablets with very basic processors that weren’t up to the task. Now this has very much changed over the last years with SoCs (System-on-a-chip) becoming more and more powerful, at least when it comes to handling video footage in FHD 1080p resolution as opposed to UHD/4K! Sure, I bet there’s still a handful of (old) budget Android devices out there that can’t handle two tracks of HD video in an editing app but mostly, having the ability to use at least two video tracks is not really tied to technical restraints anymore – if the app developers want their app to have multi-track editing then they should be able to integrate that. And you can definitely see that an increasing number of video editing apps have (added) this feature – one that’s really good, cross-platform and free without watermark is VN which I wrote about in an earlier article.

So, despite having argued that two video tracks in an editing app is not an absolute prerequisite for producing a good video story on your phone, the fact that nowadays many apps and basically all devices support this feature very much reduces the potential conflict that could arise from such an opinion. I do hope however that the mindset of the phoneography movement continues to be one of “can do” instead of “can’t do”, exploring new ways of storytelling, not just producing traditional formats with new “non-traditional” devices.

As usual, feel free to drop a comment or get in touch on the Twitter @smartfilming. If you like this blog, consider signing up for my Telegram channel t.me/smartfilming.

#8 Bare bones or full featured: best stock camera apps for shooting video on smartphones – PART 1 — 20. December 2017

#8 Bare bones or full featured: best stock camera apps for shooting video on smartphones – PART 1

One of the first steps when getting more serious about producing video content with a smartphone is to look at the more advanced video recording apps from 3rd party developers. Popular favorites like „FilmicPro“ (available for both Android and iOS) usually offer way more image composition controls, recording options and helpful pro features that you find on dedicated video cameras than the native stock camera app provided by the maker of the smartphone. While quite a few stock camera apps now actually have fairly advanced manual controls when shooting photos (ability to set ISO and shutter speed might be the most prominent example), the video mode unfortunately and frustratingly is still almost always neglected, leaving the eager user with a bare minimum of controls and options. In 2015 however, LG introduced a game changer in this regard: the V10. For the first time in smartphone history, a phone maker (also) focused on a full featured video recording mode: it included among other things the ability to set ISO and shutter speed, lock exposure, pull focus seamlessly, check audio levels via an audio level meter, adjust audio gain, set microphone directionality, use external microphones, alter the bit rate etc. etc. Sure, for certain users there were still some things missing that you could find in 3rd party apps like the option to change the frame rate to 25fps if  you’re delivering for a PAL broadcast but that’s only for a very specific use case – in general, this move by LG was groundbreaking and a bold and important statement for video production on a smartphone. But what about other phone makers? How good are their native camera apps when it comes to advanced options and controls for recording video? Can they compete with dedicated 3rd party apps?

First off, let me tell you why in most cases, you DO want to have a 3rd party app for recording video (at least if you have an Android phone): external microphones. With the exception of LG, Samsung (and I’m told OnePlus) in their recent flagship lines (plus Apple in general), no stock camera app I have come across supports the use of external microphones when recording video. Having good audio in a video is really important in most cases and external microphones (connected via headphone jack, microUSB, USB-C or Lightning connector) can be a big help in achieving that goal.

So why would you use a stock camera app over a dedicated 3rd party app at all? Familiarity. I guess many of us use the native camera app of a smartphone when snapping casual, everyday photos and maybe also videos in non-professional situations. So why not build on that familiarity? Simplicity. The default UI of most native camera apps is pretty straight-forward and simple. Some might prefer this to a more complex UI featured in more advanced 3rd party apps. Affordability. You don’t have to spend a single extra penny for it. I’m generally an avid advocate of supporting excellent 3rd party app developers by paying for their apps but others might not want to invest. The most important reason in my opinion however is: Stability/Reliability. This might not be true for every stock camera app on every phone (I think especially owners of Sony phones and lately the Essential Phone could beg to differ) but because of the fact that the app was developed by the maker of the phone and is usually less complex than 3rd party apps, chances are good that it will run more stable and is less prone to (compatibility) bugs, especially when you consider the plethora of Android devices out there. The V10’s stock camera app, despite being rather complex,  is rock-solid and hasn’t crashed on me once in almost 2 years now.

Over the last months I have taken a closer look at a whole lot of stock camera apps on smartphones from LG, Samsung, Apple, Huawei, Sony, Motorola/Lenovo, Nokia (both their older Windows Phone / Windows Mobile offerings AND their new Android handsets), HTC, Nextbit, BQ, Wiko and Google/Nexus. It goes without saying that I wasn’t able to inspect stock camera apps on all  the different phone models of a manufacturer. This is important to say because some phone makers give their flagships models a more advanced camera app than their budget devices while others offer the same native camera app across all (or at least most) of their device portfolio. Also, features might be added on newer models. So keep in mind, all I want to do is to give a rough overview from my perspective and offer some thoughts on which phone makers are paying more attention to pro features in the video recording department.

The lowest common denominator for recording video in a stock camera app on a smartphone at the moment is that you will have a button to start recording in full-auto mode with a resolution of 1920×1080 (1080p)  (1280×720 on some entry level or older devices) at a frame rate of 30fps. „full-auto“ basically means that exposure, focus and white balance (color temperature) will be set and adjusted automatically by the app depending on the situation and the algorithm / image processing routine. While this might sound like a convenient and good idea in general to get things done without much hassle, the auto-mode will not always produce the desired results because it’s not „smart“ enough to judge what’s important for you in the shot and therefore doesn’t get exposure, focus and/or white balance right. It might also change these parameters while recording when you don’t want them to, like for instance when you are panning the camera. Therefore one of the crucial features to get more control over the image is the ability to adjust and lock exposure, focus and white balance because if these parameters shift (too wildly/abruptly/randomly) while recording, it makes the video look amateurish. So let’s have a look at a couple of stock camera apps.

To be continued soon with “Part 2″…