As the SAG and WGA strike drags on, there has been a lot of discussion about the threat that artificial intelligence poses to writers and actors. But what about editors?

AI is already making the editing process more streamlined than ever. Is a transition already underway? This article will explore how AI is affecting the art and science of film editing, what it means for the future of film, and the big question: are film editors destined to become obsolete?

The Evolution of Film Editing: A Brief History

“Editing is where movies are made or broken. Many a film has been saved and many a film has been ruined in the editing room”

Joe Dante, Director

Editing is the backbone of the filmmaking process. A masterfully edited film, such as “Mad Max: Fury Road” or “The Social Network,” can elevate the story, pacing, mood, performance, and themes in a movie, and is often the difference between good and great, great and classic.

In the early days of cinema, editing was a hands-on, manual process involving physical film reels. The editing device young Sammy coveted in Steven Spielberg’s “The Fabelmans” epitomizes this age-old technique of literally cutting and joining film strips.

From Analog to Digital

By the 1990s and 2000s, editing underwent a digital transformation. A combination of technological advances in hard drive storage, processing power, and editing software revolutionized the film industry by making digital editing feasible for feature films.

These advances collectively led to the industry’s widespread adoption of digital editing by the early 2000s, supplanting analog methods and becoming the new industry standard.

Early reactions to digital editing were mixed. On one hand, many industry professionals welcomed the efficiencies and creative possibilities of digital editing systems. Software like Avid and later Adobe Premiere Pro and Final Cut Pro allowed for non-linear editing, allowing editors to easily rearrange scenes, experiment with different cuts, and collaborate more efficiently.

However, some purists in the field were initially skeptical or even resistant to the move to digital. They valued the tactile nature of cutting film manually and were concerned that the craft might lose some of its artistry in the transition to a digital medium.

Over time, as digital technologies improved and offered even more tools for creativity, many skeptics were won over. Today, all editing is done digitally, enhancing the role that technology plays in the field.

Human vs. Machine: The Intersection of Technology & Art

Software has come a long way in assisting with the mechanical aspects of editing like cuts, transitions, and even some color correction. However, editing is not just about stitching clips together; it’s a deeply human art form that demands a nuanced understanding of timing, mood, and the emotional undercurrents of a story.

For now, the heartbeat of editing remains human. Choices are at the foundation of craft. Turning raw footage into a narrative consists of millions of individual decisions, each of which has the potential to impact the pacing, sequence, and tone of the story. These choices require a level of understanding of human behavior and cultural context that artificial intelligence has yet to reach.

Neither technology nor art stands still, however. As we look at what AI is capable of today, and what it might be able to do tomorrow, where are we heading?

Advantages of AI in Film Editing

AI has already begun to impact the filmmaking and editing processes. There are 3 huge benefits:

1. Speed: Reducing the amount of time spent on editing.
2. Precision: The ability to make micro-adjustments that might be difficult for a human.
3. Collaboration: AI can serve as a “co-editor,” allowing human editors to focus on more creative tasks.

Real-World Examples of AI in Editing

These are just a few examples of how AI tools are changing the editing field. This isn’t meant to be a comprehensive guide but to give a sense of how AI is streamlining processes, and enabling editors to achieve their creative vision more efficiently.

Adobe Premiere for AI-Powered Editing

The Auto Reframe tool uses machine learning algorithms to automatically identify focal points within a frame and adjust the aspect ratio to suit various platforms, making it particularly useful for filmmakers wanting to produce trailers or snippets for social media.

Similarly, the Speech to Text feature offers AI-driven transcription services, converting spoken dialogue into text for subtitles or captions.

Although not entirely driven by AI, the Color Match tool employs intelligent algorithms to harmonize the color grade between different shots, aiding filmmakers in maintaining a consistent aesthetic throughout their project.

In addition, the Content-Aware Fill feature, inherited from Photoshop, employs AI to remove unwanted objects or imperfections within clips.

Finally, Adobe’s Sensei AI technology is integrated into various audio

iZotope Neutron for Audio Mixing

iZotope Neutron is a plugin designed for audio mixing and is widely used in both music and post-production for film. It uses machine learning algorithms to analyze audio tracks and provide suggested settings to achieve a balanced mix. Some professionals find its AI suggestions a good starting point, but human dicretion is often required to fine-tune the mix.

Movavi Video Editor for Quick Edits

Movavi Video Editor is a lightweight, user-friendly video editing software aimed at beginners and those looking for quick and straightforward video editing solutions. AI-driven features include background removal and noise removal.

Runway for Masking

Runway is an interesting platform that allows you to create full videos just from text descriptions. You describe a scene (“horse sitting in a cubicle playing the trombone”), and this tool will create a video of it for you on the spot. The technology is still fairly limited, but it’s come a long way in a very short time and will be one to watch going forward.

Colourlab.AI for Coloring

Colourlab.AI specializes in using machine learning algorithms to assist in the color grading process. The software can analyze your footage and automatically apply a color grade based on a chosen look or reference footage. It can also match different shots to ensure visual consistency. Like iZotope Neutron, many professionals find its AI-generated results a good starting point, but additional human input is generally required for final touches.

Descript for Dubbing

Descript offers “Overdub,” a feature that utilizes AI to generate a synthetic voice clone to insert new lines into your audio. For filmmakers, this could be extremely useful for tasks like dialogue editing or even temporary voice-over work before the final recording.

Artistic Considerations

The debate over AI’s role in film editing brings artistic questions to the forefront. On one hand, proponents argue that AI can handle mundane tasks, allowing human editors to focus on creative aspects; skeptics worry this could dilute the unique artistry of each film.

I wrote about this in a previous article on AI-generated art.

Aiding the Creative Process

One argument in favor of AI’s role in editing is that it acts as an advanced tool that can handle the more monotonous aspects of the job, freeing editors to focus on creative decisions. In this view, the artistry isn’t compromised but enhanced, akin to a painter using a new type of brush that allows for more intricate detail.

Devaluation of Craft

On the other hand, some argue that the involvement of AI could devalue the artistry and skill involved in film editing. They worry that relying on AI to make creative decisions—such as shot selection or pacing—might result in a homogenization of styles and stifle the human creativity that makes each film unique.

Where to Draw the Line

If one editor uses AI just for sorting and labeling footage and another for continuity editing and color correction, the debate becomes even murkier. Both are utilizing AI, but to different extents and for different tasks. Is one approach more “authentic” than the other? What amount of AI assistance would constitute something being labeled as an AI-edited film?

Limitations of AI Editing: Not Making the Cut

“Here we are in the domain of the human spirit: what do you want to say and how do you want to say it?”

Legendary Editor Walter Murch in his book In the “Blink of an Eye: A Perspective on Film Editing”

Film editing is as much about storytelling as it is about technical skills. Editors understand the emotional weight carried by a glance between characters or the suspense built through precise timing.

These judgments often rely on cultural context, making it challenging for an AI to discern universally. For instance, the editing style of many of Yasujirō Ozu’s films reflects a particular cultural moment in 1950s post-war Japan, something that isn’t easily replicated by a machine.

Moreover, the practicalities of employing AI in film editing include computational and financial concerns. AI algorithms, especially more advanced forms like neural networks or GANs, require substantial computational power. This often translates to a need for significant computing power, which can be expensive.

The Future of AI in Film Editing

In my opinion, the possibility of AI overtaking the creative nuance and emotional intelligence of seasoned editors seems unlikely. As mentioned above, editing remains a fundamentally creative endeavor, a domain where AI lags. AI often misses the small details that we, as humans, are tuned to catch – which is why AI-generated images often have hands with four fingers or a three-legged person. 

The true question might not be about outdoing exceptional talent but rather about AI becoming good enough to replace the average editor. If this happens, the few editors at the top of the field may be safe, but widespread job displacement within the industry could become a legitimate concern.

Still, AI’s current limitations make this scenario improbable in the immediate future. Despite their sophisticated algorithms, these machines often misunderstand context, misinterpret references, and inadvertently delete or misorder essential content, inhibiting the natural flow of a story.

I think we’re likely to see AI’s impact on social media platforms before it significantly affects the film industry. Imagine uploading some videos to TikTok and simply typing a prompt like, “Edit this video to tell a story about walking my dog.” Advanced AI could execute this task effortlessly, setting the stage for more complex editing tasks in the future. In theory, a Chat GPT-like editing software could interpret a prompt that requests a cut that combines the styles of iconic editors like Thelma Schoonmaker and Michael Kahn.

Five High-Level Possibilities

Still, the continued relevance of editors is not assured. Here are a few ways AI and editing could play out over the coming years and decades.

Scenario 1: AI as Assistive Tools for Human Editors

In the future, AI becomes an integral part of the video editing workflow, handling repetitive tasks like sorting clips, stabilizing footage, and basic color grading. Video editing becomes a partnership between AI and humans. Artists still guide the creative direction, leveraging the extra time and capacity AI provides them.

Scenario 2: The Rise of Medium to High-Quality AI-Generated Content

AI capabilities increase to the point where they can produce entire short-form videos, including documentaries and simple narrative pieces. Human involvement remains crucial for complex storytelling elements and artistic direction. However, simpler content can be produced quickly and inexpensively by AI (e.g. one-click documentaries).

Scenario 3: The Rise of Low-Quality AI-Generated Content (aka the TikTok Apocalypse)

AI becomes capable of churning out vast amounts of low-quality or formulaic content, like mobile games or TikTok videos. While not particularly good, the technology is widely available and this content floods the internet, capturing viewers’ attention due to sheer volume. It’s probably fair to say this is a more dystopian version of the online world we’re already living in.

Scenario 4: The Fall of Entry-Level Positions

AI advances make it possible to automate tasks typically assigned to assistant or entry-level editors. As a result, these positions become scarcer, and the career ladder becomes steeper for those entering the field.

Scenario 5: Complete AI Takeover

In the most extreme scenario, AI capabilities surpass human abilities in all aspects of video editing, from technical tasks to creative storytelling. Humans focus on high-level creative and strategic roles, possibly in new roles that we haven’t even thought of yet.

Conclusion

Artificial Intelligence is undeniably altering the landscape of film editing, with platforms leveraging advanced algorithms to automate and streamline various editing tasks. However, the technology hasn’t reached the point where it can replicate the nuanced artistry and creative intuition that human editors bring to the table. While it’s tempting to envision a future where AI might be good enough to replace human editors, for now, the heart and soul of film editing remain human.

As we stand on the precipice of what could be a new era in filmmaking, one has to wonder: If AI technologies continue to improve and become “good enough,” will we find ourselves debating not just the ethics but also the very essence of what makes film editing an art? This question serves as an invitation to think more deeply about the role and value of human creativity in an increasingly automated world.