Why using AI in filmmaking looks like studying the classics
Melbourne / Australia
〰️
Melbourne / Australia 〰️
Image: Unsplash
TL;DR: Artificial intelligence is already woven through filmmaking, says Lester Francois at the St Kilda Film Festival.
So what? If you really want to get the most out of the new tools, study the classics.
At the St Kilda Film Festival, filmmaker Lester Francois moderated the panel discussion "Unlocking the Potential of AI & the Moving Image," a pivotal exploration of AI's impact on the film industry.
Francois, known for his innovative work across various formats including VR, engaged with fellow screen artists Steve Mills, bitpixi, Rhett Mankind, and KEZIAI to delve into the uncharted creative territories opened up by AI. As the producer and director at StudioBento, Francois has a history of pioneering projects like the VR documentary RONE, which premiered at SXSW and Cannes.
In this interview, he shares his early fascination with AI, its transformative role in modern filmmaking, and offers valuable advice for aspiring filmmakers navigating this new technological landscape.
New Vacancy: When did you first develop an interest in storytelling and filmmaking? What drew you to it?
Lester Francois: When I was about 6 years old my sister would buy records and play them at home. I would listen to them and visualise stories to go along with the music. I’d daydream of grabbing the family Super 8 camera and making a short film to the music—at that young age I was already visualising story.
NV: Can you remember your first experience of using AI technology? If so—tell me about it?
LF: I saw Google DeepMind when it first came out and was curious about it, but ultimately it was too limiting and I never explored it much.
Years later during the pandemic I discovered VQGAN+Clip which was the start of the new text2image technology. It was very primitive and setting it up was a pain but I took to it instantly. I was fascinated by images being created in the mystical latent space.
Those early years working with the crude tech was a great foundation for me. Being so early I was able to grasp the building blocks of the AI technology. The work I created at the time was very janky and weird but I loved it! I became obsessed!
But I never imagined AI would accelerate so quickly where we have Midjourney in our pockets and AI video so easy to create. I thought this would still be many years away.
Image generation: Dalle-E
NV: Do you think AI is reshaping storytelling and production processes in film and TV? And, if so, what opportunities do you see for storytellers in it?
LF: Not so much mainstream storytelling at the moment, but definitely production processes and pitching projects.
We already have AI in Adobe Premiere cleaning up bad audio and soon we will be able to generate AI video directly in Premiere. Initially this will be great for low budget music videos and experimental films but as AI generated video quality increases, I can see the commercial world using it a lot for online content.
I have already worked with a major broadcaster to help generate a key image in a pitch deck. This was after a few designers had a go working on the image but in the end, they couldn’t get the dramatic image the creative team were after. I was able to come and after a lot of iterating and design work we were able to create a key image that captured the essence of the show in a photo realistic way.
NV: What ethical considerations should filmmakers keep in mind when experimenting with AI into their creative processes?
LF: You can go in any direction with AI to create anything you can imagine and more. I am less worried about filmmakers trying to tell stories with AI but very worried about bad actors using AI.
There is a coming onslaught of AI political propaganda around the world shifting opinions and soon there could be long form videos made with AI that distorts the truth and manipulates viewers. We already see this on social media with crappy AI, but what happens when the tools become very easy and the quality is indistinguishable from reality? Deepfakes are already targeting women and all of this has me very concerned.
NV: There's a debate about AI’s role as a tool or a threat to traditional filmmaking roles. How can filmmakers best adapt to ensure AI complements rather than replaces human creativity?
LF: AI is just a tool. Sometimes it speeds up the creative process for teams and other times it enables solo creatives to think more ambitiously. I’m not worried about talented filmmakers replacing their own creative vision with AI—they may use it, but won’t lose their unique voice. If anything, their unique vision could be enhanced through the technology.
At the same time, we are seeing novice creators using AI video to make films. Jumping into AI creation without having a foundation in creative theory is problematic. There is already a glut of low-quality AI work online and audiences can quickly tell the two apart—human talent with a unique vision will always win.
Image: Unsplash
NV: Are there particular AI innovations or trends you're excited about?
LF: One month in AI innovation is a year in the real world.
When I started getting serious with AI creation, the small community I was in would get excited when a new technology or process would be released every 6 months. Now there are new tools coming out every week! It’s so hard to keep up.
AI will play a big role in filmmaking whether we like it or not. It’s hard to tell how the disrupting will play out, and there is talk we may see the current AI model quality hitting a ceiling soon.
Right now, I’m really excited by the Australian company Leonardo AI. I’m in their creator program and have been a power user. It’s a very powerful platform and not just the simple text2image like Midjourney. I feel platforms like this will enable filmmakers at all levels to leverage AI to help in the early stages of development and funding. They may not use AI in the final film, but it will help in the development phase.
NV: For young filmmakers entering the industry, what advice would you give about using AI in their projects? Are there particular skills or areas of knowledge they should focus on?
I think young filmmakers are already using AI. They have been using filters, face swapping and other tools in their social apps for years already.
The very young filmmakers I’ve met are embracing AI and experimenting with it. Not to replace conventional filmmaking but to add it as another tool in their kit, especially when iterating at the concept stage.
My advice would be to have a good understanding of art history, literature, film technique, science and general knowledge. This is already a good foundation to be a creative but when AI prompting, having a command of these ideas goes a long way to creating engaging work. If your well of knowledge is empty, then the results will show.
If you are already interested in drawing, photography, music creation I would suggest keep doing what you’re doing and don’t swap that for AI. What you learn from traditional methods you can later apply to AI creation, and having traditional skills is a huge advantage when working with AI.
