Can AI be creative? IBM’s John R Smith says yes it can!

In recent decades, Artificial intelligence (AI) has gone from science fiction to science fact..

Indeed, in the 21st century as technology begins to catch up with fantasy, AI is becoming a part of everyday life. Whilst truly sentient AI, a la Space Odyssey and Terminator, is (thankfully) some way off, AI is already beginning to usurp humans in areas of traditional employment such as manufacturing.  And while the big bad AI of science fiction meta has failed to materialise, many people are feeling threatened by AI as it encroaches on the traditional workforce.

To stave off the inevitable demise, the government is pushing funding into STEM, suggesting we need young people with these skills to combat and ride the wave of the rise of tech.

Other professions with soft skills like dancing and acting and directing are being lauded as AI-proof, yet with technology advancing daily, is it possible robots will one day be creative?

According to Dr John R Smith it’s not only possible, we are already at a tipping point.

Smith suggests the emergence of AI-driven content science will radically change the future of creative work, fuelling creativity and connecting with audiences in a meaningful way.

As Head of AI Tech for IBM Research globally, Smith leads the tech giant’s R&D team into Vision, Speech, Language, Knowledge and Interaction.

He has driven the creation of audio-visual recognition tools, medical image analysis, and augmented creativity for movie trailers, ad creation and sports highlight reels.

Smith suggests creativity is the next horizon for AI and tells KBB  it’s time we asked the question can AI be creative?

“AI evolution is happening at extremely rapid pace,” says Smith. “We have computers that can see images, speech technology that can provide universal translation, we also have face detection which is giving way to deeper capabilities like understanding moods and emotions from expressions.”

So, yes, according to Smith, AI can be creative. “Not in the way people are creative. But AI can be creepy…”

Taking movie trailers as a starting point Smith championed a campaign to see whether AI might be used in creative industries to assist with the creative process.

Smith says to do this he needed to look at movies like a black box.

“Making movie trailers, we can look at the creative process as a black box – input movies output trailer.  For that to succeed we build the factual base – we give the computer context to understand the trailer.”

He elaborates: “Movies are about stories and stories are about communication and communication is about emotion and that is about what we hear and see and the language we use.”

Smith suggests while AI can’t experience emotion, it can analyse an emotion and expression and acquire knowledge.

When 20th Century Fox challenged IBM’s Watson to make a movie trailer for its suspense horror flick, Morgan, in 2016,  Smith was eager to accept.

“So we said let’s send Watson to film school,” Smith laughs. “Utilising footage from the Alfred Hitchcock classic Psycho, Watson was given a crash course in suspense.”

After ingesting Psycho, Watson learned a representation that it could then work with. But in order for Watson to be able to meet the challenge of creating a trailer it needed to know more.

“As an illustration of what it could do, it would take a scene or a photo and tag all the content. So, if you feed in a movie you could go in scene by scene and the computer would analyse everything.

“The beauty of this is a computer can’t get tired. It doesn’t need popcorn. It can watch 1000s of movies and it can get an insight into movie scenes. But to go beyond the visual scene we also looked at sentiment and emotion. Then the computer had an ability to label the content with adjectives like if it looked happy or sad or scary. The computer then also learned a model of how things sound. Whether ambient, speech or music and then that gives a feature space.”

The final element of the equation was to see what went into making up a trailer for a scary movie.

“By giving the AI historical trailers to analyse and determining what matters, Watson was able to identify and assess the best candidates for a trailer,” says Smith. “It was almost a data mining exercise of film trailers. What we found was very straightforward. There are three principal axes in this space, suspenseful, scary and tender.

“So when we fed in the new film (Morgan) it was able to apply what it had learned and make assessments about the best candidates for the trailer.

“After Watson “watched” the movie, it identified 10 moments that would be the best candidates for a trailer. But we still needed the skill of a human to put those elements together.”

Whilst Watson didn’t actually cut the scenes the computer program was able to trim the creative process considerably. Traditionally a team of editors would spend a month or more collating and viewing footage to select the best scenes for use in a trailer. Watson managed to reduce this to a 24-hour period, including the edit.

“Reducing the time of a process from weeks to hours – that is the true power of AI,” Smith explains.

“The combination of machine intelligence and human expertise is a powerful one.

While we can’t answer the deeper question – what is creativity – the process or the product? We can say assisted creativity is here to stay…”

John R Smith is a guest at Pause Fest 2018


Related Posts

Cec Busby
Cec is the managing editor of KBB. She is a multimedia professional with 20 years experience as an editor on titles as diverse as SX, CULT, Better Pictures, Total Rock, MTV, fasterlouder, mynikonlife and Fantastic Living. She has extensive experience working as a news journalist covering all the issues that matter in the political, health and LGBTIQ arena. She is the Head of Content at Pinstripe Media and a recent convert to the world of small business.


Please enter your comment!
Please enter your name here