AI

The team behind Baidu’s first smart speaker is now using AI to make films

Comment

rct studio
Image Credits: rct studio

The HBO sci-fi blockbuster Westworld has been an inspiring look into what humanlike robots can do for us in the meatspace. While current technologies are not quite advanced enough to make Westworld a reality, startups are attempting to replicate the sort of human-robot interaction it presents in virtual space.

Rct studio, which just graduated from Y Combinator and ranked among TechCrunch’s nine favorite picks from the batch, is one of them. The “Westworld” in the TV series, a far-future theme park staffed by highly convincing androids, lets visitors live out their heroic and sadistic fantasies free of consequences.

There are a few reasons why rct studio, which is keeping mum about the meaning of its deliberately lower-cased name for later revelation, is going for the computer-generated world. Besides the technical challenge, playing a fictional universe out virtually does away the geographic constraint. The Westworld experience, in contrast, happens within a confined, meticulously built park.

“Westworld is built in a physical world. I think in this age and time, that’s not what we want to get into,” Xinjie Ma, who heads up marketing for rct, told TechCrunch. “Doing it in the physical environment is too hard, but we can build a virtual world that’s completely under control.”

rct studio
Rct studio wants to build the Westworld experience in virtual worlds. / Image: rct studio

The startup appears suitable to undertake the task. The eight-people team is led by Cheng Lyu, the 29-year-old entrepreneur who goes by Jesse and helped Baidu build up its smart speaker unit from scratch after the Chinese search giant acquired his voice startup Raven in 2017. Along with several of Raven’s core members, Lyu left Baidu in 2018 to start rct.

“We appreciate a lot the support and opportunities given by Baidu and during the years we have grown up dramatically,” said Ma, who previously oversaw marketing at Raven.

Let AI write the script

Immersive films, or games, depending on how one wants to classify the emerging field, are already available with pre-written scripts for users to pick from. Rct wants to take the experience to the next level by recruiting artificial intelligence for screenwriting.

At the center of the project is the company’s proprietary engine, Morpheus. Rct feeds it mountains of data based on human-written storylines so the characters it powers know how to adapt to situations in real time. When the codes are sophisticated enough, rct hopes the engine can self-learn and formulate its own ideas.

“It takes an enormous amount of time and effort for humans to come up with a story logic. With machines, we can quickly produce an infinite number of narrative choices,” said Ma.

To venture through rct’s immersive worlds, users wear a virtual reality headset and control their simulated self via voice. The choice of audio came as a natural step given the team’s experience with natural language processing, but the startup also welcomes the chance to develop new devices for more lifelike journeys.

“It’s sort of like how the film Ready Player One built its own gadgets for the virtual world. Or Apple, which designs its own devices to carry out superior software experience,” explained Ma.

On the creative front, rct believes Morpheus could be a productivity tool for filmmakers as it can take a story arc and dissect it into a decision-making tree within seconds. The engine can also render text to 3D images, so when a filmmaker inputs the text “the man throws the cup to the desk behind the sofa,” the computer can instantly produce the corresponding animation.

Path to monetization

Investors are buying into rct’s offering. The startup is about to close its Series A funding round just months after banking seed money from Y Combinator and Chinese venture capital firm Skysaga, it told TechCrunch.

The company has a few imminent tasks before achieving its Westworld dream. For one, it needs a lot of technical talent to train Morpheus with screenplay data. No one on the team had experience in filmmaking, so it’s on the lookout for a creative head who appreciates AI’s application in films.

rct studio
Rct studio’s software takes a story arc and dissects it into a decision-making tree within seconds. / Image: rct studio

“Not all filmmakers we approach like what we do, which is understandable because it’s a very mature industry, while others get excited about tech’s possibility,” said Ma.

The startup’s entry into the fictional world was less about a passion for films than an imperative to shake up a traditional space with AI. Smart speakers were its first foray, but making changes to tangible objects that people are already accustomed to proved challenging. There has been some interest in voice-controlled speakers, but they are far from achieving ubiquity. Then movies crossed the team’s mind.

“There are two main routes to make use of AI. One is to target a vertical sector, like cars and speakers, but these things have physical constraints. The other application, like Alpha Go, largely exists in the lab. We wanted something that’s both free of physical limitation and holds commercial potential.”

The Beijing and Los Angeles-based startup isn’t content with just making the software. Eventually, it wants to release its own films. The company has inked a long-term partnership with Future Affairs Administration, a Chinese sci-fi publisher representing about 200 writers, including the Hugo award-winning Cixin Liu. The pair is expected to start co-producing interactive films within a year.

Rct’s path is reminiscent of a giant that precedes it: Pixar Animation Studios. The Chinese company didn’t exactly look to the California-based studio for inspiration, but the analog was a useful shortcut to pitch to investors.

“A confident company doesn’t really draw parallels with others, but we do share similarities to Pixar, which also started as a tech company, publishes its own films, and has built its own engine,” said Ma. “A lot of studios are asking how much we price our engine at, but we are targeting the consumer market. Making our own films carry so many more possibilities than simply selling a piece of software.”

Our 9 favorite startups from Y Combinator W19 Demo Day 2

More TechCrunch

The keynote will be focused on Apple’s software offerings and the developers that power them, including the latest versions of iOS, iPadOS, macOS, tvOS, visionOS and watchOS.

Watch Apple kick off WWDC 2024 right here

As WWDC 2024 nears, all sorts of rumors and leaks have emerged about what iOS 18 and its AI-powered apps and features have in store.

What to expect from Apple’s AI-powered iOS 18 at WWDC 2024

Welcome to Elon Musk’s X. The social network formerly known as Twitter where the rules are made up and the check marks don’t matter. Or do they? The Tesla and…

Elon Musk’s X: A complete timeline of what Twitter has become

TechCrunch has kept readers informed regarding Fearless Fund’s courtroom battle to provide business grants to Black women. Today, we are happy to announce that Fearless Fund CEO and co-founder Arian…

Fearless Fund’s Arian Simone coming to Disrupt 2024

Bridgy Fed is one of the efforts aimed at connecting the fediverse with the web, Bluesky and, perhaps later, other networks like Nostr.

Bluesky and Mastodon users can now talk to each other with Bridgy Fed

Zoox, Amazon’s self-driving unit, is bringing its autonomous vehicles to more cities.  The self-driving technology company announced Wednesday plans to begin testing in Austin and Miami this summer. The two…

Zoox to test self-driving cars in Austin and Miami 

Called Stable Audio Open, the generative model takes a text description and outputs a recording up to 47 seconds in length.

Stability AI releases a sound generator

It’s not just instant-delivery startups that are struggling. Oda, the Norway-based online supermarket delivery startup, has confirmed layoffs of 150 jobs as it drastically scales back its expansion ambitions to…

SoftBank-backed grocery startup Oda lays off 150, resets focus on Norway and Sweden

Newsletter platform Substack is introducing the ability for writers to send videos to their subscribers via Chat, its private community feature, the company announced on Wednesday. The rollout of video…

Substack brings video to its Chat feature

Hiya, folks, and welcome to TechCrunch’s inaugural AI newsletter. It’s truly a thrill to type those words — this one’s been long in the making, and we’re excited to finally…

This Week in AI: Ex-OpenAI staff call for safety and transparency

Ms. Rachel isn’t a household name, but if you spend a lot of time with toddlers, she might as well be a rockstar. She’s like Steve from Blues Clues for…

Cameo fumbles on Ms. Rachel fundraiser as fans receive credits instead of videos  

Cartwheel helps animators go from zero to basic movement, so creating a scene or character with elementary motions like taking a step, swatting a fly or sitting down is easier.

Cartwheel generates 3D animations from scratch to power up creators

The new tool, which is set to arrive in Wix’s app builder tool this week, guides users through a chatbot-like interface to understand the goals, intent and aesthetic of their…

Wix’s new tool taps AI to generate smartphone apps

ClickUp Knowledge Management combines a new wiki-like editor and with a new AI system that can also bring in data from Google Drive, Dropbox, Confluence, Figma and other sources.

ClickUp wants to take on Notion and Confluence with its new AI-based Knowledge Base

New York City, home to over 60,000 gig delivery workers, has been cracking down on cheap, uncertified e-bikes that have resulted in battery fires across the city.  Some e-bike providers…

Whizz wants to own the delivery e-bike subscription space, starting with NYC

This is the last major step before Starliner can be certified as an operational crew system, and the first Starliner mission is expected to launch in 2025. 

Boeing’s Starliner astronaut capsule is en route to the ISS 

TechCrunch Disrupt 2024 in San Francisco is the must-attend event for startup founders aiming to make their mark in the tech world. This year, founders have three exciting ways to…

Three ways founders can shine at TechCrunch Disrupt 2024

Google’s newest startup program, announced on Wednesday, aims to bring AI technology to the public sector. The newly launched “Google for Startups AI Academy: American Infrastructure” will offer participants hands-on…

Google’s new startup program focuses on bringing AI to public infrastructure

eBay’s newest AI feature allows sellers to replace image backgrounds with AI-generated backdrops. The tool is now available for iOS users in the U.S., U.K., and Germany. It’ll gradually roll…

eBay debuts AI-powered background tool to enhance product images

If you’re anything like me, you’ve tried every to-do list app and productivity system, only to find yourself giving up sooner than later because sooner than later, managing your productivity…

Hoop uses AI to automatically manage your to-do list

Asana is using its work graph to train LLMs with the goal of creating AI assistants that work alongside human employees in company workflows.

Asana introduces ‘AI teammates’ designed to work alongside human employees

Taloflow, an early stage startup changing the way companies evaluate and select software, has raised $1.3M in a seed round.

Taloflow puts AI to work on software vendor selection to reduce costs and save time

The startup is hoping its durable filters can make metals refining and battery recycling more efficient, too.

SiTration uses silicon wafers to reclaim critical minerals from mining waste

Spun out of Bosch, Dive wants to change how manufacturers use computer simulations by both using modern mathematical approaches and cloud computing.

Dive goes cloud-native for its computational fluid dynamics simulation service

The tension between incumbents and fintechs has existed for decades. But every once in a while, the two groups decide to put their competition aside and work together. In an…

When foes become friends: Capital One partners with fintech giants Stripe, Adyen to prevent fraud

After growing 500% year-over-year in the past year, Understory is now launching a product focused on the renewable energy sector.

Insurance provider Understory gets into renewable energy following $15M Series A

Ashkenazi will start her new role at Google’s parent company on July 31, after 23 years at Eli Lilly.

Alphabet brings on Eli Lilly’s Anat Ashkenazi as CFO

Tobiko aims to reimagine how teams work with data by offering a dbt-compatible data transformation platform.

With $21.8M in funding, Tobiko aims to build a modern data platform

In 1816, French physician René Laennec invented an instrument that allowed doctors to listen to the heart and lungs. That device — a stethoscope — eventually evolved from a simple…

Eko Health scores $41M to detect heart and lung disease earlier and more accurately

The number of satellites on low Earth orbit is poised to explode over the coming years as more mega-constellations come online. This will create new opportunities for bad actors to…

DARPA and Slingshot build system to detect ‘wolf in sheep’s clothing’ adversary satellites