When Yang learned last August that his uncle passed away unexpectedly while working away from home, his family knew they must conceal this devastating news from his ailing 90-year-old grandmother.
Yang, an internet industry professional in his 30s in the eastern city of Nanjing, Jiangsu province, understands artificial intelligence (AI) technology better than most, thanks to his line of work. He then paid a friend working in the AI industry around 10,000 yuan ($1,387) to “resurrect” his late uncle using AI-powered human image synthesis — a technology that generates realistic human images and renderings. The friend impersonated Yang’s uncle and made a video call to his grandmother.
During the video call, Yang’s mother and relatives opted to stay away, fearing that they would not be able to control their emotions. Yang instructed his friend to keep the conversation brief and simply greet his grandmother, as speaking too much could raise suspicion.
The friend, Zhang Zewei, is a vlogger for Super Brain, an AI-focused content creation company that predominantly operates via social media. He went from creating AI courses to starting an AI “resurrection” business. Among Zhang’s clientele, cases like Yang’s are considered necessities — typically situations where someone in the client’s family has died or been imprisoned, and there is a need to conceal it from young and old family members who are unlikely to suspect that the person on the other end of the line is an AI avatar.
Was the AI-generated image convincing? According to Yang, his grandmother didn’t notice anything awry, and his mother described it as very realistic after seeing a trial version. “Since I have some understanding of AI technology, I can usually spot it,” Yang explained. “When you know it’s fake, you can easily see the flaws. But if someone is unaware such technology exists, they won’t suspect anything is off and will accept it as real.”
Obviously, this raises numerous ethical questions about using AI to create digital avatars of the deceased. Yang said there is no simple answer, adding, “Internet industry professionals like me find it easier to embrace this technology, whereas outsiders or the elderly may find it difficult to accept.”
The companionship of digital avatars can help maintain family stability for the living, said Shen Yang, a journalism and communication professor at Tsinghua University. “If a digital avatar is made after someone passes away, their loved ones will think that person is still waiting for them at home, and they feel that the person is still there to accompany them,” he explained.
But the professor also warns of potential ethical issues, such as the right not to be digitally resurrected. “Are the deceased willing to have a permanent digital version of themselves in the world? Do we have their consent by default? These are difficult questions indeed,” Shen said. “People should make their wishes clear before death. In the future, we may need to clear up these issues in civil law.”
Role-playing with AI
Yang paid a little more than 10,000 yuan for three AI video calls. The most significant expense was 9,800 yuan to train and generate the AI, plus 200 yuan for each video call. As Zhang’s AI business sees an increase in orders, it helps lower the costs, and as AI technology continues to advance, the service’s price also drops. Aside from the AI video call service used by Yang, Zhang also offers another service — a role-playing session with the deceased that aims to provide clients with in-depth psychological counseling.
Last year, Zhang received a request from a woman who wanted to say a final farewell to her boyfriend, who died unexpectedly two years earlier. “We had dinner together just two days before his sudden death. It has been two years, but I have not been able to move on,” she told him. “I have tried therapy, and even done some extreme things. The life we planned together suddenly and completely fell apart. I want to have a dialogue with him to say goodbye properly. Maybe then I can restart my life.”
Once again, by using human image synthesis, Zhang replaced his face and voice with those of the woman’s late boyfriend and managed his facial expressions and dialogue.
When the video call began, the woman, upon seeing her boyfriend’s face, could not help but cry.
For such services that involve deep psychological interactions, aside from using AI technology to mimic people’s appearance and voice, Zhang also hires psychologists to play the role of the deceased, which is included in the service fees.
Unlike Yang’s grandmother, clients using this service understand that the person on the screen is not real. It becomes essentially a role-playing game that serves as therapy.
Aware of the potential ethical concerns from his service, Zhang signs agreements with clients in advance to protect their privacy and stipulate that the digital AI avatar cannot be used for any illegal or improper purposes. If clients want to “resurrect” a deceased person, they must provide proof of their relationship with official documentation.
Zhang initially had complicated feelings about providing such services. He said he has hardly experienced much suffering personally. But all of a sudden, he had to live through the painful experiences of hundreds of his clients. “It had a great impact on me. I began dreaming about their situations at night,” he said. As time passed, he gradually found ways to cope and grew more resilient as he gained experience. “It is like a doctor performing surgery. Initially, seeing blood might make you dizzy, but with more experience, you become numb to it.”
Shen, the Tsinghua University professor, also warned that good intentions don’t always lead to good results. “Although such use of AI is meant to provide psychological support, AI simulations of loved ones cannot completely stand in for real people. The service may lead some clients to fantasize and think that they can truly communicate with their deceased loved ones, which may result in trauma and dependence, or interfere with the normal grieving process,” he warned. “The long-term effects of such services must be taken into consideration, including the possibility of developing unhealthy attachments to the deceased.”
Using AI to generate people, especially the deceased, is challenging. It is not possible to get normal voice and video samples from the deceased, meaning the online data they left behind while they were alive is crucially important. Zhang explained that producing an AI video call last year required around 20 to 30 minutes of voice samples and one to two minutes of video samples. This year, as AI models have continuously improved, production only requires around 15 seconds of voice samples and 10 seconds of video samples. However, the more samples used, the more realistic the digital avatar.
The fewer the samples, the longer production takes, and the higher the labor and technological costs. Zhang explained that it would take around half a month and cost around 7,000 to 8,000 yuan to resurrect a loved one in a video call if there is only one photo and a voice sample of just a few seconds.
Pioneering AI resurrection business
According to Zhang, he did not initially intend to profit from this, but little did he know that his business would be a wild success. When he first received private messages from clients requesting to resurrect their loved ones, he offered the service for free. “Within one or two months, I was inundated with countless cases of human suffering. I could not bring myself to ask for payment,” he said. He recognized the potential, however, and admitted that starting with a few pro bono cases turned out to be a strategic way to open up an untapped market.
After completing dozens of cases for free, he gradually began charging clients, with prices ranging from a few thousand to tens of thousands of yuan per case. Zhang estimates the gross profit margin to be around 50%-60%, while the net profit margin is around 30%-40%.
Zhang screens clients’ inquiries to assess the genuineness of their needs and accepts less than half of the orders. As of mid-March, Zhang’s studio, which has been offering its resurrection service for almost a year, has accepted nearly a thousand orders for the service, including ongoing cases. Business revenue has reached millions of yuan.
Zhang is aware of the novelty of his business, which has attracted a frenzy of media coverage, including Agence France-Presse, French television station TF1, and The Straits Times in Singapore. As a result, Zhang has seen an increase in client consultations. “Preliminary estimates indicate that there have been 4,000-5,000 consultations, but we do not have the resources to count them all because our exposure has been so extensive during this period,” said Zhang.
Zhang’s studio currently employs five people. The cost of creating a digital avatar generally ranges from 4,000 to 5,000 yuan, and the manual labor takes around seven days. Meanwhile, AI training requires seven days of continuous computing, 24 hours a day. If there is insufficient training data, production time will be longer, even up to half a month. The more training data available from when the person was alive, the closer the digital avatar will resemble that person in terms of emotional expression and tone. It takes several months’ worth of samples to fully mimic a person’s emotional expressions; it takes years’ worth of samples to closely approximate the actual person.
Currently, Zhang’s studio uses foreign open-source software, which has been incorporated into their proprietary core software. But Zhang cannot disclose the specific foreign open-source software, citing it as a trade secret.
To train the language output of the digital avatars, Zhang’s studio employs the interface of Baidu’s ERNIE Bot 4.0 large language model, combined with the client’s personal data, to train and generate smaller, specialized language models. For computing power, they use Nvidia’s RTX 4090 graphics cards locally, along with Alibaba Cloud online.
Xing Xiaoci, chief architect of the digital space business group at SenseTime, said the cost of generating a digital avatar is less than 10,000 yuan, but additional computing costs are required for language training using foundation models after generating the digital avatar.
Currently, Zhang’s studio’s primary marketing strategy is exposure through videos on social media, aiming to generate inquiries about their services. The total number of followers across all their social media platforms currently exceeds 300,000.
The effectiveness of marketing varies on different social media platforms, including WeChat Video; Douyin, the Chinese version of TikTok; and lifestyle platform Xiaohongshu. Initially, their view count on Xiaohongshu was very high, but the platform then appeared to have restricted their traffic. “Xiaohongshu was very strange. The data was excellent at first, with each video garnering 400,000-500,000 views, and we had around 17,000 followers. Later, for some unknown reason, our channel was restricted,” Zhang explained. “We appealed to the platform, and their response was that our account was functioning normally. But in fact, we knew it was still restricted because it was impossible that identical content, which previously had hundreds of thousands of views, now only had 100-200 when posted.” He then tried to use Xiaohongshu’s paid promotion tool to increase traffic but was informed that the content did not align with the platform’s guidelines. “We do not understand why we were restricted.”
Zhang speculates that their initial success on Xiaohongshu was due to the platform’s higher proportion of female users, who tend to be more drawn to compassionate content. Most of his videos revolve around human suffering, which resonates more with empathetic viewers. He also believes WeChat Video can drive traffic and opportunities for content creators, but this window of opportunity is also closing. In his opinion, given the recommendation algorithms of short video platforms, it is challenging to promote videos with sad content to users. “If the content is particularly somber, users are unlikely to save, like, or share the video.”
Zhang’s studio was the first to launch a business resurrecting loved ones with AI in China and remains the largest to date. While some digital resurrection services are being offered on major e-commerce platforms like Taobao and Xianyu, their customer bases generally consist of only a few dozen individuals.
Though Zhang now also targets individual consumers, his business initially provided AI training courses to institutional clients. On an episode of Zhejiang Television’s “Ace vs. Ace” about combating fraud, Zhang’s studio provided technical support by swapping celebrities’ faces.
Zhang’s workload has increased as AI resurrections have gained popularity. Currently, the fan clubs of late Hong Kong singers Leslie Cheung and Coco Lee are in discussions with Zhang to create digital avatars of their idols, and now he is in the process of obtaining authorizations.
From the cost of talent to the cost of computing power, training AI foundation models is extremely expensive. Large companies can afford to conduct research and development themselves, while startups require substantial financing. Currently, Chinese companies that work on foundation models mainly focus on providing customized services to companies; few are like Zhang, who can self-finance and continue to generate revenue in the now fragmented and highly commercialized business-to-consumer market.
Zhu Xiaohu, managing director at GSR Ventures, which invests in early-stage internet, mobile, and new media companies, believes that finding product-market fit (PMF) is the key in the AI content creation market. “You can (invest) in 10 or even 100 people, and still not find (someone who can deliver PMF). It has nothing to do with the number of people or the investment amount. The key is whether you can find PMF,” he told Tencent News. “If you find it, you do not need to spend tens of millions of dollars to train a foundation model — just training with LLaMA, Meta’s open-source foundational model, for two or three months is enough, and the cost will not be high.”
Zhang said that many large companies also want to start offering AI resurrection services. “But from a business perspective, large companies will find it difficult to directly tap into this market because it is too complex,” he said.
The “complexity” Zhang refers to includes traditional Chinese culture. “For example, Qingming Festival, a day when people visit cemeteries to pay their respects to their ancestors, is more about consoling oneself,” he explained. Another aspect is the psychological needs of individual clients, such as the woman who requested to “role-play” with her deceased boyfriend to help her move on from her past. “It is necessary to adapt according to the user’s needs, which means that the production process cannot be standardized in the short term,” Zhang said.
Deepfakes versus “real” digital avatars
In addition to resurrecting deceased loved ones, another facet of Zhang’s business is creating digital avatars for the living. Trained using videos, audio, and text data provided by living individuals, these digital avatars can possess the same appearance, voice, thoughts, and even conversational habits as the real person.
The resurrection of the late Professor Tang Xiao’ou, founder of Hong Kong AI company SenseTime, at the company’s annual party at the beginning of the year was a case that stunned many. The production process of the digital avatar involved limited training data and a foundation model.
At a March 14 event by Tencent Research Institute, SenseTime’s Xing explained that the training data for Tang’s digital avatar came from published videos of him when he was alive. Four or five recordings of Tang when he was alive were used for the voice training, each lasting around 3-4 seconds, each with different styles. “Some were teasing and humorous, others were loving and sincere,” said Xing. He further explained that these different styles of audio were then used for few-shot learning — AI training based on limited data — to create Tang’s appearance and voice. For language generation, text data from when Tang was alive was used to train the foundation model, ensuring that the generated content matched Tang’s personality to form a humorous conversational style.
For any missing parts in the training data, such as Tang’s laugh while speaking, which cannot be found in the original training materials, a video generation AI model was needed. “Initially, Tang’s family members deemed the smile of the digital avatar to be too unnatural, so we continuously adjusted the parameters and used SenseTime’s video generation technology, which is still under development, to capture his smile,” Xing said.
Zhang points out that the digital avatar business is still in the stage of educating the market. “Actually, the logic behind digital avatars is the same as taking photos. After a photo is taken of someone, whether that person is alive or dead, people will view the photo nostalgically. To transition from photos to digital avatars, it is necessary to first familiarize users with the concept,” he said.
Reported by Wang Qin.
A version of this article originally appeared in BlueWhale Finance. It has been translated and edited for brevity and clarity, and is republished here with permission.
Translator: Vincent Chow; editors: Xue Ni and Elise Mak.
(Header: Fantastic Graphics/VCG)
>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : SixthTone – https://www.sixthtone.com/news/1014906/Grieving%20in%20the%20Age%20of%20AI:%20When%20Virtual%20Reunions%20Blur%20Reality