As the digital wave sweeps across the globe, the boundary between virtual and real is being continually reshaped by technology. From intelligent assistants on mobile devices in the information industry to digital clothing design in smart manufacturing; from the creation of virtual figures and characters in cultural and creative fields to digital teachers and digital doctors in the service sector—digital humans are moving from concept to everyday reality, deeply integrated into our work and lives.
How can we precisely replicate human appearance in the digital world while also achieving smooth, natural real-time interaction? To answer this, Professor Zhou Kun, Director of the State Key Laboratory of Computer Aided Design and Computer Graphics at Zhejiang University, has led his team through more than a decade of exploration, achieving breakthrough results in the key technologies of intelligent 3D digital human production and earning a National Science and Technology Progress Award (Second Class). Today, this cutting-edge technology—"grown inside industry"—is rapidly moving from the lab into every sector.
Making Digital Humans Truly "Come Alive"
The digital human is not an entirely new concept; related technological exploration dates back many years. But until recently, the technology remained "locked" inside laboratories and film animation studios—with production cycles of months, prohibitive costs, and no ability to generate or interact in real time on mobile devices. Zhou Kun's team set its sights precisely on this long-standing application gap—bringing digital human technology out of closed professional environments and into the hands of every ordinary user.
"We want to enable digital human generation and interaction in open environments using just a mobile phone, so everyone can enjoy the convenience technology brings," Zhou said. But in open environments, the modeling accuracy, interaction stability, and simulation real-time performance of digital humans all demand entirely new technological breakthroughs. "The core of this project is integrating AI technology into the entire digital human production pipeline—upgrading and transforming the whole chain."
Zhou Kun's team innovatively proposed "precise digital human modeling from single-view images," eliminating the need for multi-dimensional capture with professional equipment. A single-view image alone can replicate a person's 3D appearance—work that won first place in the IEEE CVPR 2025 Single-View Reconstruction Challenge, a top computer vision conference. With a realistic appearance in place, a digital human also needs vivid interaction. But in open environments, noise interference, rapid movement, and lighting changes can all undermine interaction stability. The digital human driving algorithm pioneered by Zhou Kun's team overcame this challenge, effectively correcting deviations caused by various disturbances and giving digital humans "all-condition" stable expressiveness unaffected by the external environment. The work received the ACM SIGGRAPH 2024 Test of Time Award, a top computer graphics conference, and was praised for "opening a path toward generating realistic facial animation on mobile devices."
Having solved "accuracy" and "stability," the team then tackled "real-time performance." Details such as clothing wrinkles and hair movement determine a digital human's quality, but the algorithms behind them are extremely complex. The team's "physics- and data-driven joint real-time simulation" strategy combines physical laws with data-driven methods, improving computational efficiency by 10 to 100 times while maintaining simulation accuracy. These three technological breakthroughs interlock, taking digital humans from the laboratory into a vast range of real-world application scenarios.
Bringing Digital Humans into Every Industry
Breakthroughs in frontier technology cannot happen without the pull of real needs and the tempering of practical implementation. "In the lab, we can't face users directly, so there's still a big gap between technology R&D and real-world deployment," Zhou acknowledged. How big is that gap? He gave an example: "In 2014, our digital human driving technology had already achieved real-time interaction on mobile devices. But going from lab metrics to consumer-grade products meant algorithm runtime had to be compressed from tens of milliseconds to just a few milliseconds, and memory usage from hundreds of megabytes to tens of megabytes." These "demanding requirements" from the front lines of industry pushed the team to keep innovating.
For Zhou, applying technology in consumer-grade scenarios also requires deep corporate involvement. "After a research team achieves technology innovation from 0 to 1, the breakthroughs from 1 to 10 and from 10 to 100 require the industrialization strength of enterprises to drive forward." Zhejiang University's all-embracing culture, its well-established industry-academia-research-application mechanisms, and Hangzhou's fertile ground for innovation and entrepreneurship all provide exceptional conditions for technology transfer.
In this project, the team has enjoyed more than a decade of strong collaboration with NetEase. The two sides have jointly undertaken multiple national-level projects and formed close ties through technology licensing and talent development. Meanwhile, FaceUnity Technology and Style3D, founded by ZJU alumni, have—with the lab's incubation and support—repeatedly refined their technologies in specific application scenarios, keeping their R&D direction aligned with industry front lines. Both have grown into national-level "little giant" enterprises specializing in niche sectors.
"Corporate collaboration is not just about licensing patented technology. More importantly, companies help us discover the real bottlenecks in practical applications." The value of research lies in solving real problems. When laboratory breakthroughs mesh deeply with industry needs, technology gains the vitality and productivity for continuous iteration.
The "Create" and "Build" of Research
Years of research and university-industry collaboration have given Zhou Kun a deep understanding of how to cultivate "people who can create" from "people who can study." "'Creation' can be broken into two parts: 'create' and 'build.' In computer science, 'create' means innovation and pioneering, while 'build' means practical implementation. If you only have ideas but can't bring them to life, that's not creation."
He often tells his team: "Think like a layperson, practice like an expert." Knowing how to ask questions is the core ability in research. "'Being able to create' ultimately means being able to ask questions." What will truly be scarce in the future are people skilled at posing "good questions." Zhou Kun's team has done exactly that—by continually probing the "bottlenecks" of technology deployment and pushing the limits of open environments, they have moved digital humans step by step from the lab into every industry.
Over the past decade, Zhou Kun's team focused on "voice and countenance"—giving digital humans both "form" and "spirit." In the next decade, they will focus on combining "emotion and intelligence." "Beyond a realistic appearance, digital humans also need highly intelligent and emotionally intelligent interaction capabilities," Zhou said. "This interaction is multimodal—eye contact, micro-expressions, gestures, full-body movement, and more. Only through multi-dimensional coordination can we present a 'digital intelligent human' with warmth." This, in turn, depends on deep interdisciplinary integration. Although the team's main field is computer graphics, the project has long incorporated computer vision, human-computer interaction, natural language processing, and other disciplinary approaches. "The trend of interdisciplinary integration means technology R&D no longer focuses on a single discipline, but uses knowledge and methods from different fields to solve problems."
From technological concept to industrial deployment: "I hope the new technologies developed in the lab can, within 5 to 10 years, find and break through the real bottlenecks through industrial application," Zhou said. Zhou Kun's team has always pursued the principle of making technology truly serve people, ensuring that every step of innovation and creation is grounded in people's aspirations for a better life and illuminates the infinite possibilities of the future.
