GuShiio.com Article source: Xin Zhiyuan
Image source: Generated by AI
A doctoral degree in machine learning from the TOP 5 universities in the United States, but the laboratory doesn’t even have a GPU that can provide a lot of computing power?
In mid-2024, a netizen posted a post on reddit, which immediately triggered a big discussion in the community–
At the end of the year, a report by Nature exposed the severe challenge faced by academia in acquiring GPUs-researchers actually had to queue up to apply for the use time of school GPU clusters.
Similarly, in the laboratories of universities in our country, the problem of severe shortage of GPUs is also common. There has even been such outrageous news that universities require students to bring their own computing power to attend classes.
It can be seen that the bottleneck of “computing power” has even turned AI itself into a course with extremely high thresholds.
There is a shortage of AI talents and insufficient computing power
At the same time, the rapid development of cutting-edge technologies such as large models and embodied intelligence is causing talent shortages around the world.
According to calculations by professors at Oxford University, the proportion of jobs requiring AI skills in the United States has increased fivefold.
During this period, growth was particularly significant in Asia.
While universities around the world are trying to help students master key AI capabilities, as mentioned earlier, computing power has now become a “luxury”.
In order to bridge this gap, cooperation between enterprises and universities has become an important means.
Kunpeng Shengteng Science and Education Innovation Incubation Center opens the layout of scientific research in universities
Fortunately, in order to cultivate the same innovation system in our country’s universities, Huawei has already begun to deploy!
Now, Huawei has signed the cooperation of the “Kunpeng Shengteng Science and Education Innovation Center of Excellence” with five top universities, Peking University, Tsinghua University, Shanghai Jiao Tong University, Zhejiang University, and Science and Technology of China.
In addition, Huawei has also simultaneously promoted cooperation with 10 universities including Fudan University, Harbin Institute of Technology, Huazhong University of Science and Technology, Xi’an Jiaotong University, Nanjing University, Beijing University of Aeronautics and Astronautics, Beijing Institute of Technology, University of Electronic Science and Technology, Southeast University, and Beijing University of Posts and Telecommunications. Cooperation with the “Kunpeng Shengteng Science and Education Innovation Incubation Center”.
The establishment of centers of excellence and incubation centers is a model for the integration of industry and education:
- By introducing Shengteng Ecology, it has made up for the shortage of computing power in colleges and universities and greatly promoted the burst of more scientific research results;
- By reforming the curriculum system and driving it by scientific research topics, industry topics, and competition topics, we will cultivate top talents in the computing industry;
- Strive to breed world-class innovation results through key architecture, computing acceleration capabilities, algorithm capabilities, and system capabilities;
- By creating many “AI+X” cross-disciplines, we will lead the development of intelligent ecology.
Build completely independent domestic computing power for AI scientific research
Today, the significance of AI for Science is self-evident.
According to a recent survey by Google DeepMind, one in three postdoctoral researchers uses large language models to assist in literature review, programming, and article writing.
This year’s Nobel Prize in Physics and Chemistry were awarded to researchers in the field of AI.
It can be seen that in the process of AI-enabled scientific research, GPUs have become precious “gold” with their excellent performance in these fields that require high-performance computing and their powerful ability to train and reason for LLM., they have been snapped up by major companies such as Microsoft, xAI, and OpenAI.
However, the United States ‘blockade of GPUs has made my country’s progress in AI and scientific research difficult.
In order to bridge this gap, we must build an autonomous and complete ecosystem and develop it.
At the computing power level, Huawei’s Shengteng series AI processors have taken on the important task of reshaping my country’s competitiveness.
On top of computing power, we also need a self-developed computing framework to adapt it to give full play to the advantages of NPU/AI processors.
As we all know, the CUDA architecture designed specifically for NVIDIA GPUs is more common in the fields of AI and data science.
The only one that truly competes with it in the country is CANN.
As Huawei’s heterogeneous computing architecture for AI scenarios, CANN supports industry mainstream AI frameworks such as PyTorch, TensorFlow, and Shengsi MindSpore on the top, and enables Shengteng AI processors on the bottom, which is a key platform for improving the computing efficiency of Shengteng AI processors.
Because of this, CANN is inherently endowed with many technical advantages. The most critical of these are more in-depth optimization of software and hardware for AI computing and a more open software stack:
- First, it can support multiple AI frameworks, including its own MindSpore, as well as third-party PyTorch, TensorFlow, etc.;
- Secondly, it provides multi-level programming interfaces for diverse application scenarios, allowing users to quickly build AI applications and services based on the Shengteng platform;
- Moreover, it also provides model migration tools to facilitate developers to quickly migrate projects to the Shengteng platform.
At present, CANN has initially built its own ecosystem. At the technical level, CANN includes a large number of applications, tools, and libraries, has a complete technical ecosystem, and provides users with a one-stop development experience. At the same time, the team of developers based on the Shengteng technology base has gradually grown, laying fertile ground for future technology applications and innovation.
On top of the heterogeneous computing architecture CANN, we also need a deep learning framework built by AI models.
Almost all AI developers need to use deep learning frameworks, and almost all DL algorithms and applications must also be implemented through deep learning frameworks.
Nowadays, mainstream frameworks such as Google TensorFlow and Meta’s PyTorch are already on the market, and a huge ecosystem has been formed.
Entering the era of large-scale model training, deep learning frameworks require effective training when faced with thousands of computers.
Huawei MindSpore, a full-scene deep learning framework officially open source in March 2020, has made up for the domestic gap in this field and achieved true autonomy and controllability.
MindSpore has key features such as cloud-side full-scene deployment, native support for large model training, and support for AI+ scientific computing. It has built a native development environment with full-scene collaboration and full-process simplicity, accelerating domestic scientific research innovation and industrial applications.
What is special is that as the “best partner” of Shengteng AI processors, MindSpore supports all scenarios of “end, edge, and cloud”, which can achieve a unified architecture, one training, and multiple deployments.
Everything from earth system simulation and autonomous driving to protein structure prediction can be achieved through MindSpore.
For deep learning open source frameworks, only a broad developer ecosystem can promote its improvement and release greater value.
The “China Artificial Intelligence Framework Market Research Report” released by research institute Omdia in 2023 shows that MindSpore has entered the first tier of AI framework utilization, second only to TensorFlow.
In addition, the reasoning application of thousands of industries is the key to releasing the value of AI. In the process of accelerating the development of GenAI, both universities and enterprises need to urgently address the high demand for accelerated reasoning speed.
For example, the high-performance optimization compiler TensorRT is a powerful tool to improve the reasoning performance of large models. With the help of quantification and sparsity, it can reduce the complexity of the model and efficiently optimize the reasoning speed of deep learning models. But the problem is that it only supports NVIDIA’s GPUs.
In the same way, we have both a computing architecture, a deep learning framework, and a matching reasoning engine-Huawei Shengteng MindIE.
MindIE is a full-scenario AI inference acceleration engine that integrates the industry’s most advanced inference acceleration technology and inherits the features of open source PyTorch.
Its design takes into account flexibility and practicality, and can seamlessly connect with multiple mainstream AI frameworks. At the same time, it supports different types of Shengteng AI processors, providing users with a multi-level programming interface.
Through full-stack joint optimization and layered open AI capabilities, MindIE can release the extreme computing power of Shengteng hardware, provide users with efficient and fast deep learning reasoning solutions, and solve the technical difficulties and development problems in model reasoning and application development. The problem of multiple steps improves model throughput performance, shortens application launch time, enables hundreds of models and thousands of states, and meets diversified AI business needs.
It can be seen that independent innovative technologies such as CANN, MindSpore, and MindIE have not only filled the shortcomings of domestic computing power, but also achieved leapfrog breakthroughs in model training, framework ease of use, reasoning performance, etc., and even directly benchmarked foreign advanced technology stacks.
Build a world-class incubation center
In addition to its technical advantages, it can be said that in the next few decades, the use of Shengteng computing power will be more in line with the needs of national conditions.
Only domestic self-research computing power can we get rid of the ever-changing external environmental impact and ensure the stability of the scientific research base.
Now that the platform is set up, how can we let college teachers and students learn to use it?
Since September 6 last year, Huawei has successively held the first Shengteng AI special training camp at Peking University, Shanghai Jiao Tong, Zhejiang University, and China University of Science and Technology. Among the hundreds of students who signed up, 90% are masters and doctoral students. The courses cover courses in various aspects such as CANN, MindSpore, MindIE, MindSpeed, HPC, and Kunpeng Development Tools in the Shengteng field.
In the special training camp, students not only have a detailed understanding of core technologies, but also have the opportunity to practice them. This arrangement is very consistent with students ‘absorption characteristics of new knowledge, from shallow to deep, step by step.
For example, in Shanghai, the first day of the course will be based on migration, allowing students to understand Shengteng AI basic software and hardware solutions, Pytorch model Shengteng native development case practice, MindIE reasoning solution features and migration cases.
The next day’s course will focus on optimization, including CANN, the development of Ascend C operator, practical operation of large model long sequence reasoning optimization, etc.
Migrating and optimizing curriculum settings can be said to be far-reaching.
You should know that the practical courses in many colleges and universities are basically set up based on CUDA/X86. However, under the influence of sanctions, the problem of insufficient computing power has become increasingly prominent. At this time, if you master the migration method, you can put the project on the platform of Shengteng so that the academic industry can continue to operate.
After mastering the basics, students can do it themselves in the practical cases section. Huawei experts will guide students hands-on, allowing them to learn the Ascending Technology Stack and experience the entire process of large-scale model reasoning in the process of large-scale model quantification, reasoning, and Codelabs code implementation and deployment.
After practical practice, students will gain more understanding of Shengteng Ecology through personal experience, laying a solid foundation for subsequent work in the technical field.
In the first special training camp of Shanghai Jiao Tong, students are doing practical exercises
In addition to courses, Huawei will also hold operator challenges for college developers to mine operator development elites.
The competition encourages developers to carry out in-depth innovation and practice based on Shengteng’s computing power resources and CANN’s basic capabilities, accelerate the integration of AI and the industry, and promote the improvement of developer capabilities.
In addition, the incubation center also attaches great importance to academic achievements.
Students who conduct academic research based on key technologies and tools of Kunpeng or Tengteng computing can also apply for postgraduate scholarships. During this period, if the paper is published in international top conferences and domestic top journals, there will also be corresponding rewards.
This program allows students to move from theory to practice and enter the real working scenarios of the company, while helping outstanding students connect with the company in advance.
Now, the Talent Talent Program has united more than 200 companies in 15 cities, provided more than 2000 technical positions, and enabled more than 10,000 college students to apply for positions.
In general, through these teaching practices and incentive plans, students ‘enthusiasm for participation can be greatly increased. Not only will they enhance their academic experience and produce scientific research results, but their experience and background will also be more eye-catching, thus gaining points in the job market and making it easier for them to gain the favor of top companies at home and abroad.
So, after mastering the latest technology and its application, how can we cultivate truly breakthrough scientific research results in today’s ever-changing AI?
Since Sora set off an AI wave for Wensheng videos in 2024, large models of Wensheng videos have been emerging. Open-Sora Plan, the open source cultural video project of Peking University and Rabbit Exhibition, has caused a stir in the industry.
In fact, the team was already preparing for the open source version of Sora when Sora was launched. However, the computing power and data requirements could not be met, and the project was put on hold. Fortunately, Peking University and Huawei jointly established the Kunpeng Shengteng Science and Education Innovation Center of Excellence, allowing the team to quickly receive computing support.
The team originally used the Nvidia A100. After migrating Shengteng Ecology, they made various surprising discoveries–
CANN’s support can achieve high-performance parallel computing and significantly speed up the processing of large-scale data sets; the Ascend C interface class library simplifies the development process of AI applications; and the operator acceleration library further optimizes algorithm performance.
More importantly, the open Shengteng ecosystem can quickly adapt large models and applications.
Therefore, although team members have been exposed to Shengteng Ecology since 0, they can quickly get started in a short period of time.
In the following training, the team continued to find surprises: for example, when using torch_npu for development, the entire code can be seamlessly trained and reasoned on Shengteng NPU.
When model segmentation needs to be carried out, Shengteng MindSpeed distributed acceleration suite provides a wealth of large-scale model distributed algorithms and parallel strategies.
In addition, in large-scale training, the stability of using MindSpeed and Shengteng hardware is much higher than that of other computing platforms, and can be uninterrupted for a week.
Therefore, in just one month, Open-Sora Plan was officially launched and gained great recognition in the industry.
The scene of “Black Myth: Wukong” generated by Open-Sora Plan is comparable to a blockbuster movie and stunned countless netizens
In addition, for Shengteng computing power, Southeast University has also developed a multi-modal transportation model MT-GPT.
In the past, the implementation of large transportation models was very difficult. The reasons included data island problems caused by data collection by different government departments, inconsistent data formats and standards, and heterogeneous and multiple sources of transportation data.
In order to solve these problems, the team specially conceived a multimodal transportation large model conceptual framework called MT-GPT (Multimodal Transportation Generative Pre-trained Transformer) to provide data-driven for multi-directional and multi-granular decision-making problems in multimodal transportation system tasks.
However, in the development and training of large models, there is no doubt that the requirements for computing power bases are extremely high.
To this end, the team chose to use Shengteng AI’s capabilities to accelerate the development, training, tuning and deployment of large transportation models.
During the development stage, the Transformer large model development kit collaboratively improves the understanding accuracy of multimodal generative problems through multi-source heterogeneous knowledge corpus and multimodal feature coding.
During the training phase, Shengteng MindSpeed distributed training acceleration kit provides multi-dimensional, multi-modal, and multi-modal acceleration algorithms for large traffic models.
During the tuning stage, Shengteng MindStudio’s full-process tool chain combines knowledge in the specialized field of fine-tuning transportation for training and fine-tuning.
During the deployment phase, Shengteng MindIE inference engine can assist one-stop reasoning for large transportation models, and can also support cross-city migration analysis, development, debugging and tuning.
To sum up, we can find that Peking University Open-Sora is a migration project that reproduces Sora. At the same time, as an open source project, it can also better empower developers around the world to apply more scenarios.
Southeast University’s multi-modal transportation model MT-GPT reflects the actual ability of Shengteng computing power to transform results and directly empowers the city’s transportation industry.
As a result, a closed loop of industry, academia and research has been fully formed.
These fruitful results also further prove this point: Centers of Excellence/Incubation Centers can not only provide universities with fertile ground for academic research and scientific innovation, but also cultivate a large number of top AI talents, and then incubate world-leading scientific research results.
For example, while the Peking University team was developing the Open-Sora Plan, Teacher Yuan Li would organize students and Huawei Shengteng team to brainstorm the development of code and algorithms every day.
In the process of crossing the river by feeling the stones, many students from the Peking University team personally participated in a high-quality scientific research practice, demonstrating extremely high scientific research creativity.
This team, with an average age of 23, has also become the backbone of promoting domestic AI video applications.
In this process, the young learning team who has mastered the ecology of Kunpeng’s Shengteng has also continued to grow.
Therefore, universities do research based on domestic computing power and platforms, which not only gain the support of top intelligence, but also expand Huawei’s technology ecosystem and applications in the process.
It can be seen that Huawei has officially set sail for a new paradigm of school-enterprise cooperation.
Since establishing the computing product line in 2019, Huawei quickly signed a smart dock cooperation project with the Ministry of Education in 2020 to carry out teaching cooperation among the top 72 universities in the country.
At that time, some technical knowledge of Kunpeng/Shengteng had been integrated into some required undergraduate courses.
However, investment in colleges and universities is a medium-and long-term cultivation process. Only by letting students and teachers give priority to understanding relevant technologies can it be of greater value in the next few years.
Therefore, Huawei plans to invest 1 billion yuan every year to develop the native ecology and talents of Kunpeng and Shengteng. The implementation of this strategy will provide university talents and developers with richer resources and broader development space. A gift plan for 100,000 Kunpeng Development Boards and Shengteng Reasoning Development Boards has also been launched to encourage them to engage in teaching experiments, actively explore and apply Kunpeng and Shengteng technologies in competition practice and technological innovation.
According to this plan, teachers and students can get close contact with and try out the development board. Whether it is teachers ‘teaching or scientific research experiments, college teachers and students can inspire new inspiration from the innovations they want to do above.
The OrangePi AIpro development board jointly launched by Xiangcheng School and Huawei Shengteng meets the needs of most AI algorithm prototype verification and inference application development. It can be widely used in AI edge computing, deep visual learning, drones, cloud computing and other fields. Demonstrate strong strength and wide applicability
On the other hand, my country’s current special situation-the technological blockade from the outside world-also means that we have not much time left. We must have an independent and controllable technology stack.
Native development is imperative in the future. Only Made in China is the most in line with the future trend of China’s national conditions.
As localization becomes the general trend, domestic technology stacks such as Kunpeng/Shengteng will also spread to various IT infrastructures.
The launch of centers of excellence and incubation centers has also given the industry more and more confidence.
It is foreseeable that after several years of incubation, scientific research talents who master the foundation of domestic technology will continue to carry forward the Kunpeng/Shengteng technical route and incubate scientific research results that are sufficient to lead the world.
