20 Years of Glory and a New Quality Future - 2024 The 20th National High Performance Computing Academic Annual Conference Was Successfully Held in Wuhan

"The thoroughfare of nine provinces reaches the world, and the light of supercomputing shines on the world." From September 24 to 26, the 20th CCF National High Performance Computing Academic Annual Conference (CCF HPC China 2024) was grandly held at the China Optics Valley Science and Technology Exhibition Center in Wuhan. The conference was hosted by the China Computer Society, organized by the High Performance Computing Professional Committee of the China Computer Society, Huazhong University of Science and Technology, and Shanghai Jiaotong University, and co-organized by China University of Geosciences (Wuhan) and Beijing Parallel Technology Co., Ltd.

With the theme of "20 Years of Glory, New Quality Future", this conference was led by 12 academicians and joined hands with more than 400 top scholars in the computing field to conduct academic exchanges and special topic sharing. The conference also held 30 theme forums and more than 30 colorful peripheral activities, providing participants with opportunities for interactive learning to the greatest extent while exchanging cutting-edge technologies. The number of participants in the conference exceeded 4,000, setting a new record.

Recalling the glorious past and looking forward to the new quality industry in the future

After 20 years of ups and downs, supercomputing has a new look. CCF HPC China was founded in 2005, and this year is the 20th. It has become one of the three most influential supercomputing events in the field of high-performance computing, on par with the SC Supercomputing Conference in the United States and the ISC Supercomputing Conference in Germany. Over the past 20 years, the High Performance Computing Professional Committee of the Chinese Computer Society (hereinafter referred to as the "High Performance Computing Committee") has built a professional, high-end and extensive communication platform for academia and industry, as well as for high-performance computing users and foreign academic peers through an academic platform such as CCF HPC China, effectively promoting the rapid development of China's high-performance computing industry. In 2024, China's high-performance computing will have an important opportunity to deeply study the close relationship between artificial intelligence and new quality productivity and computing power industries. As a top industry event that continues to open up the future, CCF HPC China is committed to adding new potential to the industry through extensive exchanges and cooperation.

Entering the new era of AIGC, artificial intelligence applications need to continuously consolidate the computing power base. As a key force leading future development, computing power is also an important part of the new quality productivity. In the face of new scientific and technological revolutions and industrial changes, new quality productivity is a more efficient and advanced production development model. It focuses not only on "new", but more importantly on achieving a leap in "productivity" and optimizing its structure. High-performance computing provides powerful computing power for artificial intelligence, cloud computing, big data, the Internet of Things, etc., paving a "highway" for the creation of new quality productivity. This CCF HPC China is a conference that empowers new quality productivity and accelerates its value release. By gathering and displaying cutting-edge academic achievements, innovating technology applications, expanding business scenarios, and developing and strengthening the ecosystem, it will promote my country's high-performance computing industry to a new level.

In 2023, the scale of Hubei's core artificial intelligence industry will exceed 70 billion yuan, a year-on-year increase of more than 30%. "AI+" has further spawned a diversified supply of computing power. Guided by the "Action Plan for Promoting the High-Quality Development of Computing Power Infrastructure and Application Industry in Wuhan (2024-2025)", Wuhan is bringing together local and even national general computing, intelligent computing, and super computing to open up the upstream and downstream supply and demand chain of computing power and promote the high-quality development of the computing power service industry. The establishment of CCF HPC China in Wuhan will play a positive role in promoting Wuhan to build a "National Central Computing Power Base". CCF HPC China is a conference of connection and transformation, allowing new and powerful computing power to better serve Wuhan's economic development and the construction of a modern urban industrial system.

CCF HPC China is an event rooted in China and renowned internationally. With 20 years of accumulation, under the leadership of the inspiring high-level committee, it brings together a huge amount of expert resources such as Turing Award winners and Gordon Bell Prize winners in high-performance computing, domestic and foreign academicians, international association fellows, authoritative experts, as well as senior scholars, corporate elites, etc., covering scientific research, education, enterprises, institutions and audiences from all walks of life. CCF HPC China is more confident and confident in building a global, high-end and open high-performance computing technology and academic sharing and exchange platform.

Building on the Future of the Connected World

On the morning of September 24, CCF HPC China 2024 was grandly opened. Professor Jin Hai, Vice President of the China Computer Federation (CCF), presided over the opening ceremony of the conference.

Sun Ninghui, Chairman of CCF, Director of the Academic Committee of the Institute of Computing Technology of the Chinese Academy of Sciences, and Academician of the Chinese Academy of Engineering, delivered a speech at the conference on behalf of the organizers. He said that in the blink of an eye, China's high-performance computing industry has made great progress and has reached world-class level in 20 years. Looking to the future, high-performance computing should continue to play the role of "locomotive", continuously strengthen academic and industrial exchanges, continuously improve technical levels, promote external cooperation and exchanges, and lead the development of future technologies in the era of intelligent computing.

High performance computing is known as the crown of computer science and engineering and is an important symbol of national strength. As the representative of the organizer, You Zheng, Vice Chairman of the China Association for Science and Technology, President of Huazhong University of Science and Technology, and Academician of the Chinese Academy of Engineering, said in his speech: "This conference thoroughly implements General Secretary Xi Jinping's important statement that developing new quality productivity is an inherent requirement and important focus for promoting high-quality development, vigorously promotes scientific and technological innovation, gathers cutting-edge academic achievements, showcases the latest application technologies, and promotes the transformation and upgrading of the supercomputing technology application ecosystem and the science and technology industry. In the past 20 years, the progress made by my country in the field of high performance computing has boosted our morale and made us full of confidence in the future development of high performance computing."

Recalling the development of CCF HPC China over the past 20 years, Tang Weiqing, Secretary General of the China Computer Society, was filled with emotion. Today, CCF HPC China has become the conference with the largest number of participants, the largest exhibition scale, the largest number of participating manufacturers, and the greatest influence among all the conference activities of the High-Level Committee. In recent years, the China Computer Society has also developed rapidly, and this year's membership is expected to exceed 120,000, and the High-Level Committee has also made important contributions. In addition to serving members and setting up more offices in Shenyang, Chengdu, Xi'an, Zhuhai and other places, the China Computer Society has also set up a large number of service agencies and project agencies, totaling 283, and its service capabilities have been greatly improved. In the future, the China Computer Society will also focus on improving the quality of academic conferences, providing good services, and achieving international development.

With the in-depth development of artificial intelligence applications, the focus of large-system interconnection networks has gradually shifted from CPUs to GPUs. How to design network protocols and how to create an industrial ecosystem to cope with such a strategic shift in focus are the original intentions of the High-Throughput Ethernet (ETH+) Alliance and the goals of future work. At the conference, the Alliance grandly released the High-Throughput Ethernet Protocol 1.0. In the future, the Alliance will continue to promote the continuous improvement of the High-Throughput Ethernet Protocol in accordance with the evolution rules of annual major versions and semi-annual minor versions. Cai Dezhong, Chairman of the High-Throughput Ethernet Alliance, Vice President of R&D of Alibaba Cloud Intelligence Group, and Head of Infrastructure Network, presented the future development roadmap of the High-Throughput Ethernet Protocol, and said that AI computing will redefine the network architecture, and Ethernet-based interconnection solutions will become the mainstream of the industry.

Focus on key issues and make breakthroughs from the basic level

Vice President Guan Haibing, Director of the CCF High Performance Computing Committee, presided over the following plenary session.

Wu Jiangxing, an academician of the Chinese Academy of Engineering, director of the National Engineering and Technology Research Center for Digital Switching Systems, dean of the Institute of Big Data at Fudan University, professor and doctoral supervisor, made an in-depth analysis of the main challenges currently facing advanced computing in his report "Opening up New Paths for Advanced Computing", especially bottlenecks in data processing, storage, and migration, as well as the overall inefficiency of computing systems, unsustainable energy consumption, and increasingly prominent computing security issues. Faced with the exponential growth of global data volume and the demand for computing performance, the traditional single architecture has been difficult to meet diverse needs, and the paradigm of using electricity to compete for computing power is also unsustainable. In order to effectively solve the above problems, Academician Wu Jiangxing proposed a transformative strategy based on the "Law of Necessary Diversity", emphasizing "responding to diversity through diversity" while improving the system's adaptability. Specifically, it is to introduce the innovative path of domain-specific software and hardware collaborative computing (mimetic computing) and software-defined system on wafer (SDSoW), while taking into account the computing needs of high efficiency, high flexibility and high performance; through the dynamic aggregation of software and hardware collaboration and resource pooling, it can realize dynamic response to multiple tasks, multiple algorithms and multiple resources, and finally provide a new green, intelligent and secure computing paradigm that can meet the needs of the future intelligent era. This innovative path provides new ideas and directions for system optimization, resource utilization and computing security in the era of intelligent computing.

At present, manufacturing enterprises are very concerned about artificial intelligence, deep learning and big models. However, the biggest challenge facing manufacturing enterprises is the lack of in-depth exploration and understanding of basic issues. In his report entitled "Several Basic Issues of "AI+Manufacturing", Li Peigen, an academician of the Chinese Academy of Engineering and professor of Huazhong University of Science and Technology, clearly pointed out that the next generation of intelligent manufacturing needs to pay attention to the following five important issues: First, it is clear that data is the foundation, and data-centric artificial intelligence is created, and enterprises must enhance their ability to obtain valuable information from massive data; second, dig insights from historical data, and do not ignore those feature data that seem to have little impact. Only in this way can we better understand the high-dimensional space of manufacturing based on various weak signals; third, in the era of digital intelligence, the "craftsmanship" of manufacturing is reflected in the data, that is, the sensitivity to data and the understanding of the essential problems behind the data; fourth, the physical space (workshop) needs to achieve the integration of IT and OT; fifth, one of the keys to the application of big models is the need for intelligent agents, which may subvert the model of software development and application. From establishing personal intelligent agents for engineers and managers to realizing the industrial metaverse, it is crucial to establish a human-machine collaborative model based on intelligent agents. Professor Li Peigen said that in order to solve the above problems, scholars and experts in various fields of IT are needed to provide easy-to-use methods and platforms.

With the rapid development of high-performance computing applications, new trends such as the integration of diverse application loads and the accelerated integration and diversification of processor architectures have emerged. In the keynote speech "Collaborative Industry Innovation, Developing a New High-Performance Computing Ecosystem", Ding Zhaohui, Chief Architect of Huawei Kunpeng High-Performance Computing, introduced Huawei's thinking and practice on the ease of use, practicality and openness of high-performance computing. Huawei's philosophy is to face the industry, collaborate with industry innovation, build a new ecosystem, and provide customers with open, easy-to-use and practical high-performance computing systems. In the past few years, Huawei has always insisted on independent innovation of key root technologies; improving high-performance computing basic software, scheduling software and tool chains; at the same time, jointly innovating with partners, driving industry convergence and development, jointly building an ecosystem, and enabling industry innovation. More than 5,000 high-performance computing software have been adapted and optimized, and have been widely used in key industries such as meteorology, life sciences, advanced manufacturing, education and scientific research. To this end, Huawei has created an end-to-end solution from software to hardware: the software mainly includes a scientific computing suite for application acceleration, the job scheduling and cluster management software Danube Suite, the Euler operating system, the development migration tool DevKit, etc.; the hardware mainly provides various forms of servers around the Kunpeng processor, as well as OceanStor Pacific storage, network, etc.

Cross-border integration and innovative applications

The plenary session in the second half of the morning was hosted by Researcher Zhang Yunquan, Deputy Director of the CCF High Performance Computing Committee.

"As a new productivity in the intelligent era, brain-like intelligent computing power will bring great changes to intelligent technology and even human society," said Zhang Xu, an academician of the Chinese Academy of Sciences and president of the Guangdong Institute of Intelligent Science and Technology. In his report entitled "Research on Brain-like Intelligence in the Intelligent Era", he analyzed the brain, the most complex information and intelligent system, which will inspire brain-like intelligence theory and technology, that is, intelligence theory and technology inspired by brain science/neuroscience. In the intelligent era, the multidisciplinary cross-research paradigm of brain science research has prompted brain-like intelligent computing and other brain-like intelligent research fields to join brain science. Academician Zhang Xu said that brain-like computing is becoming a new paradigm in brain science research. Drawing on the basic principles of brain processing information and learning, new brain-like computing systems with high energy efficiency, high speed and intelligence are developed. The use of developed brain-like computing systems can accelerate the development of brain simulation and digital brain, promote the understanding of brain operating mechanisms and the treatment of brain diseases, and develop digital brain science and brain medicine. The newly emerged pulse neural network intelligent processor has laid the foundation for the construction of large-scale brain-like intelligent computing systems. Looking to the future, brain-like super computing power is likely to exceed the computing power of the human brain, affecting the transformation of intelligent technology and the development of human society.

Computational mechanics is a relatively old branch of mechanics. In the digital era, it also faces the challenge of transformation and upgrading. Guo Xu, an academician of the Chinese Academy of Sciences, a professor at Dalian University of Technology, deputy director of the National Key Laboratory of Industrial Equipment Structural Analysis and Optimization and CAE Software, and vice president of the Chinese Society of Mechanics, said in his report "Intelligent Mechanics-Computational Mechanics in the Digital Era" that modern mechanics, which combines electronic computers with mechanical work, has a very close connection with high-performance computing, because only by performing high-performance computing can we obtain high-performance computational mechanics methods and produce the effectiveness of high-performance related algorithms. In the face of the complexity of mechanical modeling and the difficulty of calculation caused by the subjectivity of modeling, it is necessary to innovate the research paradigm. With mathematics as the cornerstone and the enhancement of artificial intelligence, the dimensional disaster of CAE software computing power can be broken. In the digital era, computational mechanics also needs to be upgraded to intelligent mechanics. "Computational mechanics cannot be called modern mechanics if it is not combined with advanced computing paradigms such as artificial intelligence and big data," said Academician Guo Xu. "Under this guiding ideology, we proposed the concept of intelligent computing mechanics, which is a theoretical method and tool system based on the intelligent science of mechanical data and using advanced computing power to solve mechanical problems in science, technology and engineering. Intelligent computing mechanics is the upgrade and evolution of computational mechanics in the digital era."

Wang Huaimin, an academician of the Chinese Academy of Sciences, a professor at the National University of Defense Technology, and a member of the China Computer Federation (CCF), and Feng Dawei, an associate researcher at the National University of Defense Technology, jointly presented a report entitled "From Computing to Intelligence: Taking Materials Science as an Example". In the era of intelligent computing, AI for Science has not only promoted the progress of basic scientific research, but also brought new opportunities and challenges to the development of high-performance computing and artificial intelligence technologies. How to combine the specific application scenarios of scientific research and give full play to the potential of high-performance computing and artificial intelligence in big data analysis, simulation computing, intelligent prediction, and experimental assistance is the focus of the current application of AI for Science. Academician Wang Huaimin said that supercomputing and intelligent computing are both related and different. The complementarity between the two means that the integration of supercomputing and intelligent computing supports future scientific research and has a very broad prospect. Starting with the empowerment case of artificial intelligence for high-performance scientific computing, Feng Dawei introduced in detail the exploration of computing power, models, experiments, etc. based on the China Computing Network, focusing on the application scenarios of materials science research, and taking the large model empowering materials science research as a case to analyze the relevant research practices and future challenges.

"Over the past 40 years, the world's high-performance computing technology has achieved amazing development. It is gratifying that my country's high-performance computing industry has 'kept up' with the world and has become an important pole in the international high-performance computing field that cannot be ignored." Academician Wang Huaimin said, "But in today's very complex international scientific and technological competition, to support China's high-quality development, we cannot just be satisfied with 'keeping up and holding on'. Today, with the increasingly rich connotation of high-performance computing, we must seize the opportunities of the times and transform from followers to leaders. This is the mission of the new generation of high-performance computing entrepreneurs."

Surge computing power "calculates" new quality productivity

On the afternoon of the 24th, the conference report session attended by many academicians was also wonderful. Researcher Tan Guangming, Secretary-General of the CCF High Performance Computing Committee, hosted the first half of the speech.

The computing network should play the role of infrastructure in the intelligent era, so it should be popularized and applied by all people like browsers and WeChat. For a new technology to be popularized, it must have a "killer" application. From now on, AIPC and AI mobile phones may become intelligent personal assistants popular among the whole people, and in the future, there may be a real demand for computing network. Only by allowing computing power to serve more people through the network and allowing the majority of users to get practical benefits from the computing network can the computing network develop rapidly. Now different units have made different efforts in computing network. The key is how to form a synergy between these research and practice. Li Guojie, an academician of the Chinese Academy of Engineering and a researcher at the Institute of Computing Technology of the Chinese Academy of Sciences, delivered a report entitled "Meta-Thinking on Computing Network". He said that large model pre-training is the main demand for computing power at present, but wide-area distributed computing is not suitable for training large models. Relying on multiple small intelligent computing centers and training large models through distributed computing may not be a solution. The research on computing network needs to have a core abstraction similar to that of web pages, and develop "hyperlinks" into "super tasks". Theoretical abstraction is not about incremental improvements in performance or SOTA rankings, but about achieving breakthroughs in qualitative research first.

In a speech entitled "Design of AI Cluster Communication Technology System", Xu Xiaofei, chief scientist of Huawei Network Technology Laboratory, pointed out that at present, the optimization of communication performance in AI clusters is often concentrated on single-point technology and has failed to form a joint force. From the perspective of giving full play to computing efficiency, Xu Xiaofei shared in detail the practice of systematically designing AI cluster communication technology system, as well as key technologies and evolution routes covering each layer of control plane and data plane. In practice, according to the characteristics of AI traffic, multiple measures should be taken, such as network balancing technology, innovative algorithms of link layer, and mapping control algorithms of the original end, to shorten the communication time as much as possible and reduce the waiting time of calculation. Huawei has innovated the non-uniform Brook algorithm, which effectively reduces the step length of communication. This method is applied to the actual project of customers. In the 8000 card cluster, the measured performance is improved by 70% compared with the traditional algorithm. In response to the problem of asymmetric layered communication, Huawei innovatively realizes the splicing of different domain algorithms based on virtual card numbers, which can improve the performance by 30% to 66%. The rapid development of models requires more flexible connections to adapt. However, traditional physical connections are usually difficult to change. To this end, Huawei continues to explore the use of OXC physical devices in conjunction with TP algorithms to achieve dynamic topology adjustment.

Today, many industry leaders and scientific research projects use cloud computing to meet the needs of business for computing and storage scale expansion. In the future, every company will become a data + AI company, and cloud computing has been practicing the Scaling Law. Wu Jiesheng, vice president of Alibaba Cloud Intelligence Group, head of elastic computing and storage, said in a speech entitled "AI-driven, innovation and development of Alibaba Cloud high-performance computing" that more than 50% of domestic large-scale model startups and more than 80% of technology companies are using Alibaba Cloud. In the face of diverse high-performance computing loads, Alibaba Cloud has built a complete high-performance computing product matrix, using Lingjun Intelligent Computing Cluster to meet the extremely tightly coupled load requirements, elastic high-performance computing EHPC to meet the tightly coupled application load requirements, and EHPC Instant to meet the loosely coupled load requirements. Currently, these solutions have produced practical cases in the fields of generative AI, autonomous driving, life sciences, energy, manufacturing, scientific computing, etc. This is inseparable from Alibaba Cloud's continuous technological innovation, including Panjiu AI computing server, high-performance storage CPFS, cloud infrastructure server chip CIPU, high-performance network HPN 7.0, elastic RDMA connection capabilities, network topology awareness, and so on.

Undoubtedly, artificial intelligence will lead a new round of industrial revolution, and big model technology is the key support to drive this round of artificial intelligence wave. In the speech "Analysis and performance optimization of computing power product selection based on big model [application operation characteristics]", Chen Jian, chairman of Beijing Parallel Technology Co., Ltd., said that big model training is a supercomputing application, and the design of supercomputing systems needs to consider the collaborative design of computing, memory access, high-speed interconnection, etc., to avoid the "barrel" short board, so as to better provide efficient computing power support for big model training. At the same time, affected by various factors such as parameter scale and parallel methods, the operation characteristics of different computing power model training applications are not the same. This requires design based on specific application operation characteristics and selecting a supercomputing architecture computing power product platform that is suitable for it, so as to improve the training performance of big models more accurately and efficiently. Starting from the supercomputing architecture computing power platform, Chen Jian introduced specific methods and implementation plans around how to improve the training and reasoning performance of big models. Parallel Technology will launch the "Computing Sea Plan Phase II" and cooperate with the Inner Mongolia Computing Power Base to build a single large cluster of 100,000 cards, which is scheduled to start external operations in October 2025.

AI+ Towards “Newness”

The plenary session in the second half of the afternoon was hosted by Professor Zhai Jidong, Deputy Director of the CCF High Performance Computing Committee.

Lenovo is optimistic about the future application prospects of large models. In order to meet the needs of artificial intelligence applications, my country is continuously investing heavily in the construction of intelligent computing power. As of May 2024, the scale of computing power that has been built and operated in my country has reached 104 EFLOPS. In a speech entitled "Building a New Quality Computing Power to Cross the Digital Future" - Introduction to Lenovo's Heterogeneous Intelligent Computing Platform, Huang Shan, Director of Strategic Management of Lenovo China Infrastructure Business Group, said that my country's new computing power construction is moving towards scale and multi-heterogeneity. The biggest challenges faced in this process are mainly reflected in the compatibility of software and hardware, the complexity of interoperability design, the low utilization rate of intelligent computing power, fault diagnosis and recovery are the constraints for improving SLA, and the refrigeration technology needs to be continuously upgraded. Lenovo has promoted the development of new quality computing power and accelerated the transition of computing power value by building a comprehensive heterogeneous intelligent computing platform, building a high-performance storage that supports all AI scenarios, providing multi-architecture, low-latency transportation capacity, and building a heterogeneous intelligent computing industry ecological alliance.

Ding Lieyun, an academician of the Chinese Academy of Engineering, a professor at Huazhong University of Science and Technology, and the chief scientist of the National Digital Construction Technology Innovation Center, gave a report entitled "AI Reshaping the Future of Architecture". He said that AI will bring revolutionary changes to the transformation and upgrading of the construction industry. Architectural design is not drawn, but calculated by models, which requires the unification of mathematical logic and formal logic. In the future, construction production will no longer be extensive construction, but will be able to build houses like cars, that is, through modularization, integration, automation, and intelligence, and pre-installation in factories. In practice, Academician Ding Lieyun not only uses intelligent technology in the field of architectural design, but also conducts extensive attempts in medical health and even sports, and has achieved very good results. AI will change the future. Construction products are not only material products formed by reinforced concrete, but also intelligent terminals for buildings that can change people's lifestyles.

Professor Chen Wenguang of the Department of Computer Science at Tsinghua University said in his report titled "Large Model Training on Domestic Supercomputing and Intelligent Computing Platforms" that large-scale pre-trained models represented by GPT are showing rich generation capabilities and have achieved important applications in various industries. At present, most pre-trained models at home and abroad are carried out on NVIDIA's platform. Professor Chen Wenguang has a targeted approach and analyzed the challenges faced by domestic supercomputing and intelligent computing platforms, as well as the progress of related computer system technologies, and introduced the experience of training dense models with hundreds of billions of parameters and sparse models with hundreds of trillions of parameters on two domestic platforms. Practice has proved that domestic supercomputing and intelligent computing platforms can support the training of large models, but there are also limitations. The biggest bottleneck lies in the peak computing power of a single chip. Although it can be alleviated to a certain extent through software optimization, it cannot completely make up for the gap. Domestic platforms still need to make persistent efforts to reach a higher level.

The light of supercomputing continues to shine

The three-day conference is full of exciting content. In addition to the main forum, theme forum, special forum, and enterprise forum, the CCF HPC China2024 Supercomputing Annual Best Application and CCF High Performance Computing Doctoral Dissertation Incentive Program will also be announced. At the same time, a large-scale exhibition will be held on site. More than 100 well-known industry media, including Guangming.com, Huanqiu.com, China Science Daily, Science and Technology Daily, and China Industrial News, will report on the grand event of the conference in all aspects, reveal the latest development trends of high performance computing, and showcase many innovative achievements.

Entering a new era of digital intelligence, the rapid development of artificial intelligence requires faster and more solid computing infrastructure construction, and at the same time requires diversified computing services to be more cost-effective and easier to achieve unified scheduling and management. This not only gives the High Performance Computing Professional Committee of the China Computer Society, CCF HPC China and other organizations a more important historical mission, but also puts forward higher requirements for their future work and development.

Carrying on the past, innovating and improving, and leading the world, CCF HPC China will become an important platform for unleashing the power of high-performance computing and enabling the development of new quality productivity.