Under the background of the 2025 target timetable for the high-quality development of computing infrastructure, what is the current situation of the ecological development of China’s generative artificial intelligence (AIGC) services and large model industries? What is the core competitiveness of general model and industry model industry development? In the process of industrial digitalization, how long will the state of "thousand-mode war" of domestic large models last?

To answer these questions, we can look for the answers from the just-concluded China Computer Conference (CNCC2023) in Shenyang, Liaoning. As a high-level and large-scale high-end academic conference in the computer field hosted by china computer federation, this year, with the theme of "developing digital infrastructure and supporting the construction of digital China", nearly 30 of the 130 technical forums and more than 20 special events have deeply analyzed the recent fiery artificial intelligence and big models. The reporter found some opinions and suggestions that can answer these questions in the special forum held at the same time, "The application of super-intelligence fusion AI big model is developing and the new paradigm of efficient computing is developed".

Computing power, algorithm and data —— Pay equal attention to the development of ——AIGC technology and safety.

"No calculation, no model". Since the explosion of Generative Artificial Intelligence Service (AIGC) and GPT large-scale model training, discussions about computing power, algorithms and data have come one after another, and the application of domestic large-scale models in Chinese context has shown a state of "thousand-model war".

This year, China has successively issued the Interim Measures for the Management of Generative Artificial Intelligence Services and the Action Plan for the High-quality Development of Computing Infrastructure, which has made a detailed plan for the computing infrastructure behind the development of AIGC technology and industry. Among them, the Action Plan puts forward quantitative indicators for the development of computing infrastructure by 2025 from four aspects: computing power, carrying capacity, storage capacity and application empowerment. In terms of computing power, the scale of computing power exceeds 300EFLOPS, and the proportion of intelligent computing power reaches 35%; In terms of storage capacity, the total storage capacity exceeds 1800EB, and the advanced storage capacity accounts for more than 30%.

Intelligent computing power is the theme of the above-mentioned special activities. Focusing on the practical problems in the process of algorithm optimization and application of artificial intelligence big model, guests from Industry-University-Research institutions such as China Academy of Sciences, China Institute of Information and Communication, China Renmin University and Beijing Super Cloud Computing Center discussed the key topics such as technology application, personnel training, industry innovation and content security.

"The artificial intelligence big model focuses on solving continuity problems from reasoning, question and answer, detection to description in the process of business training." Zhiwu Lu, a professor and doctoral supervisor at Gaochun School of Artificial Intelligence, Renmin University of China, said that there are still many problems to be solved in the generation ability of multimodal content such as text, pictures, audio and video. "From the business functions such as multi-image understanding, object positioning, OCR, etc., the multi-modal large model needs to be improved and upgraded for task pre-training and fine-tuning of data instructions. When the application scenario falls, it is very important to solve the diverse and complex needs of users through large model training."

"Data is very important for the training and application of the big model. What data and instruction set types are used to make the big model have what kind of capabilities? In essence, it is a research problem of diversity and coverage, which is different from the data of memory learning and deep learning. " Kong Qingchao, an associate researcher at the Institute of Automation, Chinese Academy of Sciences, believes that the development of artificial intelligence models in China still needs to invest in research and development, and whether the application and exploration of industry models in different industries will certainly produce results requires a lot of algorithms for logical reasoning, for example, whether there are more algorithms for the financial industry model to support the generation of investment decision-making reference opinions, and whether such a large model has the ability of iteration and learning, it needs to do sufficient pre-training and technical research and development.

Experts pointed out that data, algorithms and computing power are important components of the new information infrastructure, and the computing power infrastructure is characterized by ubiquity, intelligence, agility, safety, reliability, green and low carbon, which is of great significance for boosting industrial transformation and upgrading and empowering scientific and technological innovation and progress. In terms of generative artificial intelligence technology, we should create and improve the collaborative application ecology of computing power, data and algorithms. Both the general model and the industry model are applying "subtraction" to transform the big model from "toy" to "tool". Let the technology land, let the results generate value, and let the service match the business. This may be the future ecology of the big model that people are discussing.

Wang Jue, deputy director of the Artificial Intelligence Department of computer network information center, believes that the key to the application landing and business adaptation of domestic large models lies in data cleaning. "We are also trying various models. Whether it is accuracy or computational efficiency, the challenge is the stability of the domestic open source model. We have done a lot of work in this regard to promote the development of scientific research in various disciplines through literature data. At the same time, it is also our research direction to make the domestic large model easy to use. "

"Next-generation information technologies such as artificial intelligence, big data and cloud computing have accelerated the development of big models. The research and training of general or industrial large models cannot be separated from a lot of computing power support. " Dr. Chen Jian, Chairman of Parallel Technology, believes that from the perspective of industrial development, computing power is an industrial product, and the demand for computing power in industry model training needs to find the right direction before it can be exerted.

Technology, Application and Ecology —— Improving the accuracy and efficiency of large-scale models

The demand of large-scale model application for high-frequency computing power resources is often closely related to the multi-scenario application ability construction of large-scale model upgrade iteration. At present, what problems does the big model ecology face in the Chinese context, and what deep changes can the interdisciplinary talent training cooperation mechanism and the upstream and downstream of the industrial chain bring to the big model? When answering a reporter’s question, many guests also talked about some industry problems that need to be solved urgently in the service provided by the big model at present.

With the development of AIGC technology in the early days, a large number of "illusion" problems, that is, "serious nonsense", have arisen from various chains and semantic understanding. According to Tian Hui, director of Sunac Center of China Information and Communication Research Institute, the industrial application of emerging technologies must be in a healthy state with Industry-University-Research’s combination. "The division of labor of AIGC technology is more detailed, and the development, deployment and application of industrial applications all require the cooperation of industrial chains. For example, the secondary development of large models, iterative upgrading, and personnel training mechanisms in the fields of operation and maintenance services all need subdivision and cooperation. "

After the AIGC technology industry has entered the state of "thousand-mode war", what is the core competitiveness we ultimately rely on? The industry believes that the development direction or path of AIGC technology industry is to upgrade the core basic abilities of understanding, generation, logical reasoning and memory of large models from the aspect of artificial intelligence technology; In the aspect of scene application landing, strengthen the learning ability of large models and improve the accuracy and efficiency of generating services; In the aspect of industrial ecological construction, the intelligent level and ecological application space of large models will be improved, and practical tools will be built to assist decision-making.

"Beijing Super Cloud Computing Center adopts supercomputing architecture mode to build intelligent computing resources, and integrates the resources of major computing centers distributed in China, which can maximize the use of existing resources and reduce the losses caused by the waste of idle resources. At the same time, it can also effectively meet the high-frequency demand for computing power in artificial intelligence large model training. " Wu Di, the Beijing Super Cloud Computing Center, said that the supercomputing architecture represented by supercomputers can provide more comprehensive and efficient parallel computing capabilities, provide computing service support for large-scale model training, and achieve more efficient model training and better prediction accuracy.

Wang Jue said that the development of the industry model will widen the industry gap without computing power. "It is a research hotspot to make computing resources run these large models better, keep their accuracy or generalization ability unchanged, improve the speed and efficiency of large model training, and truly realize the localization of the whole stack in large model training as soon as possible."

"I hope that the industrial application development of the big model will see the ecology of a hundred flowers, but what needs to be paid attention to is the problem of low-level duplication of work. Whether it is quantity or type, there is still a lot of room for exploration in the products or business forms of large models. " Tian Hui believes that the ultimate development of AIGC technology lies in content services. "How to improve the accuracy and efficiency, whose platform is invested enough, the speed and efficiency of large-scale model training, and stronger application adaptation, who can break through success."