2023 Big Model Industry Review and 2024 Outlook: Challenges and Opportunities for Large Language Models

Large Language Models (LLMs) are changing the way we interact with technology – OpenAI’s GPT family, Bloom, Bard, Bert, LaMDa, LLaMa, and others – and are shaping a future where communicating with a machine is as natural as chatting with a friend. From generating creative content to assisting in advanced research, large-scale language models are being integrated into our daily lives.

2022 by OpenAI’s ChatGPT out of the mountain, bring fire AI big model industry wind mouth, just past 2023 is the big model industry outbreak of the implementation of the year, a large number of big model startups emerged, all walks of life are discussing how big models affect the development of the industry, initiated by the United States, China quickly followed, China and the United States are becoming the world’s two major bases of the big model.

Major large model maps released in recent years:

Transformer, the core technology of the big model, has its unique architectural advantages, and the practical application effect in the mainstream AI application field surpasses the traditional RNN and CNN, which is firstly initiated from the language class model (e.g., ChatGPT), and the fusion multimodal architecture of language text and image graphics was successfully realized in 2023, and various applications based on the Transformer models emerged in waves.

The AI community building the future. The platform where the machine learning community collaborates on models, datasets, and applications.

Hugging face started as a chatbot startup based in New York, they were going to start a chatbot business and then open-sourced a Transformers library on GitHub, although the chatbot business didn’t get off the ground, their library quickly became a huge hit in the machine learning community. They have now shared over 100,000 pre-trained models and 10,000 datasets, turning it into the GitHub of the AI world.

The hard power behind big model technology is big computing power, and it is mainly matrix computing, GPU is the current choice, and Nvidia, the industry-leading GPU research and development company, is the biggest beneficiary. For example: Nvidia H100 is a high-end GPU product for high-performance computing and AI applications, with powerful computing power and high memory capacity, as well as advanced interconnect technology Nvidia’s H100 is a high-end GPU for high-performance computing and AI applications, featuring powerful compute capabilities and high memory capacity, as well as advanced interconnect technology, which compute efficiency in clusters and accelerates compute-intensive tasks such as AI training and inference in areas such as deep learning, computer vision, and natural language processing.

Big model of the two core grip: large amount of data + arithmetic power, the current training of large models of data mainly from the past decades of Internet development of the data accumulated, there is a large amount of data, but the data is mixed, with the promotion of large model applications facing the Internet will be added to the new data will be mainly machine-generated data, it may lead to inbreeding of data, there is no current program to circumvent the program.

The parameters of the big model are getting more and more, now it has reached hundreds of billions of parameters, the ability of the model is getting stronger and stronger, the requirements for the arithmetic are very high, the cost of model training is very high, a training may cost tens of millions of dollars, which is unaffordable for the general size of the company or entity. For hundreds of billions of parameters of large model training, now can use of the GPU is also very limited, almost only NVIDIA’s H100 and more advanced products can be used, so if someone claims to be above the ordinary PC, or cell phone to realize the large model, that is in the playing ball, at best, it is in the inside of the installation of an entrance to ride the wind through the marketing propaganda.

With a large model Transformer, does it mean that previous AI methods such as RNNs and CNNs are completely abandoned? Actually, no. Because, many AI scenes do not need a particularly large model if you use a large model to do not only high cost and the effect is not necessarily good, or use the traditional AI algorithms enough and the effect is good and controllable. AI large model has its advantages for the scene, the traditional AI models and algorithms have the same use of the scene, and the application of the scene is quite a lot.

“Technology is not born for engineers, but for the application”, The 2024 big model LLM core point is whether to find a killer application or killer business model, so that 2024 for the big model is whether to continue to fire down the turn of the year, the current big model in various industries are promoting the application of, but are in the original business based on efficiency and optimization, in the text processing class (including images and video) work outstanding, some of the original need to do things instead of people, but the text processing class work is not a material productive work, and the substantive text class work and people directly related to the substantive process involves the legal responsibility and management authority factors, it must be responsible for the participation of people, so the remaining is the formatting work or the work of the formatting work or the work of the formatting work. The remaining is the formatting work or not so need to have the responsibility of the work (such as chat, content search, decorative painting generation, etc.) AI can be replaced.

Incorporating big models as a foundational capability into browsers and search engines is a typical application scenario, and following the introduction of AI in Microsoft’s Bing, the latest version of Google’s Chrome browser officially introduces generative AI capabilities. In the new Chrome version, generative AI features make it easier for users to get work done on the web: keep their tabs organized, get help when writing content online, and generate custom Chrome backgrounds using AI.

AI big model in the end can become a new business, the current is still in the search and debate, we recognize that the big model is useful, but in the end just as the efficiency of the enabler, can form an independent new business, just like when the WEB Internet just appeared, when we all think that this thing is valuable but do not know how to make money, until Yahoo online advertising business into a business, the Internet has become a new business. After Yahoo made online advertising business into a business, the Internet became a new form of business, and later developed into advertising, games, e-commerce, payment, and several other major business models, which led to the prosperous development of the Internet in the following decades.

Today’s AI big model is also facing the challenge of commercialization, a small fight can not become a business, AI big model of the new industry in the end where to go, kerosene lamp technology expert team believes that 2024 will be a watershed.