Baidu CEO Robin Li: The Closed-Source Model Will Continue to Lead in Capabilities

A leaked internal speech by Robin Li recently surfaced. In the internal speech, Robin Li expressed his views on the industry’s hot topics such as the choice between open source and closed source for large models, and whether AI entrepreneurs should focus on models or applications.

Robin Li admitted that there was a very intense discussion within Baidu, Inc. about whether to open source when ERNIE was just released a year ago. The reason why Baidu, Inc. decided not to open source at that time was because it was believed that there would definitely be open-source models in the market, and more than one company would do so. In this situation, it wouldn’t make much difference if Baidu opened sourced or not.

“We are looking at mainstream open source models today: like Llama, Mistral actually have considerable influence. Domestically, BAAI, Baichuan AI, and Alibaba‘s Tongyi are also open source. There is no shortage of our own open source models in this market. If we want to open source, we still need to maintain a set of open source versions ourselves, which is not cost-effective,” said Robin Li.

Robin Li further pointed out that closed-source models will continue to lead in terms of capabilities, rather than just leading temporarily. “Why is this judgment made? Because we believe that the significance of open-source models is not very significant. These open-source models are scattered outside and used for various verification applications on a small scale, without undergoing large-scale validation. Moreover, model openness does not create a situation where ‘many hands make light work.’ This is very different from traditional software open source projects like Linux and Android. Since it’s the model that’s being opened up, we don’t know why the parameters have turned out as they are, making it difficult to achieve ‘many hands make light work.’ Although Llama also encourages everyone to contribute various data and code, in reality, people understand that the main developers are those at Meta; it’s not truly a product developed collaboratively by everyone.”

Robin Li believes that closed-source has a real business model, can make money, and only by making money can it gather computing power and talent. At the same time, closed-source actually has advantages in terms of cost.

“As long as the capabilities are equal, the reasoning cost of the closed-source model must be lower, and the response speed must be faster. Conversely, under equal parameters, the capabilities of the closed-source model are also stronger. Today, whether in China or in America, the strongest basic models are all closed-source, while various small models and best small models are distilled from large models. Models made through dimensionality reduction with large models are better; this will also lead to advantages for closed-source in terms of cost and efficiency,” added Robin Li.

In addition, Robin Li also mentioned in this internal speech that the so-called ‘dual-wheel drive’ talked about by some entrepreneurial companies in modeling is not a good model.

“Both doing modeling and applications will inevitably lead to a dispersion of energy. The energy and resources of a startup company are limited, so it is self-evident that the success rate of doing two things at the same time versus doing only one thing is higher. In any case, we emphasize focus very much, ‘putting all efforts into one hole.’ When resources are limited, it is more important to focus rather than engage in so-called ‘dual-drive’.”

Robin Li emphasized that for AI entrepreneurs, the core competitiveness should not be the model itself, as it is too resource-intensive and requires a very long time to develop. The real advantage for entrepreneurs should lie in knowledge and data in a specific field. ‘If you want to find a ‘yellow pocketless men’s swim trunks’ today, you won’t find them on any e-commerce platform. This kind of demand cannot be met with current technology, but can be solved by large models with domain knowledge. This is an example of how domain knowledge can provide unique value.

SEE ALSO: Baidu’s Robin Li: Ernie Bot 4.0 Has Surpassed GPT-4 in Chinese Language Processing

“There are so many models on the market, big ones, small ones, open source ones, closed source ones. How to use these combinations of models in specific applications requires skill. This is something that entrepreneurs can do and can provide value-added.”

As for concerns from the outside world that if you use a model like ERNIE or a closed-source model and do well with it, they will copy you and eat your lunch, Robin Li believes that this kind of concern is actually unfounded. “Just take a look – in the mobile era, WeChat did not eat up Pinduoduo, Didi also did not become part of Tencent. They each provided their own unique value and have very different competitiveness. Their rise depended on a closed platform within the mobile ecosystem – WeChat. However, they are not afraid of WeChat taking away their business opportunities; therefore there is no need to worry about basic models dominating AI applications.”

“Of course, the most fundamental thing is that the function of ERNIE is the strongest and has the best cost performance. We will continue to invest in it continuously. With this point, all other things can be established.”