Netease Digital Sail Introduces Conversational BI for Vertical Large-Scale Models

On August 10th, Netease Digital Sail released a conversational BI (Business Intelligence) product called “YouShu ChatBI” in the AIGC technical direction. Compared to currently popular conversational products, the main emphasis of YouShu ChatBI is on the attribute of “data trustworthiness”.

Netease Digital Sail’s General Manager, Yu Lihua, stated that in the digital era, it has become a norm for multiple operational aspects to rely on data for decision-making. The demand for enterprise data analysis has significantly increased.

Taking a customer of Netease Digital Sail as an example, their monthly data analysis demand exceeds 200. Different positions such as planning, operation, user experience, and QA have all raised the need for using data. Among them, the proportion of temporary or urgent demands related to operations is high. However, due to the specialization of the data analysis process and the scarcity of analytical talents, traditional data analysis appears time-consuming and inefficient.

Although AI products can help improve efficiency, the credibility of their answers is a big issue. Netease Digital Sail stated that ChatGPT-like products cannot provide completely accurate responses for two main reasons. Firstly, these types of products are better at handling tasks related to natural language text data and are not specifically designed for data analysis. Secondly, general large-scale models may fabricate facts, creating an “AI illusion,” which could be a fatal problem in the field of business intelligence if fabricated fields occur during data analysis.

So, one of the top priorities for the ChatBI team is to explore the possibility of combating the ‘AI illusion’ in the field of data analysis.

SEE ALSO: NetEase Cloud Music Car Version Is Available on BYD DiLink App Market

Yu Lihua said, “The emergence of ‘AI illusions’ is due to factors such as insufficient training data and encoding/decoding errors between text and representation. Therefore, the team needs to focus on four aspects – demand understanding, process verification, user intervention, and product operation – in order to build a reliable ChatBI.”

Specifically, Netease Digital Sail needs to leverage the language understanding capability of large models to first conduct user demand analysis and assist BI novice users in determining whether the data retrieval steps are correct based on the analysis. Process verification refers to using NL2SQL capabilities based on large models for review. According to Yu Lihua, in order to enhance NL2SQL capabilities, Netease Digital Sail has customized and optimized over 300,000 different types of queries and SQLs. Currently, the tuned NL2SQL domain model has achieved a level comparable to GPT-3.5. User intervention involves structuring data models and query conditions that users can switch between. In terms of product operation, an operational feedback mechanism has been established where users can provide feedback on data accuracy. Administrators improve data reliability through an operational knowledge base, tagging, and optimizing bad cases.

In practical applications, taking the sales department scenario of a large chain supermarket as an example, when product personnel input “I want to see the profit of each month in the first half of this year in the North China region”, ChatBI can provide corresponding results and describe the logic and steps of the query in natural language. Professionals who are proficient in SQL can click on the “More” button to view the corresponding SQL. If there is a logical error, for example, if the user wants to see data with “order date” in the first half of the year but AI filters by “delivery date”, users can click on “modify query conditions” for correction.

From the perspective of product development, which side does Netease Digital Sail currently lean towards more: general large-scale models or vertical industry large-scale models?

Research Institute, and General Manager of Netease Digital Sail, Wang Yuan, stated to media such as Jiemian News that “in principle, our current situation is that we are developing both general large-scale models and vertical large-scale models. There is a supportive relationship between the two. However, the ultimate output is focused on vertical models applied in our two main areas of interest: software development and data analysis.”

He stated that in order to develop vertical models effectively, the team itself needs a “common foundation”, which is the NetEase “Yu Yan” model. Currently, this model is mainly developed by NetEase‘s core laboratory and AI teams from market research, and later on, more teams from the group will join together to build a large-scale model foundation for NetEase Group.