Application Story
Using Large Language Models to Analyze Measurement Data in ASAM ODS Format
Summary
In the automotive industry, the rapid evaluation of vast amounts of Measurement data facilitates a fast resolution of field and development issues. The integration of the Big Data format ASAM ODS for time series data and the Spark technology enables quick and highly parallelized data pro-cessing, facilitating the extraction of necessary information and the execution of AI analyses.
Traditionally, such analyses required expertise in programming and machine learning. However, the growing capabilities of Large Language Models (LLMs), like the popular ChatGPT, empower non-programmers to generate code independently. Our company (NorCom Information Technology GmbH & Co. KGaA) in collaboration with a German car manufacturer, developed a chatbot app ca-pable of translating user queries from natural language into the desired visualizations, along with explanations of the computation process.
Even in its early stages, the app allows engineers without Data Science background to quickly gen-erate custom visualizations, providing insights into one or more Measurement channels across plenty of Measurement files. The primary benefit of such a tool lies in its accessibility, as it enables personnel without programming expertise to run analyses and generate visualization.
Initial situation
The evaluation of large amounts of Measurement data, especially in the automotive industry, poses both challenges and significant opportunities, for example, for the fast identification and resolution of problems in the field and in development. The combination of the Big Data format ASAM ODS (Standard format proposed by the ASAM Organization for highly parallelized computations) with the Spark technology enables rapid and highly parallelized processing of data to generate the desired results quickly and efficiently and then carry out AI analyses.
Our company (NorCom Information Technology GmbH & Co. KGaA) in collaboration with a German car manufacturer has developed many apps on our DaSense platform, which are able to perform such AI analyses. The focus is mainly on analyzing large amounts of Measurement data (time se-ries) with focus on automated anomaly detection and root cause analysis. The goal is to resolve field and development issues more quickly and enable faster reporting.
Our DSL library (an optimized Python library to read and process time series data in ASAM ODS format in a highly parallelized way) facilitates the processing of this kind of data by simply defining Measurement channels. The results can then be used to create visualizations or for AI purposes.
However, writing code to perform such analyses on Measurement data in ASAM ODS format was only possible for programmers and data scientists and not for automotive engineers.
NorCom is committed to supporting open, standardized formats such as ASAM ODS and constantly strives to assist our customers by implementing innovative AI-based solutions. As part of this initia-tive, we are pleased to enable engineers to ‘talk’ to large amounts of Measurement data in natural language using the latest LLM technology.
Oleg (Chief Technology Officer, NorCom Information Technology)
Solution
Nowadays, Large Language Models (LLM) can generate not only natural language but also code in different programming languages that can be executed in most cases without or with few adjust-ments. To increase the number of potential users, our company in collaboration with a German car manufacturer has developed a chatbot app (also called AI Assistant) also for Measurement data that integrates a Large Language Model (LLM). This LLM can understand the documentation of our DSL library (written in natural language), generate the corresponding Python code, and execute it in the background directly. The results of the interaction with the chatbot are visualizations described in the user query.
The app incorporates a user-friendly GUI and takes inspiration from the preexisting chatbot app for documents:
The full result page looks like in the following picture:
Challenges
Although LLMs can generate code quite reliably nowadays and a lot of research effort is put in this direction, developers must face the following challenges:
- Data sensitivity. Companies do not want their sensible data to go to an external server as happens for commercial LLM (e.g. ChatGPT). The solution was to send only the user query and the DSL documentation to the LLM in the cloud. The data is processed locally with the code obtained from the LLM as response. Alternatively, it is also possible to create a com-pletely on premises solution with a local GPU.
- Data security. The chatbot app must be robust against, for example, prompt injection at-tacks where malicious people can send dangerous queries to the LLM that can lead to data loss or other harmful actions.
- Explainability. One of the greatest challenges of AI in the automotive industry is to explain the generated output. However, state-of-the-art AI models are typically black boxes. Large Language Models (LLMs) are no exception to this, and a lot of effort must be put into en-suring that the output aligns with the user's expectations. At this early stage of our chatbot app, for instance, the user can view the generated code, but more work must be done in this direction.
- Guiding the user in formulating the question in the proper way. Every user has a different writing style that may affect the result quality. Therefore, it is important to provide some ex-amples showing how a typical user query should look like.
- Computation time. Depending on the size of the measurement data, the number of parame-ters of the LLM and the computational resources assigned to the Spark cluster, the genera-tion of the visualizations can take from 30 seconds up to a few minutes. The Big Data for-mat ASAM ODS is essential here to achieve a high degree of parallelization and, as a re-sult, the fastest possible response time of the chatbot. The future development for this chatbot app is focused on enhancing its capacity to create increas-ingly intricate and diverse visualizations and analyses. However, many challenges must be solved on this path, for example, the need to comprehend more and more complex and detailed documen-tation.
Business Benefits
Even at this early stage of development, our chatbot app is already able to support the root cause analysis, for example, by visualizing the driving profile of a vehicle during an endurance test. The simple usability of the chatbot app made this kind of tools accessible to more and more automotive engineers of the customer company, who can now autonomously run AI-based analyses on Measurement data in ASAM ODS format without the support of specialized programmers and data scientists saving both time and costs without affecting the quality.
The LLM Models are relatively new technology with big room for improvement. Their rapid and con-tinuous advancement suggests that in the future, generating code for increasingly complex anal-yses and visualizations will be feasible. This implies additional business benefits, such as a faster and more efficient workflow based on a single chatbot for every type of query on data in ASAM ODS format instead of many highly specialized apps.
BACK TO OVERVIEW
Follow us on
