Member: science + computing ag, BMW AG
Featured Standard: ASAM ODS
Challenge: Performance is crucial for the user acceptance ofmeasurement data management (MDM) systems. Before establishing openMdM as a standard framework for ASAM ODS based applications, BMW wanted to test the performance of the underlying ODS server in combination with the openMdM application model and the openMdM API.
Solution: Performance of HighQSoft Avalon ASAM ODS servers was tested automatically in several test setups focussing on tree navigation and mass data retrieval. An openMDM benchmark module was developed for the handling of both, the randomized access of data and the sampling of response times.
Key Benefits: the results of the performance tests showed that the tested ASAM ODSserver performed reliably and provided useful hints on how to best use and configure odS servers when implementing an openMDMapplication.
The good and reliable server performance and the versatility of configuration and data storage patterns of ASAMODS were sound confirmations for our decision using ASAM ODS standard in context with the openMDM framework.
Dr. Reinhard Hallermayer, Project Manager Testing Systems, BMW AG
ASAMODS standard has been in use at BMW formany years for storing and accessing measurement data. BMW analysed the available openMDM framework to evaluate the applicability of this application framework based on ASAM ODS. As aconsequence the openMDM framework was established as an internal platform for application development and BMW joined the openMDM community in 2012.
science + computing has been working on ASAMODS and openMdM based applications for many years now and runs and maintains many ASAM ODS servers in various settings as well as being well versed in performance tuning.
The main objective was to prove the performance of the ASAMODS server in the context of the open- MDM framework, the API and the application model. The goal was to provide a guideline for future in-house developments on how to use and configure the ASAM ODS server for high performance. Performance tests were automated to be able to repeat them at any time with different data or parameters.
s+c developed an openMDM module for benchmarking, which simulates typical user requests through the openMDMAPI. Various parameters can be varied within the benchmark module. The tests are carried out automatically, response times are measured and stored. Two test cases were implemented:
About 12,000 measurements were generated based on existing real life data, a total volume of almost 12 GB, with the largest measurements being 20 MB in size. Two identical test environments were set up for testing two different AvalonODS server modes. In both modes descriptive data are stored in the oracle database:
Various tests were carried out in close collaboration with BMW. The test systems were based on Windows 2008 R2 servers hosting all server components. Windows 7 clients running the openMDM client and the benchmark module obtained access through the corporate network.
Performing a load test with 50 parallel users accessing ASAMODS servers randomly every 1 to 60 seconds produced very good system performance. 90% of the responses were faster than 250 msec and 99 per cent faster than 500 msec, the prerequisite for a fluent user experience.
Random access to descriptive data (valuematrix) showed up to be mostly independent of server mode and configuration. Increasing numbers of entities only had minor influence on the performance, showing a steep slope at low numbers, a wide plateau and a slow slope at very high numbers of entities (see Figure 1).
Accessing mass data (all values of ameasurement) in mixed mode turned out to be much faster than in pure mode, reaching a speed enhancement factor of 8 at very high numbers of entities. Access times using mixed mode showed a much slower slope depending on number of entities than using pure mode (see Figure 2)
The dependence of response times of the time between data accesses was another interesting result. When accessing stored result objects randomly every 2 to 30 minutes, the average response time as well as the maximum response time were almost doubled compared to random access every 2 to 30 seconds. this might be explained by the caching effects of the involved server components.
Good and reliable ASAMODS server performance is a crucial factor for users of data management tools. The acceptance of a tool used every day by many users is essentially influenced by the response times experienced with it. New in-house MDM developments at BMW will profit from the knowledge how to use, how to configure and how to test and optimize the performance of ODS servers