Application Story
ASAM ODS Server Performance Tests
Summary
Challenge: Performance is crucial for the user acceptance of Measurement data management (MDM) systems. Before establishing openMdM as a Standard framework for ASAM ODS based applications, BMW wanted to test the performance of the underlying ODS server in combination with the openMdM application model and the openMdM API.
Solution: Performance of HighQSoft Avalon ASAM ODS servers was tested automatically in several test setups focussing on tree navigation and mass data retrieval. An openMDM benchmark module was developed for the handling of both, the randomized access of data and the sampling of response times.
Key Benefits: the results of the performance tests showed that the tested ASAM ODSserver performed reliably and provided useful hints on how to best use and configure ODS servers when implementing an openMDMapplication.
The good and reliable server performance and the versatility of configuration and data storage patterns of ASAM ODS were sound confirmations for our decision using ASAM ODS Standard in context with the openMDM framework.
Dr. Reinhard Hallermayer, Project Manager Testing Systems, BMW AG
Situation
ASAM ODS Standard has been in use at BMW formany years for storing and accessing Measurement data. BMW analysed the available openMDM framework to evaluate the applicability of this application framework based on ASAM ODS. As aconsequence the openMDM framework was established as an internal platform for application development and BMW joined the openMDM community in 2012.
science + computing has been working on ASAM ODS and openMdM based applications for many years now and runs and maintains many ASAM ODS servers in various settings as well as being well versed in performance tuning.
Challenges
The main objective was to prove the performance of the ASAM ODS server in the context of the open- MDM framework, the API and the application model. The goal was to provide a guideline for future in-house developments on how to use and configure the ASAM ODS server for high performance. Performance tests were automated to be able to repeat them at any time with different data or parameters.
Success Strategy
s+c developed an openMDM module for benchmarking, which simulates typical user requests through the openMDM API. Various parameters can be varied within the benchmark module. The tests are carried out automatically, response times are measured and stored. Two test cases were implemented:
- Listing all elements above a measurement after randomly navigating there
- Loading all mass data above a measurement after randomly navigating there
About 12,000 measurements were generated based on existing real life data, a total volume of almost 12 GB, with the largest measurements being 20 MB in size. Two identical test environments were set up for testing two different Avalon ODS server modes. In both modes descriptive data are stored in the oracle Database:
- “Pure mode” with mass data stored as blobs in the oracle database
- “Mixed mode” with mass data stored as files on a NAS file server
Various tests were carried out in close collaboration with BMW. The test systems were based on Windows 2008 R2 servers hosting all server components. Windows 7 clients running the openMDM client and the benchmark module obtained access through the corporate network.
Performing a load test with 50 parallel users accessing ASAM ODS servers randomly every 1 to 60 seconds produced very good system performance. 90% of the responses were faster than 250 msec and 99 per cent faster than 500 msec, the prerequisite for a fluent user experience.
Random access to descriptive data (valuematrix) showed up to be mostly independent of server mode and configuration. Increasing numbers of entities only had minor influence on the performance, showing a steep slope at low numbers, a wide plateau and a slow slope at very high numbers of entities (see Figure 1).
Accessing mass data (all values of a Measurement) in mixed mode turned out to be much faster than in pure mode, reaching a speed enhancement factor of 8 at very high numbers of entities. Access times using mixed mode showed a much slower slope depending on number of entities than using pure mode (see Figure 2)
The dependence of response times of the time between data accesses was another interesting result. When accessing stored result objects randomly every 2 to 30 minutes, the average response time as well as the maximum response time were almost doubled compared to random access every 2 to 30 seconds. this might be explained by the caching effects of the involved server components.
Bussiness Benefits
Good and reliable ASAM ODS server performance is a crucial factor for users of data management tools. The acceptance of a tool used every day by many users is essentially influenced by the response times experienced with it. New in-house MDM developments at BMW will profit from the knowledge how to use, how to configure and how to test and optimize the performance of ODS servers
BACK TO OVERVIEW
Follow us on
MEET US AT THE
ASAM is hosting the ASAM General Assembly (members only) and the ASAM Technical Seminar (membership not required). Join us
on: Apr 16 - 17, 2024
in: Hoehenkirchen, Germany | online
Register
