Table of Contents , Decision Table Performance Optimization

Benchmarks and Performance

We have already discussed how to create and run tests in OpenL Studio. Providing Unit Tests will give you some assurance that your engine will behave the way you want it. But how fast will it perform. The processing speed is an important attribute of system's overall quality. There is no use of the smartest rule engine that produces results in 10 min when the data changes every 10 sec.

While other vendors advertise their performance on some benchmarks that may or may not have anything to do with your particular implementation, we give the benchmarks to you. The first step in any performance improvement is to know your performance. If your engine already performs fast enough, why spend time on it's improvement? On another hand, if it is too slow, may be some other technology should be considered or some more advanced deployment techniques with load balancing should be used.

The advantage to have performance knowledge at the early stages of the project is that you can take some steps in advance, for example, redesign some classes, and the earlier these steps are done the cheaper is their cost.

In OpenL Tablets, everything that can be run can be benchmarked too. The benchmark icon will appear next to the method. By clicking the icon you will start the benchmarking process. The benchmarking process is adaptive and will automatically adjust the number of runs to produce a benchmark. The run will last 4-5 seconds. The following conditions should be in place to produce a meaningful result:

the method should not have any side effects

the method should not have any printouts, they affect performance the way that render results unusable; other i/o procedures are allowed if they are necessary for the method run, but the result will be affected strongly by i/o performance - therefore it is not recommended too; the best measurements of the engine performance are done when prepared data is already stored in memory - otherwise you might end up measuring data access performance instead of the engine's performance.

during benchmark run other computer tasks should be suspended - make sure that the CPU activity before the run is close to zero

in any case, the measurement is a statistical process - the results precision is about 10%, which is usually enough for any kind of estimate; it makes sense to run the same benchmark a few times

Displaying and Comparing Benchmark Results

The benchmark results are displayed in a table. The Benchmark View keeps all the benchmark results during session life and gives you the option to compare selected results:

As you can see, all the benchmark results here consist of results of the test runs. Because OpenL Tablets knows the structure of the Test Run, it is able to display an average time per Unit Run. In case of other methods it will show the same time for Runs/sec and Units/sec. If you want to perform the tests with the finer granularity you will need to explore org.openl.benchmark package APIs.

The comparison table shows the relative performance (per unit) of the different runs. You can compare any set of benchmarks. You can even use it to compare the performance of different rule engines - make sure you make them public especially if OpenL is on top!