Details
-
Task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
After some investigation under the issue https://issues.apache.org/jira/browse/IGNITE-17423 we found out, that simple insert only benchmark has significant difference in performance between key-value and jdbc API.
Step to reproduce:
- checkout https://github.com/gridgain/apache-ignite-3/tree/ignite-17423
- run IgniteBenchmark#jdbcInsert and IgniteBenchmark#kvInsert
The difference up to 3 times: 528us vs 1606us per insert request.
key-value:
Result "org.apache.ignite.internal.configuration.IgniteBenchmark.kvInsert": N = 226939 mean = 528.170 ±(99.9%) 1.877 us/op Histogram, us/op: [ 0.000, 5000.000) = 226851 [ 5000.000, 10000.000) = 65 [10000.000, 15000.000) = 13 [15000.000, 20000.000) = 3 [20000.000, 25000.000) = 1 [25000.000, 30000.000) = 3 [30000.000, 35000.000) = 0 [35000.000, 40000.000) = 1 [40000.000, 45000.000) = 1 Percentiles, us/op: p(0.0000) = 360.448 us/op p(50.0000) = 508.416 us/op p(90.0000) = 604.160 us/op p(95.0000) = 642.048 us/op p(99.0000) = 759.808 us/op p(99.9000) = 3207.414 us/op p(99.9900) = 10262.020 us/op p(99.9990) = 41768.596 us/op p(99.9999) = 45875.200 us/op p(100.0000) = 45875.200 us/op # Run complete. Total time: 00:02:06REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial experiments, perform baseline and negative tests that provide experimental control, make sure the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts. Do not assume the numbers tell you what you want them to tell.Benchmark Mode Cnt Score Error Units IgniteBenchmark.kvInsert sample 226939 528.170 ± 1.877 us/op IgniteBenchmark.kvInsert:kvInsert·p0.00 sample 360.448 us/op IgniteBenchmark.kvInsert:kvInsert·p0.50 sample 508.416 us/op IgniteBenchmark.kvInsert:kvInsert·p0.90 sample 604.160 us/op IgniteBenchmark.kvInsert:kvInsert·p0.95 sample 642.048 us/op IgniteBenchmark.kvInsert:kvInsert·p0.99 sample 759.808 us/op IgniteBenchmark.kvInsert:kvInsert·p0.999 sample 3207.414 us/op IgniteBenchmark.kvInsert:kvInsert·p0.9999 sample 10262.020 us/op IgniteBenchmark.kvInsert:kvInsert·p1.00 sample 45875.200 us/op
jdbc:
Result "org.apache.ignite.internal.configuration.IgniteBenchmark.jdbcInsert": N = 74613 mean = 1606.345 ±(99.9%) 100.395 us/op Histogram, us/op: [ 0.000, 125000.000) = 74572 [ 125000.000, 250000.000) = 40 [ 250000.000, 375000.000) = 0 [ 375000.000, 500000.000) = 0 [ 500000.000, 625000.000) = 0 [ 625000.000, 750000.000) = 0 [ 750000.000, 875000.000) = 0 [ 875000.000, 1000000.000) = 0 [1000000.000, 1125000.000) = 0 [1125000.000, 1250000.000) = 0 [1250000.000, 1375000.000) = 0 [1375000.000, 1500000.000) = 0 [1500000.000, 1625000.000) = 0 [1625000.000, 1750000.000) = 1 [1750000.000, 1875000.000) = 0 Percentiles, us/op: p(0.0000) = 841.728 us/op p(50.0000) = 1177.600 us/op p(90.0000) = 1517.568 us/op p(95.0000) = 1765.376 us/op p(99.0000) = 4488.069 us/op p(99.9000) = 105906.176 us/op p(99.9900) = 153354.240 us/op p(99.9990) = 1707081.728 us/op p(99.9999) = 1707081.728 us/op p(100.0000) = 1707081.728 us/op # Run complete. Total time: 00:02:06 REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial experiments, perform baseline and negative tests that provide experimental control, make sure the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts. Do not assume the numbers tell you what you want them to tell. Benchmark Mode Cnt Score Error Units IgniteBenchmark.jdbcInsert sample 74613 1606.345 ± 100.395 us/op IgniteBenchmark.jdbcInsert:jdbcInsert·p0.00 sample 841.728 us/op IgniteBenchmark.jdbcInsert:jdbcInsert·p0.50 sample 1177.600 us/op IgniteBenchmark.jdbcInsert:jdbcInsert·p0.90 sample 1517.568 us/op IgniteBenchmark.jdbcInsert:jdbcInsert·p0.95 sample 1765.376 us/op IgniteBenchmark.jdbcInsert:jdbcInsert·p0.99 sample 4488.069 us/op IgniteBenchmark.jdbcInsert:jdbcInsert·p0.999 sample 105906.176 us/op IgniteBenchmark.jdbcInsert:jdbcInsert·p0.9999 sample 153354.240 us/op IgniteBenchmark.jdbcInsert:jdbcInsert·p1.00 sample 1707081.728 us/op
Also, in the branch I marked some methods with the JFR events marker and found out, that we have some leaders of the execution time inside the SQL module:
- ExecutionServiceImpl.DistributedQueryManager#submitFragment up to 920us in average for 5s window (see the attached chart)
- org.apache.ignite.internal.sql.engine.util.Commons#parse(java.io.Reader, org.apache.calcite.sql.parser.SqlParser.Config) up to 182us in average for 5s window (see attached chart)
