Thrift
  1. Thrift
  2. THRIFT-1107

improvement for compiler-generated python for 'None' object comparisons

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7
    • Component/s: Python - Compiler
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      The python code generator produces several python statements, especially the write() methods' per-field code, that compares something to None using '!= None', when it is more efficient to use the 'is not None' expression.

      From what I understand, in python it's almost always true that (x != None) == (x is not None), but the actual implementation and intent is very different. The '!= None' comparison does a by-value comparison that does much more work than an object identity 'is not None' comparison does.

      The actual performance impact isn't much, but I benchmarked the performance of 'x is not None' to 'x != None' and got some interesting results. In python 2.4, 2.7 and 3.1, it's about 2-3 times as fast to use 'is not None' over '!= None'.

      I'll attach a patch to switch to 'is not None', and attach a simple benchmark test script exercising 'is not None' vs. '!= None' and post the performance measurements to this ticket.

      These URLs are somewhat relevant about this specific issue in general:

        Activity

        Hide
        Will Pierce added a comment -

        Attaching benchmark test script: test_isnotnone.py

        This takes a single cmdline argument, the # of millions of iterations to execute. It prints out confluence/jira style markup with the benchmark results comparing "is not None" to "!= None"

        I will post that output in a moment, for python 2.4, 2.7, and 3.1.

        Show
        Will Pierce added a comment - Attaching benchmark test script: test_isnotnone.py This takes a single cmdline argument, the # of millions of iterations to execute. It prints out confluence/jira style markup with the benchmark results comparing "is not None" to "!= None" I will post that output in a moment, for python 2.4, 2.7, and 3.1.
        Hide
        Will Pierce added a comment -

        Output from benchmark script (test_isnotnone.py attached):

        % for i in python2.4 python2.7 python3.1; do $i ./test_isnotnone.py 125; done

        Testing python v2.4.6, 125.0 million iterations

        Code Time Per Iteration (microseconds) Total Test Elapsed Time (seconds)
        x != None 0.08892 usec/iteration 11.115 total sec elapsed
        x is not None 0.05089 usec/iteration 6.361 total sec elapsed

        Testing python v2.7.0, 125.0 million iterations

        Code Time Per Iteration (microseconds) Total Test Elapsed Time (seconds)
        x != None 0.06088 usec/iteration 7.611 total sec elapsed
        x is not None 0.03261 usec/iteration 4.077 total sec elapsed

        Testing python v3.1.2, 125.0 million iterations

        Code Time Per Iteration (microseconds) Total Test Elapsed Time (seconds)
        x != None 0.08650 usec/iteration 10.813 total sec elapsed
        x is not None 0.02976 usec/iteration 3.720 total sec elapsed
        Show
        Will Pierce added a comment - Output from benchmark script (test_isnotnone.py attached): % for i in python2.4 python2.7 python3.1; do $i ./test_isnotnone.py 125; done Testing python v2.4.6, 125.0 million iterations Code Time Per Iteration (microseconds) Total Test Elapsed Time (seconds) x != None 0.08892 usec/iteration 11.115 total sec elapsed x is not None 0.05089 usec/iteration 6.361 total sec elapsed Testing python v2.7.0, 125.0 million iterations Code Time Per Iteration (microseconds) Total Test Elapsed Time (seconds) x != None 0.06088 usec/iteration 7.611 total sec elapsed x is not None 0.03261 usec/iteration 4.077 total sec elapsed Testing python v3.1.2, 125.0 million iterations Code Time Per Iteration (microseconds) Total Test Elapsed Time (seconds) x != None 0.08650 usec/iteration 10.813 total sec elapsed x is not None 0.02976 usec/iteration 3.720 total sec elapsed
        Hide
        Will Pierce added a comment -

        Attached small patch to compiler: THRIFT-1107.compiler_generate_py_is_not_none.patch

        It changes 4 lines in compiler/cpp/src/generate/t_py_generator.cc from "!= None" to "is not None".

        Show
        Will Pierce added a comment - Attached small patch to compiler: THRIFT-1107 .compiler_generate_py_is_not_none.patch It changes 4 lines in compiler/cpp/src/generate/t_py_generator.cc from "!= None" to "is not None".
        Hide
        Bryan Duxbury added a comment -

        I just committed this. Thanks Will!

        Show
        Bryan Duxbury added a comment - I just committed this. Thanks Will!
        Hide
        Hudson added a comment -

        Integrated in Thrift #98 (See https://hudson.apache.org/hudson/job/Thrift/98/)
        THRIFT-1107. py: improvement for compiler-generated python for 'None' object comparisons

        This patch switches from 'x != None' to 'x is not None' for a small performance boost.

        Patch: Will Pierce

        Show
        Hudson added a comment - Integrated in Thrift #98 (See https://hudson.apache.org/hudson/job/Thrift/98/ ) THRIFT-1107 . py: improvement for compiler-generated python for 'None' object comparisons This patch switches from 'x != None' to 'x is not None' for a small performance boost. Patch: Will Pierce

          People

          • Assignee:
            Will Pierce
            Reporter:
            Will Pierce
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development