Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-1251

Python 3 Support

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sdk-py-core
    • Labels:
      None

      Description

      I have been trying to use google datalab with python3. As I see there are several packages that does not support python3 yet which google datalab depends on. This is one of them.

      https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6

        Attachments

          Issue Links

          1.
          Support Python native types in Beam typehints Sub-task Open Udi Meiri

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.5h
          2.
          Redesign test_split_at_fraction_exhaustive tests for Python 3 Sub-task In Progress Unassigned

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4.5h
          3.
          Several VcfIO tests fail in Python 3 with TypeError: cannot use a string pattern on a bytes-like object Sub-task Open Simon  
          4.
          Support DoFns with Keyword-only arguments in Python 3. Sub-task In Progress yoshiki obata

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 8h
          5.
          Using --save_main_session fails on Python 3 when main module has superclass constructor calls. Sub-task Open Valentyn Tymofieiev

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 3.5h
          6.
          Dill fails to pickle avro.RecordSchema classes on Python 3. Sub-task Open Valentyn Tymofieiev

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 7.5h
          7.
          Add Streaming wordcount test to Dataflow ValidatesContainer test suite Sub-task Open Unassigned  
          8.
          Design Py3-compatible typehints annotation support in Beam 3. Sub-task Open Udi Meiri

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 11h 50m
          9.
          Rename ToStringCoder into ToBytesCoder Sub-task Open Francesco Perera  
          10.
          Add ValidatesRunner test suite for Flink on Python 3. Sub-task Open Valentyn Tymofieiev

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2.5h
          11.
          Enable Python3 tests for Spark Sub-task Open Kyle Weaver  
          12.
          Clean up Python 2 codepaths once Beam no longer supports Python 2. Sub-task Open Unassigned  
          13.
          Improve Avro IO integration test coverage on Python 3. Sub-task Open Valentyn Tymofieiev

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 5h 10m
          14.
          Add SDK harness containers for Py 3.6, Py 3.7 Sub-task Open Hannah Jiang  
          15.
          deadlock using save_main_session and logging Sub-task Open Valentyn Tymofieiev  
          16.
          Accept Py3 wheels in SDK harness container. Sub-task Open Unassigned  
          17.
          Implement support of PEP 484 annotations for user functions in transforms such as ParDo, Combine in Py3. Sub-task Open Udi Meiri  
          18.
          Migrate to "typing" module typing types in Beam typehints (on Py2 and Py3). Sub-task Open Udi Meiri  
          19.
          Use a Python3-compatible profiler in apache_beam.utils.profiler Sub-task Open Unassigned  
          20.
          Add Python 2 deprecation warnings starting from 2.17.0 release. Sub-task Open Unassigned  
          21.
          Generate Python SDK docs using Python 3 Sub-task Open Unassigned  
          22.
          Run pylint in Python 3 Sub-task Open Unassigned  
          23.
          Add a Python 3 test scenario for MongoDB IO Sub-task Open Yichi Zhang  

            Activity

              People

              • Assignee:
                tvalentyn Valentyn Tymofieiev
                Reporter:
                eyad.alsibai@gmail.com Eyad Sibai
              • Votes:
                39 Vote for this issue
                Watchers:
                63 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 343h 20m
                  343h 20m