Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12945

DataFrame API fails to import with pandas 1.0.x

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • 2.32.0
    • 2.34.0
    • dsl-dataframe
    • None

    Description

      The DataFrame API expects pandas.DataFrame to have the attributes "backfill", "bfill", so it fails to import when pandas 1.0.x is installed:

      In [1]: import pandas as pd                                                                                                                                                                                                    [11/1919]
                                                                                                                          
      In [2]: pd.__version__                                    
      Out[2]: '1.0.5'                                           
                                                                                                                          
      In [3]: from apache_beam.dataframe.convert import to_dataframe                                                      
      ---------------------------------------------------------------------------                                         
      AttributeError                            Traceback (most recent call last)                                         
      <ipython-input-3-26ab6682f699> in <module>                
      ----> 1 from apache_beam.dataframe.convert import to_dataframe                                                      
                                                                
      ~/working_dir/beam/sdks/python/apache_beam/dataframe/convert.py in <module>                                         
           30 from apache_beam.dataframe import frame_base      
           31 from apache_beam.dataframe import schemas         
      ---> 32 from apache_beam.dataframe import transforms      
           33                                                   
           34 if TYPE_CHECKING:                                 
      
      ~/working_dir/beam/sdks/python/apache_beam/dataframe/transforms.py in <module>                                      
           30 from apache_beam import transforms                
           31 from apache_beam.dataframe import expressions     
      ---> 32 from apache_beam.dataframe import frames  # pylint: disable=unused-import                                   
           33 from apache_beam.dataframe import partitionings   
           34 from apache_beam.utils import windowed_value      
      
      ~/working_dir/beam/sdks/python/apache_beam/dataframe/frames.py in <module>                                          
          148                                                   
          149                                                   
      --> 150 class DeferredDataFrameOrSeries(frame_base.DeferredFrame):                                                  
          151   def _render_indexes(self):                      
          152     if self.index.nlevels == 1:                   
      
      ~/working_dir/beam/sdks/python/apache_beam/dataframe/frames.py in DeferredDataFrameOrSeries()                       
          297   ffill = _fillna_alias('ffill')                  
          298   bfill = _fillna_alias('bfill')                  
      --> 299   backfill = _fillna_alias('backfill')            
          300   pad = _fillna_alias('pad')                      
          301                                                   
      
      ~/working_dir/beam/sdks/python/apache_beam/dataframe/frames.py in _fillna_alias(method)                             
           94   return frame_base.with_docs_from(pd.DataFrame)(                                                           
           95       frame_base.args_to_kwargs(pd.DataFrame)(    
      ---> 96           frame_base.populate_defaults(pd.DataFrame)(wrapper)))                                             
           97                                                   
           98                                                   
      
      ~/working_dir/beam/sdks/python/apache_beam/dataframe/frame_base.py in wrap(func)                                    
          559   For internal use only. No backwards compatibility guarantees."""                                          
          560   def wrap(func):                                 
      --> 561     base_argspec = getfullargspec(unwrap(getattr(base_type, func.__name__)))                                
          562     if not base_argspec.defaults:                 
          563       return func                                 
      
      AttributeError: type object 'DataFrame' has no attribute 'backfill'
      

      Attachments

        Issue Links

          Activity

            People

              bhulette Brian Hulette
              bhulette Brian Hulette
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h