Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-42882

Pandas API Coverage Improvements

    XMLWordPrintableJSON

Details

    • Epic
    • Status: Resolved
    • Major
    • Resolution: Resolved
    • 3.4.0
    • None
    • Pandas API on Spark
    • None
    • Pandas API Coverage Improvements

    Description

      Pandas API on Spark aims to make pandas code work on Spark clusters without any changes. So full API coverage has been one of our major goals. 

      Attachments

        1.
        		 Refactor Resampler Sub-task Resolved Ruifeng Zheng
        2.
        		 Implement `ExpandingGroupby.quantile`. Sub-task Resolved Yikun Jiang
        3.
        		 Implement `RollingGroupby.quantile`. Sub-task Resolved Yikun Jiang
        4.
        		 Implement `kendall` and `min_periods` in `Series.corr` Sub-task Resolved Ruifeng Zheng
        5.
        		 Make `ddof` in `DataFrame.var` and `Series.var` accept arbitary integers Sub-task Resolved Ruifeng Zheng
        6.
        		 Implement `numeric_only` and `min_count` in `GroupBy.sum` Sub-task Resolved Ruifeng Zheng
        7.
        		 Make `ddof` in `DataFrame.sem` and `Series.sem` accept arbitary integers Sub-task Resolved Ruifeng Zheng
        8.
        		 Support ps.Index in DataFrame creation Sub-task Resolved Ruifeng Zheng
        9.
        		 ps.DataFrame(data, index) should support the same anchor Sub-task Resolved Ruifeng Zheng
        10.
        		 Implement `min_count` in `GroupBy.max` Sub-task Resolved Ruifeng Zheng
        11.
        		 Implement `ddof` in `DataFrame.cov` Sub-task Resolved Ruifeng Zheng
        12.
        		 Implement Groupby.sem Sub-task Resolved Ruifeng Zheng
        13.
        		 Implement `min_count` in `GroupBy.last` Sub-task Resolved Ruifeng Zheng
        14.
        		 `GroupBy.first` should skip nulls Sub-task Resolved Ruifeng Zheng
        15.
        		 Add resampling to API references Sub-task Resolved Ruifeng Zheng
        16.
        		 Implement `GroupBy.prod`. Sub-task Resolved Artsiom Yudovin
        17.
        		 Implement `GroupBy.nth`. Sub-task Resolved Ruifeng Zheng
        18.
        		 Make Series.mode apply PandasMode Sub-task Resolved Ruifeng Zheng
        19.
        		 Rename `_MissingPandasXXX` as `MissingPandasXXX` Sub-task Resolved Ruifeng Zheng
        20.
        		 Make `pearson` correlation in `DataFrame.corr` support missing values and `min_periods` Sub-task Resolved Ruifeng Zheng
        21.
        		 Make `ddof` in `GroupBy.std`, `GroupBy.var` and `GroupBy.sem` accept arbitary integers Sub-task Resolved Ruifeng Zheng
        22.
        		 Implement `Series.searchsorted`. Sub-task Resolved Ruifeng Zheng
        23.
        		 Improve the precision of `product` for intergral inputs Sub-task Resolved Ruifeng Zheng
        24.
        		 Implement `ddof` in `Series.cov` Sub-task Resolved Ruifeng Zheng
        25.
        		 Implement DataFrame.mode Sub-task Resolved Ruifeng Zheng
        26.
        		 Make `_reduce_for_stat_function` in `groupby` accept `min_count` Sub-task Resolved Ruifeng Zheng
        27.
        		 Implement `min_count` in `GroupBy.first` Sub-task Resolved Ruifeng Zheng
        28.
        		 Remove `pyspark.pandas.ml` Sub-task Resolved Ruifeng Zheng
        29.
        		 Implement `Rolling.quantile`. Sub-task Resolved Yikun Jiang
        30.
        		 Implement `Expanding.quantile`. Sub-task Resolved Yikun Jiang
        31.
        		 Refactor expanding and rolling test for function with input Sub-task Resolved Yikun Jiang
        32.
        		 Make `ddof` in `DataFrame.std` and `Series.std` accept arbitary integers Sub-task Resolved Ruifeng Zheng
        33.
        		 Implement `GroupBy.quantile`. Sub-task Resolved Yikun Jiang
        34.
        		 Implement `spearman` and `kendall` in `DataFrame.corrwith` Sub-task Resolved Ruifeng Zheng
        35.
        		 Implement `min_count` in GroupBy.min Sub-task Resolved Ruifeng Zheng
        36.
        		 Implement `kendall` correlation in `DataFrame.corr` Sub-task Resolved Ruifeng Zheng
        37.
        		 Make `spearman` correlation in `DataFrame.corr` support missing values and `min_periods` Sub-task Resolved Ruifeng Zheng

        Activity

          People

            Unassigned Unassigned
            XinrongM Xinrong Meng
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: