Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Implemented
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Component/s: Table API & SQL
    • Labels:
      None

      Description

      1. Introduce FlinkStatistic class, which is subclass of Calcite Statistic.
      2. Integrate FlinkStatistic with FlinkTable

        Issue Links

          Activity

          Hide
          fhueske Fabian Hueske added a comment -

          Implemented for 1.3.0 with ae0fbff76f327c008bdbf02cac0067bab507a04f

          Show
          fhueske Fabian Hueske added a comment - Implemented for 1.3.0 with ae0fbff76f327c008bdbf02cac0067bab507a04f
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3197

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3197
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fhueske commented on the issue:

          https://github.com/apache/flink/pull/3197

          merging

          Show
          githubbot ASF GitHub Bot added a comment - Github user fhueske commented on the issue: https://github.com/apache/flink/pull/3197 merging
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fhueske commented on the issue:

          https://github.com/apache/flink/pull/3197

          Thanks for the update @beyond1920!
          +1 to merge

          Show
          githubbot ASF GitHub Bot added a comment - Github user fhueske commented on the issue: https://github.com/apache/flink/pull/3197 Thanks for the update @beyond1920! +1 to merge
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user beyond1920 commented on the issue:

          https://github.com/apache/flink/pull/3197

          @fhueske , thanks for your review. I already modify code based on your advice and rebase the pr.

          Show
          githubbot ASF GitHub Bot added a comment - Github user beyond1920 commented on the issue: https://github.com/apache/flink/pull/3197 @fhueske , thanks for your review. I already modify code based on your advice and rebase the pr.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fhueske commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3197#discussion_r101022558

          — Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/stats/FlinkStatistic.scala —
          @@ -0,0 +1,87 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.flink.table.plan.stats
          +
          +import java.lang.Double
          +import java.util.

          {Collections, List}

          +
          +import org.apache.calcite.rel.

          {RelCollation, RelDistribution}

          +import org.apache.calcite.schema.Statistic
          +import org.apache.calcite.util.ImmutableBitSet
          +
          +/**
          + * The class is responsible for provide statistics for flink table.
          + *
          + * @param tableStats
          + */
          +class FlinkStatistic(tableStats: Option[TableStats]) extends Statistic {
          +
          + /**
          + * Get table stats
          + *
          + * @return table stats
          + */
          + def getTableStats: TableStats = tableStats.getOrElse(null)
          +
          + /**
          + * Get stats of specified column
          + *
          + * @param columnName which column to get stats
          + * @return stats of specified column
          + */
          + def getColumnStats(columnName: String) = tableStats match

          { + case Some(tStats) => tStats.colStats.getOrElse(columnName, null) + case None => null + }

          +
          + /**
          + * Get number of rows in the table
          + *
          + * @return number of rows in the table
          + */
          + override def getRowCount: Double = tableStats match

          { + case Some(tStats) => tStats.rowCount.toDouble + case None => null + }

          +
          + override def getCollations: List[RelCollation] = Collections.emptyList()
          +
          + override def isKey(columns: ImmutableBitSet): Boolean = false
          +
          + override def getDistribution: RelDistribution = null
          +
          +}
          +
          +/**
          + * Utility functions regarding FlinkStatistic
          + */
          +object FlinkStatistic {
          — End diff –

          Rename to `FlinkStatistics`

          Show
          githubbot ASF GitHub Bot added a comment - Github user fhueske commented on a diff in the pull request: https://github.com/apache/flink/pull/3197#discussion_r101022558 — Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/stats/FlinkStatistic.scala — @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.plan.stats + +import java.lang.Double +import java.util. {Collections, List} + +import org.apache.calcite.rel. {RelCollation, RelDistribution} +import org.apache.calcite.schema.Statistic +import org.apache.calcite.util.ImmutableBitSet + +/** + * The class is responsible for provide statistics for flink table. + * + * @param tableStats + */ +class FlinkStatistic(tableStats: Option [TableStats] ) extends Statistic { + + /** + * Get table stats + * + * @return table stats + */ + def getTableStats: TableStats = tableStats.getOrElse(null) + + /** + * Get stats of specified column + * + * @param columnName which column to get stats + * @return stats of specified column + */ + def getColumnStats(columnName: String) = tableStats match { + case Some(tStats) => tStats.colStats.getOrElse(columnName, null) + case None => null + } + + /** + * Get number of rows in the table + * + * @return number of rows in the table + */ + override def getRowCount: Double = tableStats match { + case Some(tStats) => tStats.rowCount.toDouble + case None => null + } + + override def getCollations: List [RelCollation] = Collections.emptyList() + + override def isKey(columns: ImmutableBitSet): Boolean = false + + override def getDistribution: RelDistribution = null + +} + +/** + * Utility functions regarding FlinkStatistic + */ +object FlinkStatistic { — End diff – Rename to `FlinkStatistics`
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fhueske commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3197#discussion_r101022463

          — Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/stats/FlinkStatistic.scala —
          @@ -0,0 +1,87 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.flink.table.plan.stats
          +
          +import java.lang.Double
          +import java.util.

          {Collections, List}

          +
          +import org.apache.calcite.rel.

          {RelCollation, RelDistribution}

          +import org.apache.calcite.schema.Statistic
          +import org.apache.calcite.util.ImmutableBitSet
          +
          +/**
          + * The class is responsible for provide statistics for flink table.
          + *
          + * @param tableStats
          + */
          +class FlinkStatistic(tableStats: Option[TableStats]) extends Statistic {
          — End diff –

          Rename to `FlinkStatistitcs`

          Show
          githubbot ASF GitHub Bot added a comment - Github user fhueske commented on a diff in the pull request: https://github.com/apache/flink/pull/3197#discussion_r101022463 — Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/stats/FlinkStatistic.scala — @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.plan.stats + +import java.lang.Double +import java.util. {Collections, List} + +import org.apache.calcite.rel. {RelCollation, RelDistribution} +import org.apache.calcite.schema.Statistic +import org.apache.calcite.util.ImmutableBitSet + +/** + * The class is responsible for provide statistics for flink table. + * + * @param tableStats + */ +class FlinkStatistic(tableStats: Option [TableStats] ) extends Statistic { — End diff – Rename to `FlinkStatistitcs`
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user beyond1920 opened a pull request:

          https://github.com/apache/flink/pull/3197

          FLINK-5567 [Table API & SQL]Introduce and migrate current table statistics to FlinkStatistics

          This pr includes two commits, the first commit is Related to https://github.com/apache/flink/pull/3196(url), the second commit is to introduce and migrate current table statistics to FlinkStatistics. So please focus on second commit.
          The main changes including:
          1. Introduce FlinkStatistic class, which is an implementation of Calcite Statistic.
          2. Integrate FlinkStatistic with FlinkTable.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/alibaba/flink flink-5567

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3197.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3197


          commit cadc16eefb0e0a9002e536a48b4b9f6824b6ab23
          Author: 槿瑜 <jinyu.zj@alibaba-inc.com>
          Date: 2017-01-24T06:34:01Z

          Introduce structure to hold table and column level statistics

          commit 56c51b0f8d7983b8593946f64ece2b4881f0d723
          Author: 槿瑜 <jinyu.zj@alibaba-inc.com>
          Date: 2017-01-24T06:57:08Z

          ntroduce and migrate current table statistics to FlinkStatistics


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user beyond1920 opened a pull request: https://github.com/apache/flink/pull/3197 FLINK-5567 [Table API & SQL] Introduce and migrate current table statistics to FlinkStatistics This pr includes two commits, the first commit is Related to https://github.com/apache/flink/pull/3196 (url), the second commit is to introduce and migrate current table statistics to FlinkStatistics. So please focus on second commit. The main changes including: 1. Introduce FlinkStatistic class, which is an implementation of Calcite Statistic. 2. Integrate FlinkStatistic with FlinkTable. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alibaba/flink flink-5567 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3197.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3197 commit cadc16eefb0e0a9002e536a48b4b9f6824b6ab23 Author: 槿瑜 <jinyu.zj@alibaba-inc.com> Date: 2017-01-24T06:34:01Z Introduce structure to hold table and column level statistics commit 56c51b0f8d7983b8593946f64ece2b4881f0d723 Author: 槿瑜 <jinyu.zj@alibaba-inc.com> Date: 2017-01-24T06:57:08Z ntroduce and migrate current table statistics to FlinkStatistics

            People

            • Assignee:
              jinyu.zj jingzhang
              Reporter:
              ykt836 Kurt Young
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development