Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6116 DataFrame API improvement umbrella ticket (Spark 1.5)
  3. SPARK-7135

Expression for monotonically increasing IDs

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0
    • SQL
    • Spark 1.5 doc/QA sprint

    Description

      Seems like a common use case that users might want a unique ID for each row. It is more expensive to have consecutive IDs, since that'd require two pass over the data. However, many use cases can be satisfied by just having unique ids.

      Attachments

        Issue Links

          is blocked by

          Sub-task - The sub-task of the issue SPARK-7152 Add a Column expression for partition ID

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #5709 [Github] Pull Request #5709 (rxin)

          Activity

            People

              rxin Reynold Xin
              rxin Reynold Xin
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: