Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16354

Illegal Inputs In LIMIT or TABLESAMPLE

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • None
    • SQL
    • None

    Description

      SELECT * FROM testData TABLESAMPLE (-1 rows)
      SELECT * FROM testData LIMIT -1
      

      Negative values should not be allowed in TABLESAMPLE n ROWS and LIMIT n

      In addition, Spark SQL follows the restriction of LIMIT clause in Hive. The argument to the LIMIT clause must evaluate to a constant value. It can be a numeric literal, or another kind of numeric expression involving operators, casts, and function return values. You cannot refer to a column or use a subquery. Currently, we do not detect whether the expression in LIMIT clause is foldable or not. If non-foldable, we might issue a strange error message. For example,

      SELECT * FROM testData LIMIT rand() > 0.2
      

      Then, a misleading error message is issued, like

      assertion failed: No plan for GlobalLimit (_nondeterministic#203 > 0.2)
      +- Project [key#11, value#12, rand(-1441968339187861415) AS _nondeterministic#203]
         +- LocalLimit (_nondeterministic#202 > 0.2)
            +- Project [key#11, value#12, rand(-1308350387169017676) AS _nondeterministic#202]
               +- LogicalRDD [key#11, value#12]
      
      java.lang.AssertionError: assertion failed: No plan for GlobalLimit (_nondeterministic#203 > 0.2)
      +- Project [key#11, value#12, rand(-1441968339187861415) AS _nondeterministic#203]
         +- LocalLimit (_nondeterministic#202 > 0.2)
            +- Project [key#11, value#12, rand(-1308350387169017676) AS _nondeterministic#202]
               +- LogicalRDD [key#11, value#12]
      

      Attachments

        Activity

          People

            smilegator Xiao Li
            smilegator Xiao Li
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: