Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25398

Minor bugs from comparing unrelated types

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.3.1
    • Fix Version/s: 2.4.0
    • Component/s: Mesos, Spark Core, YARN
    • Labels:
      None
    • Target Version/s:

      Description

      I noticed a potential issue from Scala inspections, like this clause in LiveEntity.scala around line 586:

       (!acc.metadata.isDefined ||
        acc.metadata.get != Some(AccumulatorContext.SQL_ACCUM_IDENTIFIER))

      The issue is that acc.metadata is Option[String], so can't equal Some[String]. This just meant to be:

       acc.metadata != Some(AccumulatorContext.SQL_ACCUM_IDENTIFIER)

      This may or may not actually cause a bug, but seems worth fixing. And then there are a number of other ones like this, mostly in tests, that might likewise mask real assertion problems.

      Many are, interestingly, flagging items like this on a Seq[String]:

      .filter(_.getFoo.equals("foo"))

      It complains that Any => Any is compared to String. Either it's wrong, or somehow, this is parsed as (_.getFoo).equals("foo")). In any event, easy enough to write this more clearly as:

      .filter(_.getFoo == "foo")

      And so on.

        Attachments

          Activity

            People

            • Assignee:
              srowen Sean R. Owen
              Reporter:
              srowen Sean R. Owen
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: