Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5284 Roll-up of final fixes for managed sort
  3. DRILL-5062

External sort refers to the deprecated HDFS fs.default.name param

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.8.0
    • Fix Version/s: 1.11.0
    • Component/s: None
    • Labels:
      None

      Description

      Running a query that uses the external sort produces the following message in the log file:

      [org.apache.hadoop.conf.Configuration.deprecation] - fs.default.name is deprecated. Instead, use fs.defaultFS
      

      External sort has the following line in the ExternalSortBatch constructor:

          conf.set("fs.default.name", config.getString(ExecConstants.EXTERNAL_SORT_SPILL_FILESYSTEM));
      

      The TestParquetScan class has the same deprecated parameter name.

      Looking elsewhere in Drill, it appears that the proper way to configure HDFS is as follows:

          fsConf.set(FileSystem.FS_DEFAULT_NAME_KEY, ...
      

      That is, use the constant provided by HDFS.

        Attachments

          Activity

            People

            • Assignee:
              paul-rogers Paul Rogers
              Reporter:
              paul-rogers Paul Rogers
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: