Hadoop Common
  1. Hadoop Common
  2. HADOOP-1267

change default config to be single node rather than "local" for both map/reduce and hdfs

    Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: conf
    • Labels:
      None

      Description

      I propose that we change the default config to be set up for a single node rather than the current "local", which uses direct file access and the local job runner.

        Activity

        Hide
        Eli Collins added a comment -

        Out of date

        Show
        Eli Collins added a comment - Out of date
        Hide
        Nigel Daley added a comment -

        From Doug: We could include a confg/pseudo-cluster.xml file, that folks could use with 'bin/hadoop -conf conf/pseudo-cluster.xml ...'.

        From Nige: I suggest something like hadoop-site-localhost.xml or localhost-cluster.xml. It would define, at a minimum, these properties:

        <property>
        <name>fs.default.name</name>
        <value>localhost:9000</value>
        </property>

        <property>
        <name>mapred.job.tracker</name>
        <value>localhost:9001</value>
        </property>

        <property>
        <name>dfs.replication</name>
        <value>1</value>
        </property>

        Show
        Nigel Daley added a comment - From Doug: We could include a confg/pseudo-cluster.xml file, that folks could use with 'bin/hadoop -conf conf/pseudo-cluster.xml ...'. From Nige: I suggest something like hadoop-site-localhost.xml or localhost-cluster.xml. It would define, at a minimum, these properties: <property> <name>fs.default.name</name> <value>localhost:9000</value> </property> <property> <name>mapred.job.tracker</name> <value>localhost:9001</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property>
        Hide
        Doug Cutting added a comment -

        Why? Right now, things work out of the box w/o starting daemons, worrying about firewalls, etc. For applications like Nutch, many (most?) users never bother running a cluster, but they get the comfort that their code is future-proofed for scalability.

        Show
        Doug Cutting added a comment - Why? Right now, things work out of the box w/o starting daemons, worrying about firewalls, etc. For applications like Nutch, many (most?) users never bother running a cluster, but they get the comfort that their code is future-proofed for scalability.

          People

          • Assignee:
            Owen O'Malley
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development