Apache Whirr (retired)
  1. Apache Whirr (retired)
  2. WHIRR-342

hadoop/hbase configuration & active roles on a node

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Labels:
      None

      Description

      The following limitations exist with the generation of hadoop-(core|hdfs|mapred).xml and hbase-site.xml (assuming WHIRR-339 applied):

      • they are not generated by all roles (e.g. tasktracker, thrift server, ...), by consequence running these roles by themselves on a node unaccompagnied of a role that does generate them will not work.
      • running two roles on the same node that generate the same files does not work as it should, as the generated contents gets appended twice to the same file, causing non-well-formed XML. This is because of the usage of jcloud's Statements.appendFile.

      The cheapest solution would be to replace Statements.appendFile with something similar but without the 'append' behavior, thus rather a 'Statements.overwriteFile' (not available in jclouds afaics).

      This of course assumes that when different roles are writing the same files, that they put the same contents in them, so that the overwriting does not matter.

      Alternatively, things could be made smarter so that the same configuration is only generated once for all roles.

      1. WHIRR-342.patch
        45 kB
        Karel Vervaeke
      2. WHIRR-342.patch
        42 kB
        Karel Vervaeke
      3. WHIRR-342.patch
        32 kB
        Karel Vervaeke
      4. WHIRR-342.patch
        8 kB
        Karel Vervaeke
      5. WHIRR-342.patch
        8 kB
        Karel Vervaeke

        Issue Links

          Activity

          Bruno Dumon created issue -
          Andrei Savu made changes -
          Field Original Value New Value
          Fix Version/s 0.6.0 [ 12316468 ]
          Andrei Savu made changes -
          Link This issue relates to WHIRR-359 [ WHIRR-359 ]
          Andrei Savu made changes -
          Fix Version/s 0.7.0 [ 12317571 ]
          Fix Version/s 0.6.0 [ 12316468 ]
          Karel Vervaeke made changes -
          Attachment WHIRR-342.patch [ 12496999 ]
          Attachment WHIRR-342.patch [ 12497000 ]
          Tom White made changes -
          Link This issue is related to WHIRR-337 [ WHIRR-337 ]
          Karel Vervaeke made changes -
          Attachment WHIRR-342.patch [ 12497796 ]
          Tom White made changes -
          Link This issue blocks WHIRR-391 [ WHIRR-391 ]
          Karel Vervaeke made changes -
          Attachment WHIRR-342.patch [ 12498868 ]
          Karel Vervaeke made changes -
          Attachment WHIRR-342.patch [ 12498877 ]
          Andrei Savu made changes -
          Link This issue is blocked by WHIRR-400 [ WHIRR-400 ]
          Andrei Savu made changes -
          Link This issue is blocked by WHIRR-400 [ WHIRR-400 ]
          Andrei Savu made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Assignee Karel Vervaeke [ karel1980 ]
          Resolution Fixed [ 1 ]

            People

            • Assignee:
              Karel Vervaeke
              Reporter:
              Bruno Dumon
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development