Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-153

Build should detect invalid archives in the dl directory

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: general
    • Labels:
      None

      Description

      The build isn't user friendly if you have an invalid tarball in your dl directory (eg because you cntrl-c'd the build while it was running or it just got a bad file). It would be useful if the build did some basic verification of the tarballs, eg checked they were valid archive files, or (perhaps optionally) checked the md5 sums.

        Activity

        Hide
        anatoli.fomenko Anatoli Fomenko added a comment -

        The following components have available md5s for the downloaded tarballs, and they can have md5 check performed post-download:
        Hadoop, Pig, Hive, Sqoop, Oozie, Whirr, Mahout, Flume, Bigtop-jsvc, Bigtop-tomcat

        The remaining components may need checking for validity of downloaded tar.gz:
        Zookeeper, HBase, Giraph, Hue, DataFu

        Show
        anatoli.fomenko Anatoli Fomenko added a comment - The following components have available md5s for the downloaded tarballs, and they can have md5 check performed post-download: Hadoop, Pig, Hive, Sqoop, Oozie, Whirr, Mahout, Flume, Bigtop-jsvc, Bigtop-tomcat The remaining components may need checking for validity of downloaded tar.gz: Zookeeper, HBase, Giraph, Hue, DataFu
        Hide
        cos Konstantin Boudnik added a comment -

        checksums seems like a good solution where they exist. However, even a trivial check of the timestamps - e.g. creation/modification time - might provide a good safeguard against using stale source code tarballs.

        Show
        cos Konstantin Boudnik added a comment - checksums seems like a good solution where they exist. However, even a trivial check of the timestamps - e.g. creation/modification time - might provide a good safeguard against using stale source code tarballs.
        Hide
        anatoli.fomenko Anatoli Fomenko added a comment -

        +1 on Cos' comment

        Show
        anatoli.fomenko Anatoli Fomenko added a comment - +1 on Cos' comment
        Hide
        rvs Roman Shaposhnik added a comment -

        Anatoli, would you want to take a stab at it?

        Show
        rvs Roman Shaposhnik added a comment - Anatoli, would you want to take a stab at it?

          People

          • Assignee:
            anatoli.fomenko Anatoli Fomenko
            Reporter:
            eli Eli Collins
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development