[IMPALA-8204] Buildall.sh should check the right Impala-lzo branch

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Infrastructure
• Epic Color: ghx-label-6
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 15/Feb/19 00:06
• Updated: 23/Dec/20 20:56
• Resolved: 23/Dec/20 20:56

Description

Recently, I encountered some build failures in Jenkins jobs for building patches in branch-2.x. They're due to the wrong Impala-lzo branch (i.e. should use 2.x/cdh5-trunck instead of master).

For example, this is a recent failure:
https://jenkins.impala.io/job/cherrypick-2.x-and-test/641 ->
https://jenkins.impala.io/job/parallel-all-tests/4992 ->
https://jenkins.impala.io/job/ubuntu-16.04-build-only/5395
 
The error messages:

[ 50%] Building CXX object CMakeFiles/impalalzo.dir/hdfs-lzo-text-scanner.cc.o
/home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc: In member function ‘virtual impala::Status impala::HdfsLzoTextScanner::GetNextInternal(impala::RowBatch*)’:
/home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:140:17: error: ‘class impala::HdfsScanNodeBase’ has no member named ‘UpdateRemainingScanRangeSubmissions’
scan_node_->UpdateRemainingScanRangeSubmissions(-1);
^
/home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc: In static member function ‘static impala::Status impala::HdfsLzoTextScanner::LzoIssueInitialRangesImpl(impala::HdfsScanNodeBase*, const std::vector<impala::HdfsFileDesc*>&)’:
/home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:168:14: error: ‘class impala::HdfsScanNodeBase’ has no member named ‘UpdateRemainingScanRangeSubmissions’
scan_node->UpdateRemainingScanRangeSubmissions(header_ranges.size());
^
In file included from /home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scanner.h:30:0,
from /home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-text-scanner.h:22,
from /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.h:9,
from /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:22:
/home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:169:59: error: no matching function for call to ‘impala::HdfsScanNodeBase::AddDiskIoRanges(std::vector<impala::io::ScanRange*>&)’
RETURN_IF_ERROR(scan_node->AddDiskIoRanges(header_ranges));
^
/home/ubuntu/tmp.CCVBRWuYra/be/src/common/status.h:291:36: note: in definition of macro ‘RETURN_IF_ERROR’
::impala::Status __status__ = (stmt); \
^
/home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:169:59: note: candidates are:
RETURN_IF_ERROR(scan_node->AddDiskIoRanges(header_ranges));
^
/home/ubuntu/tmp.CCVBRWuYra/be/src/common/status.h:291:36: note: in definition of macro ‘RETURN_IF_ERROR’
::impala::Status __status__ = (stmt); \
^
In file included from /home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scanner.h:31:0,
from /home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-text-scanner.h:22,
from /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.h:9,
from /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:22:
/home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scan-node-base.h:232:18: note: virtual impala::Status impala::HdfsScanNodeBase::AddDiskIoRanges(const std::vector<impala::io::ScanRange*>&, int)
virtual Status AddDiskIoRanges(const std::vector<io::ScanRange*>& ranges,
^
/home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scan-node-base.h:232:18: note: candidate expects 2 arguments, 1 provided
/home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scan-node-base.h:237:17: note: impala::Status impala::HdfsScanNodeBase::AddDiskIoRanges(const impala::HdfsFileDesc*)
inline Status AddDiskIoRanges(const HdfsFileDesc* file_desc) WARN_UNUSED_RESULT {
^
/home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scan-node-base.h:237:17: note: no known conversion for argument 1 from ‘std::vector<impala::io::ScanRange*>’ to ‘const impala::HdfsFileDesc*’
In file included from /home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scanner.h:30:0,
from /home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-text-scanner.h:22,
from /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.h:9,
from /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:22:
/home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc: In member function ‘impala::Status impala::HdfsLzoTextScanner::IssueFileRanges(const char*)’:
/home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:200:49: error: no matching function for call to ‘impala::HdfsScanNodeBase::AddDiskIoRanges(std::vector<impala::io::ScanRange*>)’
vector<ScanRange*>(1, zero_offset_range)));
^
/home/ubuntu/tmp.CCVBRWuYra/be/src/common/status.h:291:36: note: in definition of macro ‘RETURN_IF_ERROR’
::impala::Status __status__ = (stmt); \
^
/home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:200:49: note: candidates are:
vector<ScanRange*>(1, zero_offset_range)));
^
/home/ubuntu/tmp.CCVBRWuYra/be/src/common/status.h:291:36: note: in definition of macro ‘RETURN_IF_ERROR’
::impala::Status __status__ = (stmt); \
^
In file included from /home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scanner.h:31:0,
from /home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-text-scanner.h:22,
from /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.h:9,
from /home/ubuntu/Impala-lzo/hdfs-lzo-text-scanner.cc:22:
/home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scan-node-base.h:232:18: note: virtual impala::Status impala::HdfsScanNodeBase::AddDiskIoRanges(const std::vector<impala::io::ScanRange*>&, int)
virtual Status AddDiskIoRanges(const std::vector<io::ScanRange*>& ranges,
^
/home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scan-node-base.h:232:18: note: candidate expects 2 arguments, 1 provided
/home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scan-node-base.h:237:17: note: impala::Status impala::HdfsScanNodeBase::AddDiskIoRanges(const impala::HdfsFileDesc*)
inline Status AddDiskIoRanges(const HdfsFileDesc* file_desc) WARN_UNUSED_RESULT {
^
/home/ubuntu/tmp.CCVBRWuYra/be/src/exec/hdfs-scan-node-base.h:237:17: note: no known conversion for argument 1 from ‘std::vector<impala::io::ScanRange*>’ to ‘const impala::HdfsFileDesc*’
At global scope:
cc1plus: error: unrecognized command line option "-Wno-unused-local-typedef" [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/impalalzo.dir/build.make:62: recipe for target 'CMakeFiles/impalalzo.dir/hdfs-lzo-text-scanner.cc.o' failed
make[6]: *** [CMakeFiles/impalalzo.dir/hdfs-lzo-text-scanner.cc.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/impalalzo.dir/all' failed
make[5]: *** [CMakeFiles/impalalzo.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make[4]: *** [all] Error 2
Error in ../Impala-lzo/build.sh at line 38: "${MAKE_CMD:-make}"
CMakeFiles/impala-lzo.dir/build.make:57: recipe for target 'CMakeFiles/impala-lzo' failed
make[3]: *** [CMakeFiles/impala-lzo] Error 2
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/impala-lzo.dir/all' failed
make[2]: *** [CMakeFiles/impala-lzo.dir/all] Error 2
CMakeFiles/Makefile2:79: recipe for target 'CMakeFiles/impala-lzo.dir/rule' failed
make[1]: *** [CMakeFiles/impala-lzo.dir/rule] Error 2
Makefile:129: recipe for target 'impala-lzo' failed
make: *** [impala-lzo] Error 2
Error in /home/ubuntu/tmp.CCVBRWuYra/bin/make_impala.sh at line 180: ${MAKE_CMD} ${MAKE_ARGS} ${MAKE_TARGETS}
Error in bin/jenkins/build-only.sh at line 28: bin/bootstrap_build.sh

 
Job ubuntu-16.04-build-only just run buildall.sh and use the existing Impala-lzo dir that points at master branch. Thus cause the failure. We can either checkout to the right impala-lzo branch in the Jenkins job or in buildall.sh.

Issue Links

• is fixed by: IMPALA-9709 Remove Impala-lzo support