[IMPALA-11152] impala::CheckLogSize() could log error messages indefinitely

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 4.1.0
• Fix Version/s: Impala 4.1.0
• Component/s: Backend
• Epic Color: ghx-label-12
• Assignee: Riza Suminto
• Reporter: Fang-Yu Rao
• Created: 24/Feb/22 23:59
• Updated: 08/Mar/22 21:22
• Resolved: 08/Mar/22 21:22

Description

We found in a Jenkins run that impala::CheckLogSize() could output error messages indefinitely if 'log_path' does not exist (for some reason) when FileSystemUtil::ApproximateFileSize() is called.

E0222 13:45:31.906388 12146 logging.cc:236] Failed to check log file size: Path does not exist!
I0222 13:45:32.906513 12146 status.cc:129] Path does not exist!
    @          0x1fa92c9  impala::Status::Status()
    @          0x299005e  impala::FileSystemUtil::ApproximateFileSize()
    @          0x1fa5129  impala::CheckLogSize()
    @          0x1f9cf38  LogMaintenanceThread()
    @          0x1fa39c4  boost::detail::function::void_function_invoker0<>::invoke()
    @          0x23c8733  boost::function0<>::operator()()
    @          0x2a9e71e  impala::Thread::SuperviseThread()
    @          0x2aa7156  boost::_bi::list5<>::operator()<>()
    @          0x2aa707a  boost::_bi::bind_t<>::operator()()
    @          0x2aa703b  boost::detail::thread_data<>::run()
    @          0x43c2a70  thread_proxy
    @     0x7f86e5af3e24  start_thread
    @     0x7f86e250b34c  __clone
E0222 13:45:32.955483 12146 logging.cc:236] Failed to check log file size: Path does not exist!

In the context of end-to-end tests, this would result in a timeout after 7200s. It may be good that we stop outputting error messages for the same 'log_path' if the reason of error is always the same.