Bigtop
  1. Bigtop
  2. BIGTOP-689

Oozie use of /var/tmp in some CentOS versions create issues

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.3.0
    • Fix Version/s: 0.5.0
    • Component/s: RPM
    • Labels:
      None

      Description

      It appears that centos is configured by default to clear /var/tmp every 30 days

      Oozie creates some JAR files there at startup and uses them for the duration of the Oozie server process.

      Oozie should use a different TMP dir that is not cleaned up.

      1. BIGTOP-689.patch.txt
        0.6 kB
        Roman Shaposhnik

        Activity

        Hide
        Harsh J added a comment -

        I posted that in response to the below as it seemed slightly apt for it:

        Now that I thought about it, shouldn't it be fixed upstream via introduction of something like OOZIE_TMP location and a proper handling of its cleanup/creation phase in upstream strartup scripts?

        Show
        Harsh J added a comment - I posted that in response to the below as it seemed slightly apt for it: Now that I thought about it, shouldn't it be fixed upstream via introduction of something like OOZIE_TMP location and a proper handling of its cleanup/creation phase in upstream strartup scripts?
        Hide
        Roman Shaposhnik added a comment -

        Thanks for the heads-up, but I'm slightly confused – is there any way the upstream ticket could affect Oozie in Bigtop?

        Show
        Roman Shaposhnik added a comment - Thanks for the heads-up, but I'm slightly confused – is there any way the upstream ticket could affect Oozie in Bigtop?
        Hide
        Harsh J added a comment -

        Upstream ticket for Roman's "proper way" recommendation is possibly OOZIE-711.

        Show
        Harsh J added a comment - Upstream ticket for Roman's "proper way" recommendation is possibly OOZIE-711 .
        Hide
        Alejandro Abdelnur added a comment -

        +1

        Show
        Alejandro Abdelnur added a comment - +1
        Hide
        Andrew Bayer added a comment -

        +1

        Show
        Andrew Bayer added a comment - +1
        Hide
        Roman Shaposhnik added a comment -

        We can do a trivial change like the one in the attached patch, but my fear is that since it is a Catalina-wide tmp directory won't Catalina itself clog /var/lib/oozie with all sort of junk that Oozie scripts then would have to clean up?

        Now that I thought about it, shouldn't it be fixed upstream via introduction of something like OOZIE_TMP location and a proper handling of its cleanup/creation phase in upstream strartup scripts?

        Show
        Roman Shaposhnik added a comment - We can do a trivial change like the one in the attached patch, but my fear is that since it is a Catalina-wide tmp directory won't Catalina itself clog /var/lib/oozie with all sort of junk that Oozie scripts then would have to clean up? Now that I thought about it, shouldn't it be fixed upstream via introduction of something like OOZIE_TMP location and a proper handling of its cleanup/creation phase in upstream strartup scripts?
        Hide
        Alejandro Abdelnur added a comment -

        This seems to affect RightScale CentOS.

        We would need to change /usr/lib/oozie/bin/oozie-env.sh file and change the value of {{CATALINA_TMPDIR|| to a location where there is no cleanup daemon. Just keep in mind that currently Oozie does not clean up files written in there, relies on the OS restarts for that.

        In addition, we should get the Oozie daemons to do the clean up of this files on stop.

        Show
        Alejandro Abdelnur added a comment - This seems to affect RightScale CentOS. We would need to change /usr/lib/oozie/bin/oozie-env.sh file and change the value of {{CATALINA_TMPDIR|| to a location where there is no cleanup daemon. Just keep in mind that currently Oozie does not clean up files written in there, relies on the OS restarts for that. In addition, we should get the Oozie daemons to do the clean up of this files on stop.

          People

          • Assignee:
            Roman Shaposhnik
            Reporter:
            Alejandro Abdelnur
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development