Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9762

CoreOptions.TMP_DIRS wrongly managed on Yarn

    XMLWordPrintableJSON

Details

    Description

      The issue on Yarn is that it is impossible to have different LOCAL_DIRS on JobManager and TaskManager, despite LOCAL_DIRS value depends on the container.

      The issue is that CoreOptions.TMP_DIRS is configured to the default value during JobManager initialization and added to the configuration object. When TaskManager is launched the appropriate configuration object is cloned with LOCAL_DIRS which makes sense only for Job Manager container. When YARN container with TaskManager from his point of view CoreOptions.TMP_DIRS is always equal either to path in flink.yml or to the or to the LOCAL_DIRS of Job Manager (default behaviour). Is TaskManager’s container do not have an access to another folders, that folders allocated by YARN TaskManager cannot be started.

      Attachments

        Issue Links

          Activity

            People

              Oleksandr Nitavskyi Oleksandr Nitavskyi
              Oleksandr Nitavskyi Oleksandr Nitavskyi
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: