Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1998

Framework for common implementation of HA commands

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: HA
    • Labels:
      None

      Description

      @SuppressWarnings("unchecked")
          public void updateShareLib(HttpServletRequest request, HttpServletResponse response) throws IOException {
              JSONArray jsonArray = new JSONArray();
              JobsConcurrencyService jc = Services.get().get(JobsConcurrencyService.class);
              if (jc.isAllServerRequest(request.getParameterMap())) {
                  Map<String, String> servers = jc.getOtherServerUrls();
                  for (String otherUrl : servers.values()) {
                      // It's important that we specify ALL_SERVERS_PARAM=false, so that other oozie server should not call other oozie
                      //servers to update sharelib (and creating an infinite recursion)
                      String serverUrl = otherUrl + "/v2/admin/" + RestConstants.ADMIN_UPDATE_SHARELIB + "?"
                              + RestConstants.ALL_SERVER_REQUEST + "=false";
                      try {
                          Reader reader = AuthUrlClient.callServer(serverUrl);
                          JSONObject json = (JSONObject) JSONValue.parse(reader);
                          jsonArray.add(json);
                      }
                      catch (Exception e) {
                          JSONObject errorJson = new JSONObject();
                          errorJson.put(JsonTags.SHARELIB_UPDATE_HOST, otherUrl);
                          errorJson.put(JsonTags.SHARELIB_UPDATE_STATUS, e.getMessage());
                          JSONObject newJson = new JSONObject();
                          newJson.put(JsonTags.SHARELIB_LIB_UPDATE, errorJson);
                          jsonArray.add(newJson);
                      }
                  }
                  //For current server
                  JSONObject newJson = new JSONObject();
                  newJson.put(JsonTags.SHARELIB_LIB_UPDATE, updateLocalShareLib(request));
                  jsonArray.add(newJson);
                  sendJsonResponse(response, HttpServletResponse.SC_OK, jsonArray);
              }
              else {
                  JSONObject newJson = new JSONObject();
                  newJson.put(JsonTags.SHARELIB_LIB_UPDATE, updateLocalShareLib(request));
                  sendJsonResponse(response, HttpServletResponse.SC_OK, newJson);
              }
          }
      
          @SuppressWarnings("unchecked")
          private JSONObject updateLocalShareLib(HttpServletRequest request) {
              ShareLibService shareLibService = Services.get().get(ShareLibService.class);
              JSONObject json = new JSONObject();
              json.put(JsonTags.SHARELIB_UPDATE_HOST, ConfigUtils.getOozieEffectiveUrl());
              try {
                  json.putAll(shareLibService.updateShareLib());
                  json.put(JsonTags.SHARELIB_UPDATE_STATUS, "Successful");
              }
              catch (Exception e) {
                  json.put(JsonTags.SHARELIB_UPDATE_STATUS, e.getClass().getName() + ": " + e.getMessage());
              }
              return json;
          }
      
      

      Currently for each HA command, Oozie need to propagate http request to all servers.
      Each of this input request calls ZK to get list of HA servers and call all other server to get response.
      Current server combines response from all other server + current server and send back response to user.

      In current implementation, each command have their own implementation (leading to multiple duplicate code).
      To minimize duplicate code and ease of implementation of HA command, we can have framework which can does above steps.

        Attachments

          Activity

            People

            • Assignee:
              puru Purshotam Shah
              Reporter:
              puru Purshotam Shah
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: