Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-80

Counter formatting for the user logs should be pulled out of the public Counters API

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Not A Problem
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Currently the mapred.Counters class includes an obsolete format (makeCompactString) and a format that is only used by the user logs (makeEscapeCompactString). makeCompactString should be deprecated and later removed.The makeEscapeCompactString should be refactored to a non-public class.

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        1162d 17h 7m 1 Harsh J 31/Dec/11 09:56
        Harsh J made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Not A Problem [ 8 ]
        Hide
        Harsh J added a comment -

        This has gone stale as well, not a problem anymore.

        Show
        Harsh J added a comment - This has gone stale as well, not a problem anymore.
        Owen O'Malley made changes -
        Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
        Key HADOOP-4516 MAPREDUCE-80
        Component/s mapred [ 12310690 ]
        Nigel Daley made changes -
        Field Original Value New Value
        Fix Version/s 0.20.0 [ 12313438 ]
        Hide
        Suhas Gogate added a comment -

        adding the requirement here to make the fromEscapedCompactString() API available. It is deprecated in 0.20. Hadoop vaidya contrib project uses it to parse the counter string from job history. (See HADOOP-5582 for more details. It is marked duplicate of this issue.).

        Show
        Suhas Gogate added a comment - adding the requirement here to make the fromEscapedCompactString() API available. It is deprecated in 0.20. Hadoop vaidya contrib project uses it to parse the counter string from job history. (See HADOOP-5582 for more details. It is marked duplicate of this issue.).
        Hide
        Owen O'Malley added a comment -

        The escaped compact format is only used for the log files. If there is another use, then we can make it public. In the mean time it just makes the public interface bigger and more confusing for users.

        However, it goes deeper than that. In particular, embedding various output formats in the Counters -> Groups -> Counters classes isn't necessary or good. It is better to have the different encodings in different classes that use the standard apis.

        Show
        Owen O'Malley added a comment - The escaped compact format is only used for the log files. If there is another use, then we can make it public. In the mean time it just makes the public interface bigger and more confusing for users. However, it goes deeper than that. In particular, embedding various output formats in the Counters -> Groups -> Counters classes isn't necessary or good. It is better to have the different encodings in different classes that use the standard apis.
        Hide
        Amar Kamat added a comment -

        makeCompactString should be deprecated

        +1

        The makeEscapeCompactString should be refactored to a non-public class.

        What will be reason for doing so? There might a user code that might need to stringify a local counter to write it to a file. The api to de-stringify can be used by the applications that process job logs externally and want to reconstruct the counter.

        Show
        Amar Kamat added a comment - makeCompactString should be deprecated +1 The makeEscapeCompactString should be refactored to a non-public class. What will be reason for doing so? There might a user code that might need to stringify a local counter to write it to a file. The api to de-stringify can be used by the applications that process job logs externally and want to reconstruct the counter.
        Owen O'Malley created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development