[PIG-4809] Implement to collect metric data like getSMMSpillCount() in SparkJobStats

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: spark-branch
• Component/s: spark
• Assignee: Unassigned
• Reporter: liyunzhang
• Created: 15/Feb/16 01:29
• Updated: 15/Feb/16 01:29

Description

In current code, we don't implement to collect metric data in SparkJobStats like:
SparkJobStats#getSMMSpillCount:

    @Override
    public long getSMMSpillCount() {
        throw new UnsupportedOperationException();
    }

We just throw UnsupportedOperationException when this function is called.
We need implement following functions in SparkJobStats if the metric data can be collected in spark mode:

    @Override
    public int getNumberMaps() {
        throw new UnsupportedOperationException();
    }

    @Override
    public int getNumberReduces() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getMaxMapTime() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getMinMapTime() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getAvgMapTime() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getMaxReduceTime() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getMinReduceTime() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getAvgREduceTime() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getMapInputRecords() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getMapOutputRecords() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getReduceInputRecords() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getReduceOutputRecords() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getSMMSpillCount() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getProactiveSpillCountObjects() {
        throw new UnsupportedOperationException();
    }

    @Override
    public long getProactiveSpillCountRecs() {
        throw new UnsupportedOperationException();
    }

  @Override
    public Map<String, Long> getMultiStoreCounters() {
        throw new UnsupportedOperationException();
    }

    @Override
    public Map<String, Long> getMultiInputCounters() {
        throw new UnsupportedOperationException();
    }