Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-581

java: add reducer that separates keys and values when map output is pairs

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0
    • java
    • None
    • Reviewed

    Description

      We should add a Pair<K,V> class, implementing SpecificRecord, that combines instances of two schemas (specific or generic). Pairs would be compared by key, ignoring value. The template for its schema would be:

      {"type": "record", "name": "org.apache.avro.mapred.Pair", "fields":[
        {"name": "key", "type":" <<insert key schema here>>},
        {"name": "value", "order": "ignore", "type": <<insert value schema>>}
      ]}
      

      When map outputs are instances of this class, a reducer may be used whose reduce method is something like:

      public abstract void reduce(K key, Iterable<V> values);

      Attachments

        1. AVRO-581.patch
          94 kB
          Doug Cutting
        2. AVRO-581.patch
          87 kB
          Doug Cutting
        3. AVRO-581.patch
          85 kB
          Doug Cutting
        4. AVRO-581.patch
          83 kB
          Doug Cutting
        5. weather.avro
          0.3 kB
          Doug Cutting
        6. weather-sorted.avro
          0.3 kB
          Doug Cutting

        Issue Links

          Activity

            People

              cutting Doug Cutting
              cutting Doug Cutting
              Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: