Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-2481

HostDatum deltas(previous step statistics) and Metadata expressions

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: hostdb
    • Labels:
      None

      Description

      To allow the usage of previous step statistics(deltas of fetched,unfetced etc) in hostdb. The motivation is usage of this statistics in generate with maxCount expressions.

       

      The solution allows to fill in metadata of hostdatum based on custom JEXL expression using two hostdatum: before update(previousHostDatum) and after update(currentHostDatum)..

      For example to fill in difference in quantity of fetched at round t and t-1 we can use the following expression

      <property>
      <name>hostdb.deltaExpression</name>
      <value>{return new ("javafx.util.Pair","FetchedDelta", currentHostDatum.fetched - previousHostDatum.fetched);}</value>
      </property>

      A pull request will be provided shortly.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                semyon.semyonov@mail.com Semyon Semyonov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: