Solr
  1. Solr
  2. SOLR-3255

OpenExchangeRates.Org Exchange Rate Provider

    Details

      Description

      An exchange rate provider for CurrencyField using the freely available feed from http://openexchangerates.org/

      1. SOLR-3255-OpenExchangeRatesOrgProvider-plugin.jar
        6 kB
        Jan Høydahl
      2. SOLR-3255-noggit-eof-bug.patch
        0.5 kB
        Jan Høydahl
      3. SOLR-3255-3x.patch
        3 kB
        Jan Høydahl
      4. SOLR-3255-3x.patch
        11 kB
        Jan Høydahl
      5. SOLR-3255.patch
        28 kB
        Jan Høydahl
      6. SOLR-3255.patch
        28 kB
        Jan Høydahl
      7. SOLR-3255.patch
        27 kB
        Jan Høydahl
      8. SOLR-3255.patch
        27 kB
        Jan Høydahl

        Issue Links

          Activity

          Hide
          Jan Høydahl added a comment -

          Here's the provider implementation with tests. See http://wiki.apache.org/solr/CurrencyField for documentation. Highlights:

          • Uses open, free exchange rates REST API
          • Plugs into CurrencyField in schema.xml
          • Can load rates json from any URL or through ResourceLoader
          • Configurable refresh of rates, enforces max every 60 min (since that's the update rate of the API)

          This patch also changes the ExchangeRateProvider interface slightly:

          • Instead of listCurrencies() returning FROM,TO pairs (which would be 25.000 lines for all available pairs for this provider, it takes an argument, so that listCurrencies(false) returns a list of supported currencies, while listCurrencies(true) returns list of pairs

          Known limitations/questions:

          • The reflection for the providerClass param uses Class.forName() to instantiate the provider. But then the solr.MyClass alias does not work. How to solve this?
          • Is the correct location o.a.s.schema for these providers or should we make a new package somewhere else?
          Show
          Jan Høydahl added a comment - Here's the provider implementation with tests. See http://wiki.apache.org/solr/CurrencyField for documentation. Highlights: Uses open, free exchange rates REST API Plugs into CurrencyField in schema.xml Can load rates json from any URL or through ResourceLoader Configurable refresh of rates, enforces max every 60 min (since that's the update rate of the API) This patch also changes the ExchangeRateProvider interface slightly: Instead of listCurrencies() returning FROM,TO pairs (which would be 25.000 lines for all available pairs for this provider, it takes an argument, so that listCurrencies(false) returns a list of supported currencies, while listCurrencies(true) returns list of pairs Known limitations/questions: The reflection for the providerClass param uses Class.forName() to instantiate the provider. But then the solr.MyClass alias does not work. How to solve this? Is the correct location o.a.s.schema for these providers or should we make a new package somewhere else?
          Hide
          Jan Høydahl added a comment -

          Slightly improved Noggit JSON parsing loop. Removed a few unnecessary imports. Fixed order of assertEquals() params.

          Show
          Jan Høydahl added a comment - Slightly improved Noggit JSON parsing loop. Removed a few unnecessary imports. Fixed order of assertEquals() params.
          Hide
          Jan Høydahl added a comment -

          Fixed the class loading issue, by using schema.getResourceLoader().findClass() when loading the providerClass.

          Any other comments?

          Show
          Jan Høydahl added a comment - Fixed the class loading issue, by using schema.getResourceLoader().findClass() when loading the providerClass. Any other comments?
          Hide
          Mark Miller added a comment -

          Haven't had a chance to look close, but cool!

          Show
          Mark Miller added a comment - Haven't had a chance to look close, but cool!
          Hide
          Hoss Man added a comment -

          looks pretty cool, but that listAvailableCurrencies smells kind of fishy in general, and with this patch smells even fishier (depending on the arg, it either returs a list of string codes, or a list of string code perumtations with a comma separator?)

          If we're seeing now, with multiple Provider impls, that the API doens't make sense – we should bite the bullet and change it before it's public.

          perhaps two methods: listAvailableCurrencies() that returns a Set<String> and listAvailableConversions that returns Map<String,String> ?

          Show
          Hoss Man added a comment - looks pretty cool, but that listAvailableCurrencies smells kind of fishy in general, and with this patch smells even fishier (depending on the arg, it either returs a list of string codes, or a list of string code perumtations with a comma separator?) If we're seeing now, with multiple Provider impls, that the API doens't make sense – we should bite the bullet and change it before it's public. perhaps two methods: listAvailableCurrencies() that returns a Set<String> and listAvailableConversions that returns Map<String,String> ?
          Hide
          Jan Høydahl added a comment -

          Yea, or just implement the listAvailableCurrencies() for now and let a concrete use case add listAvailableConversions() later if needed.

          Show
          Jan Høydahl added a comment - Yea, or just implement the listAvailableCurrencies() for now and let a concrete use case add listAvailableConversions() later if needed.
          Hide
          Jan Høydahl added a comment -

          Attaching the latest patch, which was just committed to TRUNK. It does:

          • Add the OpenExchangeRatesOrg Provider
          • Cleans up the ExchangeRateProvider. We now return a plain list of currencies supported, not all pairs:
            public Set<String> listAvailableCurrencies()
          • Uses proper resource loader to load provider class with solr.* support
          • Adds tests
          • Adds support for loading json file locally using ResourceLoader or from URL
          Show
          Jan Høydahl added a comment - Attaching the latest patch, which was just committed to TRUNK. It does: Add the OpenExchangeRatesOrg Provider Cleans up the ExchangeRateProvider. We now return a plain list of currencies supported, not all pairs: public Set<String> listAvailableCurrencies() Uses proper resource loader to load provider class with solr.* support Adds tests Adds support for loading json file locally using ResourceLoader or from URL
          Hide
          Jan Høydahl added a comment -

          This is the patch for merge back to branch_3x. ant clean test javadocs-all dist succeeds.

          Despite the freeze, I argue this should go in 3.6 as planned, as it stabilizes the interface for ExchangeRate Providers compared to SOLR-2202, it fixes a classloading bug and gives a 2nd example for how to implement providers. The Wiki docs at http://wiki.apache.org/solr/CurrencyField is already prepared for 3.6

          Show
          Jan Høydahl added a comment - This is the patch for merge back to branch_3x. ant clean test javadocs-all dist succeeds. Despite the freeze, I argue this should go in 3.6 as planned, as it stabilizes the interface for ExchangeRate Providers compared to SOLR-2202 , it fixes a classloading bug and gives a 2nd example for how to implement providers. The Wiki docs at http://wiki.apache.org/solr/CurrencyField is already prepared for 3.6
          Hide
          Robert Muir added a comment -

          I don't think we should be adding new features. But if there is a bugfix, we
          should separate that out into its own issue.

          Show
          Robert Muir added a comment - I don't think we should be adding new features. But if there is a bugfix, we should separate that out into its own issue.
          Hide
          Jan Høydahl added a comment -

          Ok, I committed the stabilization to the interface to branch_3x as part of SOLR-2202. This new patch SOLR-3255-3x.patch is a pure backport of the new OpenExchangeRatesOrgProvider. It's basically adding one class for the provider, with a test case class, as well as a comment in schema.xml.

          If there is interest in shipping this with 3.6 it's a push of a button away

          Show
          Jan Høydahl added a comment - Ok, I committed the stabilization to the interface to branch_3x as part of SOLR-2202 . This new patch SOLR-3255 -3x.patch is a pure backport of the new OpenExchangeRatesOrgProvider . It's basically adding one class for the provider, with a test case class, as well as a comment in schema.xml. If there is interest in shipping this with 3.6 it's a push of a button away
          Hide
          Jan Høydahl added a comment -

          Found a minor bug in noggit parsing, we do not catch the EOF

          INFO: Noggit UNKNOWN_EVENT_ID:EOF
          

          Attached is a simple patch which will be committed now.

          Show
          Jan Høydahl added a comment - Found a minor bug in noggit parsing, we do not catch the EOF INFO: Noggit UNKNOWN_EVENT_ID:EOF Attached is a simple patch which will be committed now.
          Hide
          Jan Høydahl added a comment -

          Attaching a jar which can be added to your class path to get this provider in Solr3.6

          Show
          Jan Høydahl added a comment - Attaching a jar which can be added to your class path to get this provider in Solr3.6

            People

            • Assignee:
              Jan Høydahl
              Reporter:
              Jan Høydahl
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development