Uploaded image for project: 'OODT (Retired)'
  1. OODT (Retired)
  2. OODT-605

Upgrade the CAS-Product web application to use JAX-RS

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.6
    • 0.7
    • product server

    Description

      This is the parent JIRA issue for adding JAX-RS support to the CAS-Product web application (webapp/fmprod).

      The primary goal is to use Apache CXF's JAX-RS implementation to provide all of the functionality of the existing RSS, RDF and data servlets in the CAS-Product web application, i.e. to provide various ways of accessing File Manager products and product sets using JAX-RS.

      After completing the primary goal, a secondary goal would be to extend the service to provide additional ways of accessing File Manager products (e.g. providing different output formats for metadata, and using additional parameters to find and filter lists of products and product groups).

      I'd like to volunteer to work on this and I'm hoping to do it as a Google Summer of Code project.

      I'll use this JIRA issue to record the schedule and plans for the project. I'm hoping that if other members of the OODT community are interested they can find information here and make comments, suggestions, etc. and see how everything's progressing.

      Attachments

        Issue Links

          Activity

            rlaidlaw Ross Laidlaw added a comment - - edited

            Below is a draft project plan:

            April 29th to May 3rd (1 week)

            • Project planning
            • Submit proposal to GSoC

            May 4th to May 26th (3 weeks)

            • Preparation work
              • read up more on Apache CXF, JAX-RS
              • finalize plans after discussion with OODT community

            May 27th to June 9th (2 weeks)

            • Add draft designs (e.g. UML) to JIRA
            • Update project POM file (webapp/fmprod/pom.xml) to include Apache CXF
            • Add new skeleton Java package for the project code (e.g. named org.apache.oodt.cas.product.service)

            June 10th to June 30th (3 weeks)

            • Add Java classes and XML configuration files to allow products and metadata to be served as resources via JAX-RS RESTful interface (URLs). (i.e. port DataDeliveryServlet and DatasetDeliveryServlet functionality to the new cas.product.service package)
              • products retrieved by product ID
              • product sets retrieved by product type
              • returns product(s) and metadata as application/zip

            July 1st to July 21st (3 weeks)

            • Add Java classes and XML configuration files to allow information on products to be output as RDF streams via JAX-RS RESTful interface (URLs). (i.e. port RDFProductServlet and RDFDatasetServlet functionality to the new cas.product.service package)
              • products retrieved by product ID
              • product sets retrieved by product type (plus filtering, queries)
              • add capability to retrieve information on current product transfers (as currently available in the RSS servlet)
              • returns information on products, product sets or product transfers as RDF streams

            July 22nd to August 11th (3 weeks)

            • Add Java classes and XML configuration files to allow information on products to be output as RSS streams via JAX-RS RESTful interface (URLs). (i.e. port RSSProductServlet and RSSProductTransferServlet functionality to the new cas.product.service package)
              • products retrieved by product ID
              • product sets retrieved by product type (plus filtering, queries, including option to limit number of results returned)
              • include capability to retrieve information on current product transfers
              • returns information on products, product sets or product transfers as RSS streams

            August 12th to August 25th (2 weeks)

            • Review, update and consolidate the XML configuration files
            • Code refactoring

            August 26th to September 15th (3 weeks)

            • Extension: add Java classes and XML configuration files to allow information on products to be output in JSON format via JAX-RS RESTful interface (URLs).
              • products, product sets and current transfers to be retrieved in a similar fashion to that used for the other services (parameters, filters, queries)
              • returns information on products, product sets or product transfers in JSON format

            September 16th to September 23rd (1 week)

            • finalize documentation - code comments, user guides
            • close out JIRA issues
            • prepare final reports and code samples
            rlaidlaw Ross Laidlaw added a comment - - edited Below is a draft project plan: April 29th to May 3rd (1 week) Project planning Submit proposal to GSoC May 4th to May 26th (3 weeks) Preparation work read up more on Apache CXF, JAX-RS finalize plans after discussion with OODT community May 27th to June 9th (2 weeks) Add draft designs (e.g. UML) to JIRA Update project POM file (webapp/fmprod/pom.xml) to include Apache CXF Add new skeleton Java package for the project code (e.g. named org.apache.oodt.cas.product.service) June 10th to June 30th (3 weeks) Add Java classes and XML configuration files to allow products and metadata to be served as resources via JAX-RS RESTful interface (URLs). (i.e. port DataDeliveryServlet and DatasetDeliveryServlet functionality to the new cas.product.service package) products retrieved by product ID product sets retrieved by product type returns product(s) and metadata as application/zip July 1st to July 21st (3 weeks) Add Java classes and XML configuration files to allow information on products to be output as RDF streams via JAX-RS RESTful interface (URLs). (i.e. port RDFProductServlet and RDFDatasetServlet functionality to the new cas.product.service package) products retrieved by product ID product sets retrieved by product type (plus filtering, queries) add capability to retrieve information on current product transfers (as currently available in the RSS servlet) returns information on products, product sets or product transfers as RDF streams July 22nd to August 11th (3 weeks) Add Java classes and XML configuration files to allow information on products to be output as RSS streams via JAX-RS RESTful interface (URLs). (i.e. port RSSProductServlet and RSSProductTransferServlet functionality to the new cas.product.service package) products retrieved by product ID product sets retrieved by product type (plus filtering, queries, including option to limit number of results returned) include capability to retrieve information on current product transfers returns information on products, product sets or product transfers as RSS streams August 12th to August 25th (2 weeks) Review, update and consolidate the XML configuration files Code refactoring August 26th to September 15th (3 weeks) Extension: add Java classes and XML configuration files to allow information on products to be output in JSON format via JAX-RS RESTful interface (URLs). products, product sets and current transfers to be retrieved in a similar fashion to that used for the other services (parameters, filters, queries) returns information on products, product sets or product transfers in JSON format September 16th to September 23rd (1 week) finalize documentation - code comments, user guides close out JIRA issues prepare final reports and code samples
            riverma Rishi Verma added a comment -

            Hi Ross,

            Thanks for providing this excellent roadmap! It really looks great.

            I've considered doing the same thing to the CAS-Product webapp because it makes it easier to extend/implement new webservice capabilities when the webapp is written in JAX-RS (as opposed to just Java servlets). So a huge +1 here.

            I would suggest two things though:
            1. Let's look to implement fmprod in JAX-RS in such a way that the existing fmprod URL extensions do not change at all. In other words, the endpoints specified in [1] still work the same.
            2. You might want to take a look at the Curator RESTful API [2]. It provides some of the secondary goal capabilities you were describing (i.e. metadata queries, deleting). Actually, I looks like I have not updated the Curator REST API documentation [3] to reflect some recent code pushes to 0.6 (like deleting met) so thanks for the reminder.

            Looking forward to helping out on this as well.

            –
            [1] https://cwiki.apache.org/OODT/file-manager-rest-api.html
            [2] http://svn.apache.org/repos/asf/oodt/trunk/curator/src/main/java/org/apache/oodt/cas/curation/service/MetadataResource.java
            [3] https://cwiki.apache.org/OODT/cas-curator-rest-api.html

            riverma Rishi Verma added a comment - Hi Ross, Thanks for providing this excellent roadmap! It really looks great. I've considered doing the same thing to the CAS-Product webapp because it makes it easier to extend/implement new webservice capabilities when the webapp is written in JAX-RS (as opposed to just Java servlets). So a huge +1 here. I would suggest two things though: 1. Let's look to implement fmprod in JAX-RS in such a way that the existing fmprod URL extensions do not change at all. In other words, the endpoints specified in [1] still work the same. 2. You might want to take a look at the Curator RESTful API [2] . It provides some of the secondary goal capabilities you were describing (i.e. metadata queries, deleting). Actually, I looks like I have not updated the Curator REST API documentation [3] to reflect some recent code pushes to 0.6 (like deleting met) so thanks for the reminder. Looking forward to helping out on this as well. – [1] https://cwiki.apache.org/OODT/file-manager-rest-api.html [2] http://svn.apache.org/repos/asf/oodt/trunk/curator/src/main/java/org/apache/oodt/cas/curation/service/MetadataResource.java [3] https://cwiki.apache.org/OODT/cas-curator-rest-api.html

            Reesh, would you like to mentor Ross? Any OODT PMC member can apply to be a GSoC mentor? Would be great..

            chrismattmann Chris A. Mattmann added a comment - Reesh, would you like to mentor Ross? Any OODT PMC member can apply to be a GSoC mentor? Would be great..
            riverma Rishi Verma added a comment -

            Hey Chris, Ross,

            I would me more than happy to. Let me look into the process. Chris, if you have any direct pointers, do let me know.

            Thanks guys, looking forward to making OODT-605 a success!
            rishi

            riverma Rishi Verma added a comment - Hey Chris, Ross, I would me more than happy to. Let me look into the process. Chris, if you have any direct pointers, do let me know. Thanks guys, looking forward to making OODT-605 a success! rishi
            rlaidlaw Ross Laidlaw added a comment - - edited

            Hi Rishi,

            Many thanks for your comments and advice, much appreciated!

            I like your comment about a JAX-RS webapp being more extensible. Hopefully we can write/design the code in such a way that we can easily add new output formats (and configurations too).

            I can see advantages of keeping the same URL format as before - for example, existing users of OODT wouldn't need to reconfigure their projects to use the new system. Was this the main reason you were thinking of too?

            For reference for anyone looking at this JIRA issue, here's a list of the main URLs for the CAS-Product webapp that I found in the classes, documentation, wikis, etc:

            For raw data (DataDeliveryServlet, DatasetDeliveryServlet):

            /data
            /dataset

            For RDF streams (RDFProductServlet, RDFDatasetServlet):

            /rdf
            /rdf/dataset

            For RSS streams (RSSProductServlet, RSSProductTransferServlet):

            /viewRecent
            /viewTransfers

            As well as keeping these intact, it might also be useful for us to define a consistent URL scheme/plan that will fit well with extended features we'd like to introduce later. For example, we could have something like this:

            /[type]/[format]?parameters

            Where [format] is the output format and [type] can be 'product', 'dataset', 'transfer', etc., e.g.

            /product/rdf?...
            /dataset/rss?...
            /transfer/json?...

            Or we could specify the format as a parameter, e.g.:

            /product?format=JSON...
            /transfer?format=RSS...
            /dataset?format=RDF...

            I'm sure there are many alternative arrangements we could experiment with!

            Regarding GSoC, it seems that Apache prefers applications from new recruits rather than students who have already participated in previous years.[1] Since I participated in GSoC last year with Apache OODT, I've decided not to apply this year after all. But I'd like to work on OODT-605 anyway as a normal non-GSoC project. I'm happy to do the work if that's ok with everyone and will gladly accept advice, guidance, etc from the OODT community. In fact, it would be awesome to work together with you on this Rishi under your guidance, if you have time available.

            Ross

            [1] http://community.apache.org/mentee-ranking-process.html

            rlaidlaw Ross Laidlaw added a comment - - edited Hi Rishi, Many thanks for your comments and advice, much appreciated! I like your comment about a JAX-RS webapp being more extensible. Hopefully we can write/design the code in such a way that we can easily add new output formats (and configurations too). I can see advantages of keeping the same URL format as before - for example, existing users of OODT wouldn't need to reconfigure their projects to use the new system. Was this the main reason you were thinking of too? For reference for anyone looking at this JIRA issue, here's a list of the main URLs for the CAS-Product webapp that I found in the classes, documentation, wikis, etc: For raw data (DataDeliveryServlet, DatasetDeliveryServlet): /data /dataset For RDF streams (RDFProductServlet, RDFDatasetServlet): /rdf /rdf/dataset For RSS streams (RSSProductServlet, RSSProductTransferServlet): /viewRecent /viewTransfers As well as keeping these intact, it might also be useful for us to define a consistent URL scheme/plan that will fit well with extended features we'd like to introduce later. For example, we could have something like this: / [type] / [format] ?parameters Where [format] is the output format and [type] can be 'product', 'dataset', 'transfer', etc., e.g. /product/rdf?... /dataset/rss?... /transfer/json?... Or we could specify the format as a parameter, e.g.: /product?format=JSON... /transfer?format=RSS... /dataset?format=RDF... I'm sure there are many alternative arrangements we could experiment with! Regarding GSoC, it seems that Apache prefers applications from new recruits rather than students who have already participated in previous years. [1] Since I participated in GSoC last year with Apache OODT, I've decided not to apply this year after all. But I'd like to work on OODT-605 anyway as a normal non-GSoC project. I'm happy to do the work if that's ok with everyone and will gladly accept advice, guidance, etc from the OODT community. In fact, it would be awesome to work together with you on this Rishi under your guidance, if you have time available. Ross [1] http://community.apache.org/mentee-ranking-process.html

            +1 to keeping the old scheme (being back compat).
            +1 to defining a new URL scheme that Ross L. proposes below which is awesome and way more consistent!

            Then in later versions of OODT (perhaps before 1.0 but after 0.6 or 0.7) we could revisit deprecating
            and even removing the old APIs (with lots of notice in CHANGES.txt and warnings and other good stuff).

            chrismattmann Chris A. Mattmann added a comment - +1 to keeping the old scheme (being back compat). +1 to defining a new URL scheme that Ross L. proposes below which is awesome and way more consistent! Then in later versions of OODT (perhaps before 1.0 but after 0.6 or 0.7) we could revisit deprecating and even removing the old APIs (with lots of notice in CHANGES.txt and warnings and other good stuff).

            Ross I think you can still apply to GSoC if you want – I'm not sure how the ranking will work out so the docs you posted may be right but you'll never know if you don't try. Either way is fine with me though and I appreciate and thank you for being willing to do the work regardless!

            chrismattmann Chris A. Mattmann added a comment - Ross I think you can still apply to GSoC if you want – I'm not sure how the ranking will work out so the docs you posted may be right but you'll never know if you don't try. Either way is fine with me though and I appreciate and thank you for being willing to do the work regardless!
            rlaidlaw Ross Laidlaw added a comment -

            Hi Chris!

            Many thanks for your comments and advice. I see what you mean about GSoC. I'll try to submit a proposal before Friday's deadline

            Ross

            rlaidlaw Ross Laidlaw added a comment - Hi Chris! Many thanks for your comments and advice. I see what you mean about GSoC. I'll try to submit a proposal before Friday's deadline Ross
            riverma Rishi Verma added a comment -

            +1 to still applying for GSoC - go for it Ross! You have an excellent game plan here, so it will be valuable either way it goes.

            Rishi

            riverma Rishi Verma added a comment - +1 to still applying for GSoC - go for it Ross! You have an excellent game plan here, so it will be valuable either way it goes. Rishi
            rlaidlaw Ross Laidlaw added a comment -

            Thanks Rishi & Chris!

            Today I submitted a GSoC proposal. It should be visible to everyone but please let me know if you can't access the link.

            Ross

            rlaidlaw Ross Laidlaw added a comment - Thanks Rishi & Chris! Today I submitted a GSoC proposal . It should be visible to everyone but please let me know if you can't access the link. Ross

            I clicked wish to mentor to make sure you had at least one of those so the proposal has a shot to get in.

            Rishi please ASAP click "wish to mentor" so you can be a mentor too.

            Thanks Ross!

            chrismattmann Chris A. Mattmann added a comment - I clicked wish to mentor to make sure you had at least one of those so the proposal has a shot to get in. Rishi please ASAP click "wish to mentor" so you can be a mentor too. Thanks Ross!
            rlaidlaw Ross Laidlaw added a comment -

            Thanks Chris, much appreciated!

            rlaidlaw Ross Laidlaw added a comment - Thanks Chris, much appreciated!
            riverma Rishi Verma added a comment -

            Hey Chris,

            Should be ready to do that soon - just waiting for Apache to approve my GSoC mentor connection.

            Thanks -
            Rishi

            riverma Rishi Verma added a comment - Hey Chris, Should be ready to do that soon - just waiting for Apache to approve my GSoC mentor connection. Thanks - Rishi

            Reesh nothing for Apache to approve the ACK is all you need and you got it.

            Outside of the ACK are you registered in Melange? If so, the next step is to indicate your "willing to mentor" on proposals (aka Ross's) and then to rank the proposals per latest instructions from Uli.

            chrismattmann Chris A. Mattmann added a comment - Reesh nothing for Apache to approve the ACK is all you need and you got it. Outside of the ACK are you registered in Melange? If so, the next step is to indicate your "willing to mentor" on proposals (aka Ross's) and then to rank the proposals per latest instructions from Uli.
            riverma Rishi Verma added a comment -

            Hey Chris,

            I meant waiting for the Melange system to get a response from Apache. My current status on Melange (after signing up for a "connection with ASF") is the following: 'Status: Org Action Required'. I currently don't see any links for looking at proposals or signing up for 'willing to mentor'.

            I think I have to wait for a response from the ASF, no? If not, could you send me a sample link of where on Melange one can look over proposals etc? I can check to see if I have access to that page.

            Thanks!
            Rishi

            riverma Rishi Verma added a comment - Hey Chris, I meant waiting for the Melange system to get a response from Apache. My current status on Melange (after signing up for a "connection with ASF") is the following: 'Status: Org Action Required'. I currently don't see any links for looking at proposals or signing up for 'willing to mentor'. I think I have to wait for a response from the ASF, no? If not, could you send me a sample link of where on Melange one can look over proposals etc? I can check to see if I have access to that page. Thanks! Rishi

            Gotcha Reesh – can you bug Ulrich Stark and the code-awards@apache.org list for info on this?

            chrismattmann Chris A. Mattmann added a comment - Gotcha Reesh – can you bug Ulrich Stark and the code-awards@apache.org list for info on this?
            riverma Rishi Verma added a comment -

            Hey Chris - just an update. It went through and I've added Ross' project to my list of "willing to mentor".

            Thanks!
            rishi

            riverma Rishi Verma added a comment - Hey Chris - just an update. It went through and I've added Ross' project to my list of "willing to mentor". Thanks! rishi
            • push to 0.7
            chrismattmann Chris A. Mattmann added a comment - push to 0.7
            rlaidlaw Ross Laidlaw added a comment -

            A working JAX-RS web application has been checked into the OODT trunk. The Java code is in a new 'jaxrs' package within the CAS Product web application (in the webapp/fmprod subdirectory).

            There are still many improvements and enhancements to be made, but this provides a foundation to build on.

            rlaidlaw Ross Laidlaw added a comment - A working JAX-RS web application has been checked into the OODT trunk. The Java code is in a new 'jaxrs' package within the CAS Product web application (in the webapp/fmprod subdirectory). There are still many improvements and enhancements to be made, but this provides a foundation to build on.
            riverma Rishi Verma added a comment -

            Great work Ross!

            riverma Rishi Verma added a comment - Great work Ross!

            People

              rlaidlaw Ross Laidlaw
              rlaidlaw Ross Laidlaw
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 3,360h
                  3,360h
                  Remaining:
                  Remaining Estimate - 3,360h
                  3,360h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified