Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-683

new NotebookRepo storage: P2P using IPFS or BitTorrent protocol

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      Zeppelin stores Notebooks in a pluggable NotebookStorage and implementations for local FS, S3, local Git repository and Azure already exist.

      The idea of this project would be to experiment with new, distributed way of storing notebook - though P2P protocols like IPFS or dat or ZeroNet.

      The main goal would be - to get at least one new NotebookRepo implementation using p2p technology, that supports notebook versioning merged by the end of the summer.

      It includes a research on which one p2p system fits better, design and implementation of new NotebookRepo backend, web app GUI update for versioning and repository selection.

        Issue Links

          Activity

          Hide
          Onkar Shedge Onkar Shedge added a comment - - edited

          Hi,
          I am Onkar from Pune, India pursuing Computer Science Undergraduate at Pune Institute of Computer Technology(PICT). I am interested in this project for Gsoc 2016.Right now I am investigating the protocols mentioned above. Please guide me on how to proceed further.
          Thanks

          Show
          Onkar Shedge Onkar Shedge added a comment - - edited Hi, I am Onkar from Pune, India pursuing Computer Science Undergraduate at Pune Institute of Computer Technology(PICT). I am interested in this project for Gsoc 2016.Right now I am investigating the protocols mentioned above. Please guide me on how to proceed further. Thanks
          Hide
          rravill@ncsu.edu roshiniravilla added a comment -

          Hi Mr. Alexander,

          I am currently pursuing my masters in Computer science at North Carolina State University. I am currently working on a project for creating peer to peer systems communication. I think this project suggested here would be a great opportunity to extend my current knowledge on P2P. Please help me by suggesting me how to proceed with the proposal.

          Thank you.

          Show
          rravill@ncsu.edu roshiniravilla added a comment - Hi Mr. Alexander, I am currently pursuing my masters in Computer science at North Carolina State University. I am currently working on a project for creating peer to peer systems communication. I think this project suggested here would be a great opportunity to extend my current knowledge on P2P. Please help me by suggesting me how to proceed with the proposal. Thank you.
          Hide
          bzz Alexander Bezzubov added a comment -

          It's great to see you guys interested in this project.

          Thank you for expressing it here and for sharing drafts with the community on the project mailing list!

          Hope you guys have submitted the proposals before the official deadline today!

          Show
          bzz Alexander Bezzubov added a comment - It's great to see you guys interested in this project. Thank you for expressing it here and for sharing drafts with the community on the project mailing list! Hope you guys have submitted the proposals before the official deadline today!
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user onkarshedge opened a pull request:

          https://github.com/apache/zeppelin/pull/989

          Added ipfsnotebookrepo [ Zeppelin-683 WIP ]

              1. What is this PR for?
                IPFSNoteBookRepo IPFS storage and sharing zeppelin notes.
              1. What type of PR is it?
                Feature
              1. Todos
          • [API Multiaddress to connect to ipfs daemon get from configuration ]
          • [Import Note from other peers when a ipfs multihash is given ]
          • [UI support to import the Note when hash is given as input]
          • [Testing ]
              1. What is the Jira issue?
              1. How should this be tested?
                [Google Document](https://docs.google.com/document/d/1eYjQL0t7VlBTRMYoA0nWRJAnZ6RHUF7S7G3rx9xH5Pc/edit?usp=sharing)
              1. Screenshots (if appropriate)
                [Google Document](https://docs.google.com/document/d/1eYjQL0t7VlBTRMYoA0nWRJAnZ6RHUF7S7G3rx9xH5Pc/edit?usp=sharing)
              1. Questions:
          • Does the licenses files need update?
          • Is there breaking changes for older versions?
          • Does this needs documentation?

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/onkarshedge/incubator-zeppelin ipfs

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/989.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #989


          commit d4eb14b47221b10e6ad2eea482f77c23caa81255
          Author: onkarshedge <shedge31onkar@gmail.com>
          Date: 2016-06-10T07:55:59Z

          Added ipfsnotebookrepo


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user onkarshedge opened a pull request: https://github.com/apache/zeppelin/pull/989 Added ipfsnotebookrepo [ Zeppelin-683 WIP ] What is this PR for? IPFSNoteBookRepo IPFS storage and sharing zeppelin notes. What type of PR is it? Feature Todos [API Multiaddress to connect to ipfs daemon get from configuration ] [Import Note from other peers when a ipfs multihash is given ] [UI support to import the Note when hash is given as input] [Testing ] What is the Jira issue? [Jira issue] ( https://issues.apache.org/jira/browse/ZEPPELIN-683 ) How should this be tested? [Google Document] ( https://docs.google.com/document/d/1eYjQL0t7VlBTRMYoA0nWRJAnZ6RHUF7S7G3rx9xH5Pc/edit?usp=sharing ) Screenshots (if appropriate) [Google Document] ( https://docs.google.com/document/d/1eYjQL0t7VlBTRMYoA0nWRJAnZ6RHUF7S7G3rx9xH5Pc/edit?usp=sharing ) Questions: Does the licenses files need update? Is there breaking changes for older versions? Does this needs documentation? You can merge this pull request into a Git repository by running: $ git pull https://github.com/onkarshedge/incubator-zeppelin ipfs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/989.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #989 commit d4eb14b47221b10e6ad2eea482f77c23caa81255 Author: onkarshedge <shedge31onkar@gmail.com> Date: 2016-06-10T07:55:59Z Added ipfsnotebookrepo
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user onkarshedge opened a pull request:

          https://github.com/apache/zeppelin/pull/1231

          [Zeppelin - 683][WIP] BittorrentNotebookRepo

              1. What is this PR for?
                To share notes and download via a magnet link. Trackerless, only via DHT.
              1. What type of PR is it?
                [Feature]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
              1. How should this be tested?
                Outline the steps to test the PR here.
              1. Screenshots (if appropriate)
              1. Questions:
          • Does the licenses files need update?
          • Is there breaking changes for older versions?
          • Does this needs documentation?

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/onkarshedge/incubator-zeppelin bittorrent

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1231.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1231


          commit f1b2c5308e14efe82e755235c7ced80095c898d8
          Author: onkarshedge <shedge31onkar@gmail.com>
          Date: 2016-07-27T05:05:45Z

          added bittorrent repo


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user onkarshedge opened a pull request: https://github.com/apache/zeppelin/pull/1231 [Zeppelin - 683] [WIP] BittorrentNotebookRepo What is this PR for? To share notes and download via a magnet link. Trackerless, only via DHT. What type of PR is it? [Feature] Todos [ ] - Task What is the Jira issue? [Jira issue] ( https://issues.apache.org/jira/browse/ZEPPELIN-683 ) How should this be tested? Outline the steps to test the PR here. Screenshots (if appropriate) Questions: Does the licenses files need update? Is there breaking changes for older versions? Does this needs documentation? You can merge this pull request into a Git repository by running: $ git pull https://github.com/onkarshedge/incubator-zeppelin bittorrent Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1231.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1231 commit f1b2c5308e14efe82e755235c7ced80095c898d8 Author: onkarshedge <shedge31onkar@gmail.com> Date: 2016-07-27T05:05:45Z added bittorrent repo
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user onkarshedge closed the pull request at:

          https://github.com/apache/zeppelin/pull/989

          Show
          githubbot ASF GitHub Bot added a comment - Github user onkarshedge closed the pull request at: https://github.com/apache/zeppelin/pull/989
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user onkarshedge reopened a pull request:

          https://github.com/apache/zeppelin/pull/989

          Added ipfsnotebookrepo [ Zeppelin-683 ]

              1. What is this PR for?
                IPFSNoteBookRepo IPFS storage and sharing zeppelin notes.
              1. What type of PR is it?
                Feature
              1. Todos
          • [x] checkpoint revisions, get particular revision
          • [x] List of revision history
          • [x] remove note and also a particular revision
          • [x] support to import the Note when hash is given as input
          • [x] Documentation
              1. What is the Jira issue?
              1. How should this be tested?
                You need the [ipfs jar](https://github.com/ipfs/java-ipfs-api/blob/master/pom.xml) in the local .m2 repository. `mvn install:install-file` hers is complete [command](https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html)
                Also please refer the `storage document` for more help.
              1. Screenshots (if appropriate)
                [Successful import](https://gifyu.com/image/52t)
                [Failure import](https://gifyu.com/image/52Y)
              1. Questions:
          • Does the licenses files need update? Yes
          • Is there breaking changes for older versions?
          • Does this needs documentation? Yes

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/onkarshedge/incubator-zeppelin ipfs

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/989.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #989


          commit 61a070449f5d75d53199a3bb41604a48fd175aed
          Author: onkarshedge <shedge31onkar@gmail.com>
          Date: 2016-06-10T07:55:59Z

          Added ipfsnotebookrepo with Versioning and import feature
          backend and frontend

          commit 8a6ff363d89571eb8fb25d2af86e5a7080061072
          Author: onkarshedge <shedge31onkar@gmail.com>
          Date: 2016-08-17T14:44:47Z

          added import status as ng-toast

          commit 06040835d6f97a4d6bbce7a6ee274d118aed8ca4
          Author: onkarshedge <shedge31onkar@gmail.com>
          Date: 2016-08-18T06:12:42Z

          Added documentation

          commit b09876d70abf99a448e791363e6363d3cf7c3ac6
          Author: onkarshedge <shedge31onkar@gmail.com>
          Date: 2016-08-22T15:48:54Z

          refactored code to remove any dependencies and use httpclient


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user onkarshedge reopened a pull request: https://github.com/apache/zeppelin/pull/989 Added ipfsnotebookrepo [ Zeppelin-683 ] What is this PR for? IPFSNoteBookRepo IPFS storage and sharing zeppelin notes. What type of PR is it? Feature Todos [x] checkpoint revisions, get particular revision [x] List of revision history [x] remove note and also a particular revision [x] support to import the Note when hash is given as input [x] Documentation What is the Jira issue? [Jira issue] ( https://issues.apache.org/jira/browse/ZEPPELIN-683 ) How should this be tested? You need the [ipfs jar] ( https://github.com/ipfs/java-ipfs-api/blob/master/pom.xml ) in the local .m2 repository. `mvn install:install-file` hers is complete [command] ( https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html ) Also please refer the `storage document` for more help. Screenshots (if appropriate) [Successful import] ( https://gifyu.com/image/52t ) [Failure import] ( https://gifyu.com/image/52Y ) Questions: Does the licenses files need update? Yes Is there breaking changes for older versions? Does this needs documentation? Yes You can merge this pull request into a Git repository by running: $ git pull https://github.com/onkarshedge/incubator-zeppelin ipfs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/989.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #989 commit 61a070449f5d75d53199a3bb41604a48fd175aed Author: onkarshedge <shedge31onkar@gmail.com> Date: 2016-06-10T07:55:59Z Added ipfsnotebookrepo with Versioning and import feature backend and frontend commit 8a6ff363d89571eb8fb25d2af86e5a7080061072 Author: onkarshedge <shedge31onkar@gmail.com> Date: 2016-08-17T14:44:47Z added import status as ng-toast commit 06040835d6f97a4d6bbce7a6ee274d118aed8ca4 Author: onkarshedge <shedge31onkar@gmail.com> Date: 2016-08-18T06:12:42Z Added documentation commit b09876d70abf99a448e791363e6363d3cf7c3ac6 Author: onkarshedge <shedge31onkar@gmail.com> Date: 2016-08-22T15:48:54Z refactored code to remove any dependencies and use httpclient

            People

            • Assignee:
              Unassigned
              Reporter:
              bzz Alexander Bezzubov
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:

                Development