Pig
  1. Pig
  2. PIG-366

PigPen - Eclipse plugin for a graphical PigLatin editor

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      This is an Eclipse plugin that provides a GUI that can help users create PigLatin scripts and see the example generator outputs on the fly and submit the jobs to hadoop clusters.

      1. PigPen.tgz
        5.84 MB
        Shubham Chopra
      2. pigPen.patch
        175 kB
        Shubham Chopra
      3. pigpen.patch
        467 kB
        Shubham Chopra
      4. org.apache.pig.pigpen-0.7.5.tar.gz
        9.42 MB
        Robert Gibbon
      5. org.apache.pig.pigpen-0.7.4.tar.gz
        9.46 MB
        Robert Gibbon
      6. org.apache.pig.pigpen-0.7.2.tar.gz
        9.44 MB
        Robert Gibbon
      7. org.apache.pig.pigpen-0.7.0.tar.gz
        9.41 MB
        Robert Gibbon
      8. org.apache.pig.pigpen_0.7.5.jar
        9.51 MB
        Robert Gibbon
      9. org.apache.pig.pigpen_0.7.4.jar
        9.51 MB
        Robert Gibbon
      10. org.apache.pig.pigpen_0.7.2.jar
        9.49 MB
        Robert Gibbon
      11. org.apache.pig.pigpen_0.0.4.jar
        5.76 MB
        Shubham Chopra
      12. org.apache.pig.pigpen_0.0.1.tgz
        9.50 MB
        Shubham Chopra
      13. org.apache.pig.pigpen_0.0.1.jar
        5.65 MB
        Shubham Chopra

        Activity

        Hide
        Shubham Chopra added a comment -

        Patch adding the eclipse plugin source to Pig contrib

        Show
        Shubham Chopra added a comment - Patch adding the eclipse plugin source to Pig contrib
        Hide
        Romain Rigaux added a comment -

        Hello, is there a simple way to compile it or a location where the plugin is already compiled?

        Thanks!

        Show
        Romain Rigaux added a comment - Hello, is there a simple way to compile it or a location where the plugin is already compiled? Thanks!
        Hide
        Shubham Chopra added a comment -

        Compiled plugin ready to use. Includes some minor modifications. Just unpack the tarball in eclipse/plugins directory. Path to the configuration directory containing hadoop-site.xml and pig.properties will have to be provided by setting the variable ConfigurationPath in Pig Preferences (Window -> Preferences -> Pig).

        Show
        Shubham Chopra added a comment - Compiled plugin ready to use. Includes some minor modifications. Just unpack the tarball in eclipse/plugins directory. Path to the configuration directory containing hadoop-site.xml and pig.properties will have to be provided by setting the variable ConfigurationPath in Pig Preferences (Window -> Preferences -> Pig).
        Hide
        Alan Gates added a comment -

        Shubham, sorry to take so long to get to looking at this.

        I have several comments.

        One, the files lack the Apache license header.

        Two, they use tabs not spaces.

        Three, there isn't a build file. At the PigPen level there should be a build.xml so that users can build the plugin for themselves.

        Show
        Alan Gates added a comment - Shubham, sorry to take so long to get to looking at this. I have several comments. One, the files lack the Apache license header. Two, they use tabs not spaces. Three, there isn't a build file. At the PigPen level there should be a build.xml so that users can build the plugin for themselves.
        Hide
        Ian Holsman added a comment -

        shouldn't there be a build.xml or something in the patch?

        Show
        Ian Holsman added a comment - shouldn't there be a build.xml or something in the patch?
        Hide
        Shubham Chopra added a comment -

        Thanks for the comments Alan. I am working on a few improvements of the Plugin and should be able to submit a patch in about a week. I will make sure the next patch meets the requirements.

        Show
        Shubham Chopra added a comment - Thanks for the comments Alan. I am working on a few improvements of the Plugin and should be able to submit a patch in about a week. I will make sure the next patch meets the requirements.
        Hide
        Shubham Chopra added a comment - - edited

        Updated pigpen patch.

        This is the source code of the PigPen Eclipse plugin.

        The pigPen text editor was adapted from Sausage written by Benjamin Francisoud (http://benjamin.francisoud.googlepages.com/sausage) and the entire graphical part of the plugin was extended from the initial work done by Chris Olston.

        Feedback and suggestions are most welcome and can be mailed to shubhamc@yahoo-inc.com or put as comments here.

        Show
        Shubham Chopra added a comment - - edited Updated pigpen patch. This is the source code of the PigPen Eclipse plugin. The pigPen text editor was adapted from Sausage written by Benjamin Francisoud ( http://benjamin.francisoud.googlepages.com/sausage ) and the entire graphical part of the plugin was extended from the initial work done by Chris Olston. Feedback and suggestions are most welcome and can be mailed to shubhamc@yahoo-inc.com or put as comments here.
        Hide
        Shubham Chopra added a comment - - edited

        PigPen plugin jar containing the compiled plugin. I was not able to get a build.xml for the plugin since it needs to be compiled in eclipse environment.

        Notes on using the plugin:
        1. The plugin right now supports script editing from in the text form. You can switch to the graphic view to see the alternate graph representation of the script.
        2. You would have to supply a suitable hadoop-site.xml for the plugin to be able to connect to a cluster. You can also submit the job using the plugin.
        3. The plugin might not work in cases where the access to cluster is restricted/firewalled and is only allowed through gateways.
        4. You can copy a part of the data (The example generator uses 10,000 records) and copy that to a local single-node pseudo-distributed cluster that can easily be setup (http://hadoop.apache.org/core/docs/r0.18.2/quickstart.html#PseudoDistributed). You can then use the plugin with this pseudo-cluster to tweak the script and see the examples.

        The plugin uses the example generator in PIG (http://wiki.apache.org/pig/ExampleGenerator). An introduction to the plugin can be seen here
        http://wiki.apache.org/pig/PigPen

        Show
        Shubham Chopra added a comment - - edited PigPen plugin jar containing the compiled plugin. I was not able to get a build.xml for the plugin since it needs to be compiled in eclipse environment. Notes on using the plugin: 1. The plugin right now supports script editing from in the text form. You can switch to the graphic view to see the alternate graph representation of the script. 2. You would have to supply a suitable hadoop-site.xml for the plugin to be able to connect to a cluster. You can also submit the job using the plugin. 3. The plugin might not work in cases where the access to cluster is restricted/firewalled and is only allowed through gateways. 4. You can copy a part of the data (The example generator uses 10,000 records) and copy that to a local single-node pseudo-distributed cluster that can easily be setup ( http://hadoop.apache.org/core/docs/r0.18.2/quickstart.html#PseudoDistributed ). You can then use the plugin with this pseudo-cluster to tweak the script and see the examples. The plugin uses the example generator in PIG ( http://wiki.apache.org/pig/ExampleGenerator ). An introduction to the plugin can be seen here http://wiki.apache.org/pig/PigPen
        Hide
        Shubham Chopra added a comment -

        The plugin can be used straight-away by putting the plugin jar into eclipse/plugins/ folder.

        Show
        Shubham Chopra added a comment - The plugin can be used straight-away by putting the plugin jar into eclipse/plugins/ folder.
        Hide
        Alan Gates added a comment -

        Shubham,

        I think we need a build.xml in this plugin. Otherwise how can we rebuild it for new versions, etc.? I think you mentioned somewhere that you didn't include it because it requires eclipse libraries to build, correct? Then we should either include those eclipse libraries in a lib directory in this patch or provide instructions on where to fetch them.

        Show
        Alan Gates added a comment - Shubham, I think we need a build.xml in this plugin. Otherwise how can we rebuild it for new versions, etc.? I think you mentioned somewhere that you didn't include it because it requires eclipse libraries to build, correct? Then we should either include those eclipse libraries in a lib directory in this patch or provide instructions on where to fetch them.
        Hide
        Olga Natkovich added a comment -

        Shubham, any update on this?

        Show
        Olga Natkovich added a comment - Shubham, any update on this?
        Hide
        Shubham Chopra added a comment -

        Updated plugin jar. This addresses a lot of minor issues with the earlier version. I am still working on the ant build script that will enable the plugin builds from the source code.

        Show
        Shubham Chopra added a comment - Updated plugin jar. This addresses a lot of minor issues with the earlier version. I am still working on the ant build script that will enable the plugin builds from the source code.
        Hide
        Shubham Chopra added a comment -

        Tar-ball containing the PigPen update-site. Untar it to a suitable location and direct the eclipse software update to use that for installing PigPen.

        I have tried this with Eclipse Ganymede. The update site should address the dependency issues by downloading and installing the required dependencies.

        Show
        Shubham Chopra added a comment - Tar-ball containing the PigPen update-site. Untar it to a suitable location and direct the eclipse software update to use that for installing PigPen. I have tried this with Eclipse Ganymede. The update site should address the dependency issues by downloading and installing the required dependencies.
        Hide
        patrick o'leary added a comment -

        What version of hadoop is PigPen designed to use?
        Am getting the following error
        Caused by: org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol org.apache.hadoop.mapred.JobSubmissionProtocol version mismatch. (client = 11, server = 10)

        Currently using pigpen pigpen_0.0.4.jar and hadoop 0.18.3

        The wiki should contain version numbers and be updated to point to the new tar ball

        Show
        patrick o'leary added a comment - What version of hadoop is PigPen designed to use? Am getting the following error Caused by: org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol org.apache.hadoop.mapred.JobSubmissionProtocol version mismatch. (client = 11, server = 10) Currently using pigpen pigpen_0.0.4.jar and hadoop 0.18.3 The wiki should contain version numbers and be updated to point to the new tar ball
        Hide
        Alan Gates added a comment -

        At this point no one has picked up PigPen recently and kept it up to date. I know it worked with Pig 0.2.0, but it has not been updated since then.

        Show
        Alan Gates added a comment - At this point no one has picked up PigPen recently and kept it up to date. I know it worked with Pig 0.2.0, but it has not been updated since then.
        Hide
        patrick o'leary added a comment -

        I'm guessing the 2008-11-12 12:25 AM patch isn't upto date?
        The tar doesn't contain the src

        Show
        patrick o'leary added a comment - I'm guessing the 2008-11-12 12:25 AM patch isn't upto date? The tar doesn't contain the src
        Hide
        Jeff Zhang added a comment -

        Anyone know where's the latest source code ? It seems the source code of pigpen wasn't integrated into Pig trunk, and PigPen.tgz only has the .jar.

        Show
        Jeff Zhang added a comment - Anyone know where's the latest source code ? It seems the source code of pigpen wasn't integrated into Pig trunk, and PigPen.tgz only has the .jar.
        Hide
        Jeff Zhang added a comment -

        Does anyone continue maintain this issue ? And could the author contribute the latest source code, I can help about this jira.

        Show
        Jeff Zhang added a comment - Does anyone continue maintain this issue ? And could the author contribute the latest source code, I can help about this jira.
        Hide
        Olga Natkovich added a comment -

        I don't think we have an owner. This could is looking for one

        Show
        Olga Natkovich added a comment - I don't think we have an owner. This could is looking for one
        Hide
        Alan Gates added a comment -

        I talked to Shubham (the original author). He indicated that the code in pigpen.patch from 11-12-2008 is the latest code.

        Show
        Alan Gates added a comment - I talked to Shubham (the original author). He indicated that the code in pigpen.patch from 11-12-2008 is the latest code.
        Hide
        Daniel Dai added a comment -

        Mark it to be a candidate project for "Google summer of code 2010" program.

        Notes for GSOC 2010 applicants:
        1. A good starting point for this project is Sigmod paper "Generating Example Data for Dataflow Programs"
        2. Current code is out-dated and is no longer working. We need your help to bring this work up-to-date.

        Show
        Daniel Dai added a comment - Mark it to be a candidate project for "Google summer of code 2010" program. Notes for GSOC 2010 applicants: 1. A good starting point for this project is Sigmod paper "Generating Example Data for Dataflow Programs" 2. Current code is out-dated and is no longer working. We need your help to bring this work up-to-date.
        Hide
        Robert Gibbon added a comment -

        I submitted some basic fixes to the pig-eclipse plugin today, so that it works on my environment (osx / java1.5 32bit) - maybe I should volunteer to take on PigPen, if no further progress has been made on this?

        Lemme know if I can help

        Show
        Robert Gibbon added a comment - I submitted some basic fixes to the pig-eclipse plugin today, so that it works on my environment (osx / java1.5 32bit) - maybe I should volunteer to take on PigPen, if no further progress has been made on this? Lemme know if I can help
        Hide
        Olga Natkovich added a comment -

        Hi Robert,

        We would love it if you decide to own PigPen - it is all yours!

        Show
        Olga Natkovich added a comment - Hi Robert, We would love it if you decide to own PigPen - it is all yours!
        Hide
        Renato Javier Marroquín Mogrovejo added a comment -

        Hey Robert, I would be happy to help out too (=
        I have been looking for an interesting pig project, let me know how I can help, or how we can share the work load.

        Renato M.

        Show
        Renato Javier Marroquín Mogrovejo added a comment - Hey Robert, I would be happy to help out too (= I have been looking for an interesting pig project, let me know how I can help, or how we can share the work load. Renato M.
        Hide
        Robert Gibbon added a comment -

        Well I got a basic, working build up from the patch.

        @Renato: Give me a few days for code familiarization and to isolate some nasty creepy crawlies, and I'll come up with a shortlist of new features and bugfixes.

        Show
        Robert Gibbon added a comment - Well I got a basic, working build up from the patch. @Renato: Give me a few days for code familiarization and to isolate some nasty creepy crawlies, and I'll come up with a shortlist of new features and bugfixes.
        Hide
        Robert Gibbon added a comment -

        Still working on this. Some holidays and some life got in the way.

        I have stripped back a lot of functionality and focused on improving the script editor for now. When I did some simple tests I noticed that the 0.7 pig parser borks when confronted with %default instructions - maybe you already have a ticket for that.

        I'll try to tidy up the code and attach it sometime tomorrow.

        Show
        Robert Gibbon added a comment - Still working on this. Some holidays and some life got in the way. I have stripped back a lot of functionality and focused on improving the script editor for now. When I did some simple tests I noticed that the 0.7 pig parser borks when confronted with %default instructions - maybe you already have a ticket for that. I'll try to tidy up the code and attach it sometime tomorrow.
        Hide
        Robert Gibbon added a comment -

        A simplified editor for pig scripts with syntax highlighting and validation. More to come very soon.

        Show
        Robert Gibbon added a comment - A simplified editor for pig scripts with syntax highlighting and validation. More to come very soon.
        Hide
        Robert Gibbon added a comment -

        Syntax highlighting and validation, and can launch a job on the cluster too.

        Show
        Robert Gibbon added a comment - Syntax highlighting and validation, and can launch a job on the cluster too.
        Hide
        Robert Gibbon added a comment -

        The binary for the plugin. Place in the "plugins" subdirectory of eclipse.

        Show
        Robert Gibbon added a comment - The binary for the plugin. Place in the "plugins" subdirectory of eclipse.
        Hide
        Yan Zhou added a comment -

        Robert, first, thanks for your effort to pick up this feature.

        You mentioned in your 09/08 Comment that you "stripped back" a lot of functionality and focused on the script editor. I'm wondering if it is possible to add your fixes/improvements on top of Shubham's patch. Specifically, I'm interested in the example generator use in PigPen, which seems to absent from your patches. FYI, I'm currently working on improving and enhancing the example generator left over by Shubham about 2 years ago.

        Show
        Yan Zhou added a comment - Robert, first, thanks for your effort to pick up this feature. You mentioned in your 09/08 Comment that you "stripped back" a lot of functionality and focused on the script editor. I'm wondering if it is possible to add your fixes/improvements on top of Shubham's patch. Specifically, I'm interested in the example generator use in PigPen, which seems to absent from your patches. FYI, I'm currently working on improving and enhancing the example generator left over by Shubham about 2 years ago.
        Hide
        Alan Gates added a comment -

        Robert,

        This looks great. A couple of questions.

        1. The most reasonable place to put this would be under contrib, since it's really a standalone tool for use with Pig. Does that seem reasonable?
        2. As far as I know you're the only person working on this at the moment. Do you see yourself continuing to work on it for a while? If so, then I think this is a great contribution and we're happy to support your work on it. If it's a one off thing I'm less inclined to check it in as it will rot again (as it did from 0.2 on) and not be useful for users.

        And a question to the rest of the Pig community. Any volunteers out there to take this for a drive around the block and see how it does? Feedback from power Eclipse users would be particularly valuable.

        A few nitpicks about the patch itself:

        1. We use 4 spaces rather than tabs in Pig code, so the files will have to be reformatted to match that.
        2. One file (MessageRunner.java) is missing the Apache license header.
        Show
        Alan Gates added a comment - Robert, This looks great. A couple of questions. The most reasonable place to put this would be under contrib, since it's really a standalone tool for use with Pig. Does that seem reasonable? As far as I know you're the only person working on this at the moment. Do you see yourself continuing to work on it for a while? If so, then I think this is a great contribution and we're happy to support your work on it. If it's a one off thing I'm less inclined to check it in as it will rot again (as it did from 0.2 on) and not be useful for users. And a question to the rest of the Pig community. Any volunteers out there to take this for a drive around the block and see how it does? Feedback from power Eclipse users would be particularly valuable. A few nitpicks about the patch itself: We use 4 spaces rather than tabs in Pig code, so the files will have to be reformatted to match that. One file (MessageRunner.java) is missing the Apache license header.
        Hide
        Robert Gibbon added a comment -

        Yan Zhou: I plan to reintroduce all the original features incrementally. Are you working on the example generator as part of the Pig server backend? I can hook the plugin in to a backend API or let you take over that part of the plugin. Let me know what is best for you.

        Alan:

        contrib seems a sensible place. I would like to integrate it with the Apache build chain, via ivy. What are your thoughts on that?
        I'm happy to run with this for as long as it is needed and useful.
        It would be remiss of me not to mention that I built this on Eclipse Helios. It needs a bit of work to get it hooked up to Galileo, which I also plan to do.

        The minors: Noted - always happy to comply with coding standards. Also it needs to be reformatted as a .patch, if I'm not mistaken? I'll rerelease asap

        Show
        Robert Gibbon added a comment - Yan Zhou: I plan to reintroduce all the original features incrementally. Are you working on the example generator as part of the Pig server backend? I can hook the plugin in to a backend API or let you take over that part of the plugin. Let me know what is best for you. Alan: contrib seems a sensible place. I would like to integrate it with the Apache build chain, via ivy. What are your thoughts on that? I'm happy to run with this for as long as it is needed and useful. It would be remiss of me not to mention that I built this on Eclipse Helios. It needs a bit of work to get it hooked up to Galileo, which I also plan to do. The minors: Noted - always happy to comply with coding standards. Also it needs to be reformatted as a .patch, if I'm not mistaken? I'll rerelease asap
        Hide
        Yan Zhou added a comment -

        Yes. But the original patch by Shubham had hooked the plugin to the example generator interface unless you will have found something funky in that patch. I have no intention to change the interface.

        Show
        Yan Zhou added a comment - Yes. But the original patch by Shubham had hooked the plugin to the example generator interface unless you will have found something funky in that patch. I have no intention to change the interface.
        Hide
        Yan Zhou added a comment -

        One more clearification: by design example generator does not submit any jobs to hadoop, it just runs at the client as a local application.

        Show
        Yan Zhou added a comment - One more clearification: by design example generator does not submit any jobs to hadoop, it just runs at the client as a local application.
        Hide
        Olga Natkovich added a comment -

        I think it used to use "true local mode in pig". However, we no longer support this and the new version need to be connected to the current local mode in pig which is basically hadoop's local mode

        Show
        Olga Natkovich added a comment - I think it used to use "true local mode in pig". However, we no longer support this and the new version need to be connected to the current local mode in pig which is basically hadoop's local mode
        Hide
        Robert Gibbon added a comment -

        Ok I will implement a classloader to avoid hardwiring the plugin to a specific release. I'll do same for the parser feature. I made a new diff to fix the formatting but it is probably less useful than a tarball. I'll upload it in the morning (I'm on dialup)

        Show
        Robert Gibbon added a comment - Ok I will implement a classloader to avoid hardwiring the plugin to a specific release. I'll do same for the parser feature. I made a new diff to fix the formatting but it is probably less useful than a tarball. I'll upload it in the morning (I'm on dialup)
        Hide
        Yan Zhou added a comment -

        Robert,

        Could you put down a step-by-step instruction on how to use this jar as an eclipse plug-in? Thanks.

        Show
        Yan Zhou added a comment - Robert, Could you put down a step-by-step instruction on how to use this jar as an eclipse plug-in? Thanks.
        Hide
        Robert Gibbon added a comment -

        Here's the README

        • Download the latest version of the jar binary. Right now that's org.apache.pig.pigpen_0.7.4.jar
        • Put the jar in the $ECLIPSE_HOME/plugins/ directory, where $ECLIPSE_HOME is your eclipse installation directory.
        • Optionally edit the file $ECLIPSE_HOME/eclipse.ini and add the parameter "-clean" (no quotes) to make sure the deployment is fresh
        • Start eclipse.
        • Open a .pig script or make a new one. It should come up with syntax colouring etc. when you start keying in your script. If it doesn't, post your report here
        • When you save a script for the first time there might be a slight delay. normal. Any errors in your script should be marked up so you can see them.
        • To run a script on the cluster, first open the eclipse "preferences" dialog => "PigPen" => and change the settings to your liking. You must have configuration.path pointing to where your pig config files are located (typically $PIG_HOME/conf/). You should specify your preferred pig runtime jar (typically $PIG_HOME/pig-x.x.x-core.jar). log.path defaults to your temp directory, but you can set it to whatever you like. ssh.gateway is untested, so if you don't use it, delete that key. Go ahead and click on "OK".
        • Select the script you want to run from the package explorer and click the little pig icon on the toolbar. This kicks off a new JVM and submits your job. You can track it in the console. If you want to cancel it for some reason, click the little pig icon with the red cross next to the console tab. This simply kills the JVM process.

        That's it for now. have fun...

        Show
        Robert Gibbon added a comment - Here's the README Download the latest version of the jar binary. Right now that's org.apache.pig.pigpen_0.7.4.jar Put the jar in the $ECLIPSE_HOME/plugins/ directory, where $ECLIPSE_HOME is your eclipse installation directory. Optionally edit the file $ECLIPSE_HOME/eclipse.ini and add the parameter "-clean" (no quotes) to make sure the deployment is fresh Start eclipse. Open a .pig script or make a new one. It should come up with syntax colouring etc. when you start keying in your script. If it doesn't, post your report here When you save a script for the first time there might be a slight delay. normal . Any errors in your script should be marked up so you can see them. To run a script on the cluster, first open the eclipse "preferences" dialog => "PigPen" => and change the settings to your liking. You must have configuration.path pointing to where your pig config files are located (typically $PIG_HOME/conf/). You should specify your preferred pig runtime jar (typically $PIG_HOME/pig-x.x.x-core.jar). log.path defaults to your temp directory, but you can set it to whatever you like. ssh.gateway is untested, so if you don't use it, delete that key. Go ahead and click on "OK". Select the script you want to run from the package explorer and click the little pig icon on the toolbar. This kicks off a new JVM and submits your job. You can track it in the console. If you want to cancel it for some reason, click the little pig icon with the red cross next to the console tab. This simply kills the JVM process. That's it for now. have fun...
        Hide
        Robert Gibbon added a comment -

        Added support for Windows and environments where only a JRE has been installed (no need for JDK)

        Show
        Robert Gibbon added a comment - Added support for Windows and environments where only a JRE has been installed (no need for JDK)
        Hide
        Benjamin Reed added a comment -

        hi robert, i'm giving your plugin a try. it detects that my script is a pig script and puts a pig icon next to the file name. the syntax highlighting also seems to be working. for some reason it says that there is a syntax error and it doesn't give a line number. the error is "failed to parse script". My script is one line: a = load 'frog';

        any ideas what is going on?

        Show
        Benjamin Reed added a comment - hi robert, i'm giving your plugin a try. it detects that my script is a pig script and puts a pig icon next to the file name. the syntax highlighting also seems to be working. for some reason it says that there is a syntax error and it doesn't give a line number. the error is "failed to parse script". My script is one line: a = load 'frog'; any ideas what is going on?
        Hide
        Robert Gibbon added a comment -

        Yep, it's likely you're missing the JDT/Core plugin in your Eclipse installation. I'm planning to push a new release in the next couple of days, and I'll remove the dependency at that time.

        Show
        Robert Gibbon added a comment - Yep, it's likely you're missing the JDT/Core plugin in your Eclipse installation. I'm planning to push a new release in the next couple of days, and I'll remove the dependency at that time.
        Hide
        Yan Zhou added a comment -

        Currently ILLUSTRATE-based PigPen is currently not supported. We have a plan to revive the feature for 0.9. Please refer to http://wiki.apache.org/pig/PigIllustrate for details of that effort.

        Show
        Yan Zhou added a comment - Currently ILLUSTRATE-based PigPen is currently not supported. We have a plan to revive the feature for 0.9. Please refer to http://wiki.apache.org/pig/PigIllustrate for details of that effort.
        Hide
        Benjamin Reed added a comment -

        yeah, i just have the basic eclipse for java developers i'll wait for your next release.

        Show
        Benjamin Reed added a comment - yeah, i just have the basic eclipse for java developers i'll wait for your next release.
        Hide
        Robert Gibbon added a comment -

        Resolves some little issues, adds a configurable classpath option, removes dynamic compilation dependancy

        Show
        Robert Gibbon added a comment - Resolves some little issues, adds a configurable classpath option, removes dynamic compilation dependancy
        Hide
        Benjamin Reed added a comment -

        ok, i'm trying the latest jar. it detects the pig script and syntax highlighting is working. it doesn't indicate an error in the editor, but i'm still getting a "Pig Latin syntaxt error" the description is "script is invalid (scripts need to end with a new line)

        Show
        Benjamin Reed added a comment - ok, i'm trying the latest jar. it detects the pig script and syntax highlighting is working. it doesn't indicate an error in the editor, but i'm still getting a "Pig Latin syntaxt error" the description is "script is invalid (scripts need to end with a new line)
        Hide
        Benjamin Reed added a comment -

        has anyone else succeeded in getting the plugin to work?

        Show
        Benjamin Reed added a comment - has anyone else succeeded in getting the plugin to work?
        Hide
        Robert Gibbon added a comment -

        Maybe you could try terminating your script with a new line? Like this:

        1 x = load 'frog';
        2

        According to the Grunt parser a single line of pig latin is not valid. I don't maintain the grunt parser, so perhaps you could open a ticket if you need to have a single line pig file for some reason.

        Cheers

        Show
        Robert Gibbon added a comment - Maybe you could try terminating your script with a new line? Like this: 1 x = load 'frog'; 2 According to the Grunt parser a single line of pig latin is not valid. I don't maintain the grunt parser, so perhaps you could open a ticket if you need to have a single line pig file for some reason. Cheers
        Hide
        Nero Couvalli added a comment -

        Hi,

        I just copied the .jar file in the /usr/share/eclipse/plugins directory of my ubuntu 10.10 system, but it doesn't show in eclipse environment, there is no syntax colouring or anything that shows me that the plugin is working properly.
        Is there any other step I should do in order for this to work?

        Thank you for your time,

        Nero.

        Show
        Nero Couvalli added a comment - Hi, I just copied the .jar file in the /usr/share/eclipse/plugins directory of my ubuntu 10.10 system, but it doesn't show in eclipse environment, there is no syntax colouring or anything that shows me that the plugin is working properly. Is there any other step I should do in order for this to work? Thank you for your time, Nero.
        Hide
        Robert Gibbon added a comment -

        Hi Nero
        Could you try copying the jar to /usr/lib/eclipse/plugins instead?
        Thanks

        Show
        Robert Gibbon added a comment - Hi Nero Could you try copying the jar to /usr/lib/eclipse/plugins instead? Thanks
        Hide
        Panagiotis Papadimitriou added a comment -

        I installed org.apache.pig.pigpen_0.7.5.jar on Helios and Windows 7. I have a bug report and a question.

        Bug:
        Syntax highlighting works when I open pig script from the workspace. However, when I try to open a pig script using File->Open file I get an error screen (I am attaching a screenshot).

        Question:
        Can I use the plugin to submit jobs to hadoop cluster from my Windows machine?

        Thanks!

        Show
        Panagiotis Papadimitriou added a comment - I installed org.apache.pig.pigpen_0.7.5.jar on Helios and Windows 7. I have a bug report and a question. Bug: Syntax highlighting works when I open pig script from the workspace. However, when I try to open a pig script using File->Open file I get an error screen (I am attaching a screenshot). Question: Can I use the plugin to submit jobs to hadoop cluster from my Windows machine? Thanks!
        Hide
        Andreas Paepcke added a comment -

        I too installed PigPen_0.7.5.jar under Helios, with Hadoop 0.20.2, Pig
        0.8.0, and Windows Vista. After a -clean start of Eclipse I saw four
        traces of PigPen within Eclipse: A 'Launch pig job' button, a 'Kill
        the pig job' button, a PigPen entry under Window->Preferences, and
        syntax highlighting in .pig files.

        Unfortunately, the 'Launch pig job' button seemingly does
        nothing. Nothing appears in the console window. I also don't see a Pig
        perspective, or the special editor with the three tabs at the bottom
        (text, graph, etc.).

        In the Window->Preferences->PigPen I added JobConf properties under
        two keys: ConfigurationPath, and configuration.path. I saw both in
        various pages on the Web. I set both values to ...Pigtrunk/conf.
        My 'Pig jar location' field is set to ...Pigtrunk/pig.jar

        In ...Pigtrunk/conf I have pig.properties, and I also placed copies of
        the (default) Hadoop config files core-site.xml, hdfs-site.xml, and
        mapred-site.xml from ...Hadooptrunk/conf.

        Is there any debugging/logging I can do to help solve this problem?
        For now I'm mostly just after executing scripts within Eclipse (as
        opposed to the visual programming tools).

        Thanks,

        Andreas

        Show
        Andreas Paepcke added a comment - I too installed PigPen_0.7.5.jar under Helios, with Hadoop 0.20.2, Pig 0.8.0, and Windows Vista. After a -clean start of Eclipse I saw four traces of PigPen within Eclipse: A 'Launch pig job' button, a 'Kill the pig job' button, a PigPen entry under Window->Preferences, and syntax highlighting in .pig files. Unfortunately, the 'Launch pig job' button seemingly does nothing. Nothing appears in the console window. I also don't see a Pig perspective, or the special editor with the three tabs at the bottom (text, graph, etc.). In the Window- >Preferences ->PigPen I added JobConf properties under two keys: ConfigurationPath, and configuration.path. I saw both in various pages on the Web. I set both values to ...Pigtrunk/conf. My 'Pig jar location' field is set to ...Pigtrunk/pig.jar In ...Pigtrunk/conf I have pig.properties, and I also placed copies of the (default) Hadoop config files core-site.xml, hdfs-site.xml, and mapred-site.xml from ...Hadooptrunk/conf. Is there any debugging/logging I can do to help solve this problem? For now I'm mostly just after executing scripts within Eclipse (as opposed to the visual programming tools). Thanks, Andreas
        Hide
        Yan Zhou added a comment -

        The ExampleGenerator's interface has been changed in PIG-1712's illustrator_7.patch to use the new logical plan as has been planned for 0.9 release. Specifically, the getExamples method returns a map from the new logical operators to the data bags. Once illustrator_7.patch is committed, likely today, the PigPen will likely need to be modified to use the new logical operators. The constructor remains unchanged at this moment. Once the old plan is completely removed (PIG-1775), the constructor will take the new logical plan too.

        Show
        Yan Zhou added a comment - The ExampleGenerator's interface has been changed in PIG-1712 's illustrator_7.patch to use the new logical plan as has been planned for 0.9 release. Specifically, the getExamples method returns a map from the new logical operators to the data bags. Once illustrator_7.patch is committed, likely today, the PigPen will likely need to be modified to use the new logical operators. The constructor remains unchanged at this moment. Once the old plan is completely removed ( PIG-1775 ), the constructor will take the new logical plan too.
        Hide
        Robert Gibbon added a comment -

        @Andreas, Panagiotis & Yan Zhou,

        I should have more time to look at your issue reports and to revisit the example generator in the coming days - apologies, I've been busy with other things.

        Hang in there

        Show
        Robert Gibbon added a comment - @Andreas, Panagiotis & Yan Zhou, I should have more time to look at your issue reports and to revisit the example generator in the coming days - apologies, I've been busy with other things. Hang in there
        Hide
        Alex Rovner added a comment -

        Anyone monitoring the Issues page on google source? How can I submit patches?

        Show
        Alex Rovner added a comment - Anyone monitoring the Issues page on google source? How can I submit patches?
        Hide
        Otis Gospodnetic added a comment -

        This sounds like something people I'm working with might like.
        Are there any screenshots of what this environment looks like? Thanks!

        Show
        Otis Gospodnetic added a comment - This sounds like something people I'm working with might like. Are there any screenshots of what this environment looks like? Thanks!
        Hide
        Alexander Lehmann added a comment -

        Is it allowed to open new issues for PigPen in the PIG category? The comment thread is a bit difficult to read since it contains all info about the old versions and issues to no longer apply.
        I have noticed a few very minor things I would like to address while trying to get the tool to run in my environment.

        Show
        Alexander Lehmann added a comment - Is it allowed to open new issues for PigPen in the PIG category? The comment thread is a bit difficult to read since it contains all info about the old versions and issues to no longer apply. I have noticed a few very minor things I would like to address while trying to get the tool to run in my environment.
        Hide
        Olga Natkovich added a comment -

        Its is absolutely allowed and encouraged to create new tickets for separate issues. In this case, please, beware that pigpen is not actively developed or supported and is looking for a new loving owner.

        Show
        Olga Natkovich added a comment - Its is absolutely allowed and encouraged to create new tickets for separate issues. In this case, please, beware that pigpen is not actively developed or supported and is looking for a new loving owner.
        Hide
        Alex Rovner added a comment -

        What would ownership entail? I would be glad to take ownership and submit a few fixes etc. What would be a good place to host this project? I couldn't find a way to email the owners to find out how to submit patches etc.

        Show
        Alex Rovner added a comment - What would ownership entail? I would be glad to take ownership and submit a few fixes etc. What would be a good place to host this project? I couldn't find a way to email the owners to find out how to submit patches etc.
        Hide
        Alan Gates added a comment -

        By ownership here Olga didn't mean taking the code out of Pig. Pig owns the code. She meant someone who would be an expert in the area and work on it.

        Just adding a few patches for now would be a great start. File issues on JIRA and attach your patches. The Pig committers will work with you to get them in.

        Show
        Alan Gates added a comment - By ownership here Olga didn't mean taking the code out of Pig. Pig owns the code. She meant someone who would be an expert in the area and work on it. Just adding a few patches for now would be a great start. File issues on JIRA and attach your patches. The Pig committers will work with you to get them in.
        Hide
        Alexander Lehmann added a comment -

        Until now, it is not in the Pig subversion, I assume that would be a first step for a new "owner".
        I have tried to get it to compile with pig 0.8.0, but that turns out to be rather complicated due to api changes (or maybe because I'm not an Eclipse expert ...).
        I'll try to file a new issue if I get it working.

        Show
        Alexander Lehmann added a comment - Until now, it is not in the Pig subversion, I assume that would be a first step for a new "owner". I have tried to get it to compile with pig 0.8.0, but that turns out to be rather complicated due to api changes (or maybe because I'm not an Eclipse expert ...). I'll try to file a new issue if I get it working.
        Hide
        Alex Rovner added a comment -

        Alan,

        Thanks for the clarification. As Alex L mentioned the code currently lives in another repository (Google Code).

        I would think we need to:

        A. move the code to the same repository.
        B. Create a subproject / module or a new project for this.

        Follow the same submissions protocol as you have now in PIG.

        Show
        Alex Rovner added a comment - Alan, Thanks for the clarification. As Alex L mentioned the code currently lives in another repository (Google Code). I would think we need to: A. move the code to the same repository. B. Create a subproject / module or a new project for this. Follow the same submissions protocol as you have now in PIG.
        Hide
        Alexander Lehmann added a comment -

        the google code repository is not current, it has not received commits after the inital checkin Jan 28, 2010

        the changes by Robert are included in the source archives attached to this issue (org.apache.pig.pigpen-0.7.5.tar.gz)

        Show
        Alexander Lehmann added a comment - the google code repository is not current, it has not received commits after the inital checkin Jan 28, 2010 the changes by Robert are included in the source archives attached to this issue (org.apache.pig.pigpen-0.7.5.tar.gz)
        Hide
        Alexander Lehmann added a comment -

        an answer to the question from "Andreas Paepcke" above (if you are still reading):

        I noticed that the current version is build against pig-0.7.0 and doesn't work with pig 0.8.0 due to a change in a method. Eclipse doesn't give an error message, but you will see MethodNotFoundException in /workspace/.metadata/.log

        If you are using pig 0.7.0, it should work.

        Show
        Alexander Lehmann added a comment - an answer to the question from "Andreas Paepcke" above (if you are still reading): I noticed that the current version is build against pig-0.7.0 and doesn't work with pig 0.8.0 due to a change in a method. Eclipse doesn't give an error message, but you will see MethodNotFoundException in /workspace/.metadata/.log If you are using pig 0.7.0, it should work.
        Hide
        sheetal added a comment -

        Hi Shubham,
        I am very new to eclipse and PIG. Can you please help me on how to set configurepath variable in Eclipse. I am using "Indigo Service Release 1"
        Thanks in advance.
        Sheetal

        Show
        sheetal added a comment - Hi Shubham, I am very new to eclipse and PIG. Can you please help me on how to set configurepath variable in Eclipse. I am using "Indigo Service Release 1" Thanks in advance. Sheetal
        Hide
        Sheejo added a comment -

        Hi Everyone,
        If you have any Documentation regarding the Eclipse plug in setup activity, please publish. I am facing some issue while setting up the plug in in Eclipse.
        Thanks in Advance
        Sheejo Rapheal

        Show
        Sheejo added a comment - Hi Everyone, If you have any Documentation regarding the Eclipse plug in setup activity, please publish. I am facing some issue while setting up the plug in in Eclipse. Thanks in Advance Sheejo Rapheal
        Hide
        Alex Rovner added a comment -

        Since this project has long been abandoned, anyone interested in starting an open source pig eclipse plugin project?

        Show
        Alex Rovner added a comment - Since this project has long been abandoned, anyone interested in starting an open source pig eclipse plugin project?
        Hide
        Alan Gates added a comment -

        Since this project has long been abandoned, anyone interested in starting an open source pig eclipse plugin project?

        I wouldn't be able to contribute to such a project, but I think its a great idea. I suspect we'd be ok with migrating the existing code out of Pig into another project if that would be helpful.

        Show
        Alan Gates added a comment - Since this project has long been abandoned, anyone interested in starting an open source pig eclipse plugin project? I wouldn't be able to contribute to such a project, but I think its a great idea. I suspect we'd be ok with migrating the existing code out of Pig into another project if that would be helpful.
        Hide
        Alex Rovner added a comment -

        Alan,

        Thanks for the offer. Since the project was initially integrated with Pig 0.7 most of the code is not usable at this point. I am thinking to create a new project on GitHub and use the PigPen project as a reference point. Would that be ok?

        My short term goal is to get the script validation working within the editor. My first commit will include a simple editor with syntax highlighting only.

        It would be great to have support from the Pig team though as you are the most familiar with the API's. How can we make that happen?

        Thanks
        Alex

        Show
        Alex Rovner added a comment - Alan, Thanks for the offer. Since the project was initially integrated with Pig 0.7 most of the code is not usable at this point. I am thinking to create a new project on GitHub and use the PigPen project as a reference point. Would that be ok? My short term goal is to get the script validation working within the editor. My first commit will include a simple editor with syntax highlighting only. It would be great to have support from the Pig team though as you are the most familiar with the API's. How can we make that happen? Thanks Alex
        Hide
        Gianmarco De Francisci Morales added a comment -

        Hi Alex,
        this is a great idea!
        You might want to check this project out before starting from scratch:
        https://github.com/romainr/PigEditor

        Show
        Gianmarco De Francisci Morales added a comment - Hi Alex, this is a great idea! You might want to check this project out before starting from scratch: https://github.com/romainr/PigEditor
        Hide
        Alexander Lehmann added a comment -

        I always wondered if the pig eclipse plugin wasn't a bit over-complicated since it relies on a specific version of pig instead of running the script inside a ruglar pig installation similar to running e.g. an ant task. Most of the versioning problems come from that, I guess.

        Haven't used it for a while however, I tried running it with pig 0.8 which required some small changes, if this is more complicated with the current version, using the source is probably not easy.

        Show
        Alexander Lehmann added a comment - I always wondered if the pig eclipse plugin wasn't a bit over-complicated since it relies on a specific version of pig instead of running the script inside a ruglar pig installation similar to running e.g. an ant task. Most of the versioning problems come from that, I guess. Haven't used it for a while however, I tried running it with pig 0.8 which required some small changes, if this is more complicated with the current version, using the source is probably not easy.
        Hide
        Alexander Lehmann added a comment -

        I have now tried to run the plugin version I modified for pig 0.8 about a year ago and that works without modification for pig 0.10
        (I have some problems actually running a local job, this seems to be a Windows problem, though)

        If you want to pick up the version from the 0.7.5 version (by Robert Gibbon) to fork a new version that is actively being developed, I can try to submit my fixes based on that.

        Show
        Alexander Lehmann added a comment - I have now tried to run the plugin version I modified for pig 0.8 about a year ago and that works without modification for pig 0.10 (I have some problems actually running a local job, this seems to be a Windows problem, though) If you want to pick up the version from the 0.7.5 version (by Robert Gibbon) to fork a new version that is actively being developed, I can try to submit my fixes based on that.
        Hide
        Alex Rovner added a comment -

        Thanks everyone for the input. I have decided to fork https://github.com/romainr/PigEditor

        Forked repo: https://github.com/alexrovner/PigEditor

        Please feel free to file issues and patches.

        Show
        Alex Rovner added a comment - Thanks everyone for the input. I have decided to fork https://github.com/romainr/PigEditor Forked repo: https://github.com/alexrovner/PigEditor Please feel free to file issues and patches.
        Hide
        gustavo riveros added a comment -

        Alex Rovner I'm new to pig and I have trouble getting started, I've been trying to install PigEditor in Eclipse Helios(sr2), but once I add the following file to the plugins folder of eclipse: "alexrovner-PigEditor-398a1af/PigEclipseUpdateSite/plugin / org.apache.pigeditor_1.0.0.4.jar ", no change occurs within the platform, not start anything pig environment, appreciate your help as I see that you are an expert in pig.
        (is part of an implementation for my thesis)

        Show
        gustavo riveros added a comment - Alex Rovner I'm new to pig and I have trouble getting started, I've been trying to install PigEditor in Eclipse Helios(sr2), but once I add the following file to the plugins folder of eclipse: "alexrovner-PigEditor-398a1af/PigEclipseUpdateSite/plugin / org.apache.pigeditor_1.0.0.4.jar ", no change occurs within the platform, not start anything pig environment, appreciate your help as I see that you are an expert in pig. (is part of an implementation for my thesis)

          People

          • Assignee:
            Robert Gibbon
            Reporter:
            Shubham Chopra
          • Votes:
            7 Vote for this issue
            Watchers:
            22 Start watching this issue

            Dates

            • Created:
              Updated:

              Development