Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      This web service will:

      • provide cross language compatibility.
      • provide data access without direct access to nodes.
      • provide work-around for port restrictions.

        Activity

        Hide
        Tim Williams added a comment -

        I realize the desire to avoid additional dependencies, but HTTP isn't where you want to skimp. You may not want Jersey or somesuch, but at least just embed Jetty for the serving itself.

        Show
        Tim Williams added a comment - I realize the desire to avoid additional dependencies, but HTTP isn't where you want to skimp. You may not want Jersey or somesuch, but at least just embed Jetty for the serving itself.
        Hide
        Michael Wall added a comment -

        Hey David,

        Interesting class. What exactly is it doing besides opening a socket? I would have expected to see it look at the incoming request and then either retrieve data or store data. Maybe I didn't understand the intent of your ticket. I can certainly appreciate simplicity. What I have in mind is more like http://wiki.apache.org/hadoop/Hbase/HbaseRest. It will take some effort to make it useful and performant.

        Looks this is still a ways off if ACCUMULO-482 needs to be fixed first. I'll start a separate project on github with my ideas and post a link when I have something useful.

        Thanks

        Show
        Michael Wall added a comment - Hey David, Interesting class. What exactly is it doing besides opening a socket? I would have expected to see it look at the incoming request and then either retrieve data or store data. Maybe I didn't understand the intent of your ticket. I can certainly appreciate simplicity. What I have in mind is more like http://wiki.apache.org/hadoop/Hbase/HbaseRest . It will take some effort to make it useful and performant. Looks this is still a ways off if ACCUMULO-482 needs to be fixed first. I'll start a separate project on github with my ideas and post a link when I have something useful. Thanks
        Hide
        jv added a comment -

        My comment on 482 does apply a bit here as well. The short answer is we want a client side proxy, not only to allow simple apis, but we also want to avoid letting client code see !METADATA entries

        Show
        jv added a comment - My comment on 482 does apply a bit here as well. The short answer is we want a client side proxy, not only to allow simple apis, but we also want to avoid letting client code see !METADATA entries
        Hide
        David Medinets added a comment -

        I was thinking something much simpler. Something that could be just a java process so there are few dependencies. Perhaps something like this:

        public class SingleFileHTTPServer extends Thread {

        private static Logger logger = Logger.getLogger(CriteriaWatchServer.class);

        private int port = 9876;

        private String encoding = "ASCII";

        private int pagesServed = 1;

        private IDB db = null;

        UrlRequestProcessor urlRequestProcessor = null;

        private boolean endProcess = false;

        public SingleFileHTTPServer() throws IOException

        { StopWatch.add("httpServer"); }

        public void run() {
        //logger.debug("SingleFileHTTPServer; running.");

        try {
        ServerSocket server = new ServerSocket(this.port);
        server.setSoTimeout(1000);
        logger.info("Accepting connections on port " + server.getLocalPort());

        while (!endProcess) {
        Socket connection = null;
        int bufferSize = 1000;
        StringBuffer request = new StringBuffer(bufferSize);

        try {
        connection = server.accept();
        OutputStream out = new BufferedOutputStream(connection.getOutputStream());
        InputStream in = new BufferedInputStream(connection.getInputStream());
        int bufferIndex = 0;
        while (bufferIndex++ < bufferSize)

        { int c = in.read(); if (c == '\r' || c == '\n' || c == -1) break; request.append((char) c); // If this is HTTP/1.0 or later send a MIME header }

        DecimalFormat myFormat = (DecimalFormat) DecimalFormat.getInstance();
        myFormat.setDecimalSeparatorAlwaysShown(true);

        //logger.fatal("qs: " + request.toString());

        try

        { out.write(urlRequestProcessor.process(request.toString()).getBytes(encoding)); }

        catch (StopServerException e)

        { logger.debug("Stopping the Server."); endProcess = true; }

        //logger.debug("SingleFileHTTPServer; running serving page [" + myFormat.format(pagesServed) + "].");

        out.flush();
        out.close();

        pagesServed++;

        } catch (InterruptedIOException e)

        { // ignore this error. when the access() times out, then // a new accept() is started. The timeout lets calling // processes easily stop the server. }

        catch (IOException e)

        { logger.error("SingleFileHTTPServer; IOException [" + e.getMessage() + "]"); }

        finally {
        if (connection != null)

        { //logger.debug("SingleFileHTTPServer; closing connection."); connection.close(); }

        }

        } // end while(!endProcess)

        } // end try
        catch (IOException e)

        { // LogConfiguration.fatal("F063: Could not start HTTP server. Port // Occupied, probably another process is already running."); }

        } // end run
        }

        This would make a swell separate github project as well.

        Show
        David Medinets added a comment - I was thinking something much simpler. Something that could be just a java process so there are few dependencies. Perhaps something like this: public class SingleFileHTTPServer extends Thread { private static Logger logger = Logger.getLogger(CriteriaWatchServer.class); private int port = 9876; private String encoding = "ASCII"; private int pagesServed = 1; private IDB db = null; UrlRequestProcessor urlRequestProcessor = null; private boolean endProcess = false; public SingleFileHTTPServer() throws IOException { StopWatch.add("httpServer"); } public void run() { //logger.debug("SingleFileHTTPServer; running."); try { ServerSocket server = new ServerSocket(this.port); server.setSoTimeout(1000); logger.info("Accepting connections on port " + server.getLocalPort()); while (!endProcess) { Socket connection = null; int bufferSize = 1000; StringBuffer request = new StringBuffer(bufferSize); try { connection = server.accept(); OutputStream out = new BufferedOutputStream(connection.getOutputStream()); InputStream in = new BufferedInputStream(connection.getInputStream()); int bufferIndex = 0; while (bufferIndex++ < bufferSize) { int c = in.read(); if (c == '\r' || c == '\n' || c == -1) break; request.append((char) c); // If this is HTTP/1.0 or later send a MIME header } DecimalFormat myFormat = (DecimalFormat) DecimalFormat.getInstance(); myFormat.setDecimalSeparatorAlwaysShown(true); //logger.fatal("qs: " + request.toString()); try { out.write(urlRequestProcessor.process(request.toString()).getBytes(encoding)); } catch (StopServerException e) { logger.debug("Stopping the Server."); endProcess = true; } //logger.debug("SingleFileHTTPServer; running serving page [" + myFormat.format(pagesServed) + "] ."); out.flush(); out.close(); pagesServed++; } catch (InterruptedIOException e) { // ignore this error. when the access() times out, then // a new accept() is started. The timeout lets calling // processes easily stop the server. } catch (IOException e) { logger.error("SingleFileHTTPServer; IOException [" + e.getMessage() + "]"); } finally { if (connection != null) { //logger.debug("SingleFileHTTPServer; closing connection."); connection.close(); } } } // end while(!endProcess) } // end try catch (IOException e) { // LogConfiguration.fatal("F063: Could not start HTTP server. Port // Occupied, probably another process is already running."); } } // end run } This would make a swell separate github project as well.
        Hide
        Josh Elser added a comment -

        ACCUMULO-482 might be good to keep in mind for this. Also, take this comment as an offer of developer support too.

        Show
        Josh Elser added a comment - ACCUMULO-482 might be good to keep in mind for this. Also, take this comment as an offer of developer support too.
        Hide
        Michael Wall added a comment -

        I have been thinking about this very problem for quite some time. What is the suggested approach, Thrift? I have some JRuby classes that wrap an earlier version of Accumulo's Java API and had hoped to find time to port that to the latest version. It would then be fairly easy to add a Rails app on top of that and provide a RESTful interface. The Rails app could easily run on the JVM via JRuby. I envisioned an API like this:

        GET table/colf/colq
        PUT table/colf/colq with the data being the value for a new record
        POST table/colf/colq with data being the value to update a record
        GET table/colf?startcolq=abc&endcolq=def
        DELETE table/colf/colq

        We could return xml, json, or whatever by adding a .json or .xml to the request. Handling authentication might take some work. Need to provide a way to give the user column visibilities. I have some generic ideas there as well. I haven't worked out in my mind yet how to handle paging.

        Expanding that, we could then provide client libraries in different languages, like the MongoDB drivers. If we provided a Javascript one, we would need to support JSONP, Dojo's window.name transport or some other hack to get around Javascript's SOP restriction so you could POST, PUT and GET from a different host.

        I have no idea how this approach would compare to a Thrift service. Whatever the design, I would love to help work on this.

        Show
        Michael Wall added a comment - I have been thinking about this very problem for quite some time. What is the suggested approach, Thrift? I have some JRuby classes that wrap an earlier version of Accumulo's Java API and had hoped to find time to port that to the latest version. It would then be fairly easy to add a Rails app on top of that and provide a RESTful interface. The Rails app could easily run on the JVM via JRuby. I envisioned an API like this: GET table/colf/colq PUT table/colf/colq with the data being the value for a new record POST table/colf/colq with data being the value to update a record GET table/colf?startcolq=abc&endcolq=def DELETE table/colf/colq We could return xml, json, or whatever by adding a .json or .xml to the request. Handling authentication might take some work. Need to provide a way to give the user column visibilities. I have some generic ideas there as well. I haven't worked out in my mind yet how to handle paging. Expanding that, we could then provide client libraries in different languages, like the MongoDB drivers. If we provided a Javascript one, we would need to support JSONP, Dojo's window.name transport or some other hack to get around Javascript's SOP restriction so you could POST, PUT and GET from a different host. I have no idea how this approach would compare to a Thrift service. Whatever the design, I would love to help work on this.

          People

          • Assignee:
            Unassigned
            Reporter:
            David Medinets
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development