Details
-
Wish
-
Status: Closed
-
Minor
-
Resolution: Not A Problem
-
None
-
None
-
None
-
None
Description
Consider a midlet (for iPhone and Blackberry) that puts up status of HBase, HDFS, and MapReduce services. Extend Stargate for back end support. (Support active-active failover of Stargate service.) Visualize cluster status with a dashboard. Make the summary view fit without needing to scroll. Summarize load with a weather motif. For detail provide a navigable tree view: service -> rack -> server -> daemon -> resource. Display metrics and time series graphs sourced from Ganglia/RRD in the leaves if available, with controls for selecting metric and time range filtered for the appropriate resource or aggregate view. For HBase, visualize region deployment and key and query densities by region; also, load and performance metrics by region. Buttons for control actions at cluster, server, and daemon scope. Control actions for MapReduce job tracker. Command line with auto-completion. Support a background mode that maintains a connection open to the cluster and receives push telemetry. Optimize communication for high latency lossy transport. (Could be based on Avro over compressed SSH tunnelling with suitable framing and flushing.) If possible the Stargate extension should monitor the telemetry stream and do non-parametric outlier detection (incremental LOF?). (The cluster could help with that, look at distributed outlier detection results in the Sensor Networks literature.) As appropriate alert the device user and provide shortcut(s) on the dashboard to most relevant resource that supports control actions.
While travelling in New York I had the "opportunity" to do wizard level cluster diagnostics and recovery over ssh on the Blackberry while sitting in Times Square. Let's just say it was inconvenient. So the above is my wish. 