Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-527

GCE Libcloud Driver list_nodes and list_volumes Scaling Improvement

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.14.0-beta3
    • None
    • None

    Description

      The current code for the GCE Driver has two significant bottle necks in the list_nodes and list_volumes code. In both cases the slowdown can be traced to the response to object methods _to_node and _to_storage_volume. These methods are called in serial from both list_node and list_volume methods and each call to one of these methods requires an additional http request. For list_nodes the price is doubled since each _to_node request requires a _to_storage_volume request.

      Basically, list_nodes and list_volumes require 1 + 2 * Nodes and 1 + Volumes requests to complete. I've found that at large numbers of nodes this is a considerable pain point.

      I've been working on a branch at
      https://github.com/RussellSpitzer/libcloud/tree/GCE_perf_to_node
      that I would be happy to contribute upstream if there is any desire for it.

      Attachments

        Activity

          People

            erjohnso Eric Johnson
            rspitzer Russell Spitzer
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: