[SOLR-15842] Collection Backup Status doesn't calculate de IndexSizeMb correctly.

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 8.10, 8.10.1
• Fix Version/s: 9.0
• Component/s: Backup/Restore, SolrCloud
• Assignee: Houston Putman
• Reporter: Ricardo Ruiz Maldonado
• Created: 09/Dec/21 21:58
• Updated: 12/May/22 00:26
• Resolved: 24/Jan/22 18:09

Description

When backing up] a collection either for the S3 Repository or the LocalFileSystemRepository, if I provide the async parameter and then check the status of the backup with the REQUESTSTATUS] endpoint, even if the backup finishes successfully, the indexSizeMB parameter is always 0.
If I do a sync backup and wait until it finishes, then the indexSizeMB parameter has the right value.

Here are some examples of the responses for each case:

S3 Backup (Sync)

{
  "responseHeader":{
    "status":0,
    "QTime":30640},
  "success":{
    "10.9.21.42:8983_solr":{
      "responseHeader":{
        "status":0,
        "QTime":5857},
      "response":[
        "startTime","2021-12-09T03:16:14.944860Z",
        "indexFileCount",18,
        "uploadedIndexFileCount",18,
        "indexSizeMB",0.026,
        "uploadedIndexFileMB",0.026,
        "shard","shard2",
        "endTime","2021-12-09T03:16:20.694631Z",
        "shardBackupId","md_shard2_0"]},
    "10.9.21.42:8983_solr":{
      "responseHeader":{
        "status":0,
        "QTime":5891},
      "response":[
        "startTime","2021-12-09T03:16:14.951702Z",
        "indexFileCount",18,
        "uploadedIndexFileCount",18,
        "indexSizeMB",0.133,
        "uploadedIndexFileMB",0.133,
        "shard","shard1",
        "endTime","2021-12-09T03:16:20.735084Z",
        "shardBackupId","md_shard1_0"]}},
  "response":[
    "collection","collection",
    "numShards",2,
    "backupId",0,
    "indexVersion","8.10.1",
    "startTime","2021-12-09T03:16:14.381680Z",
    "indexSizeMB",0.159]}

S3 Backup (async)

{
    "responseHeader": {
        "status": 0,
        "QTime": 4
    },
    "success": {
        "10.9.21.42:8983_solr": {
            "responseHeader": {
                "status": 0,
                "QTime": 2
            }
        },
        "10.9.21.42:8983_solr": {
            "responseHeader": {
                "status": 0,
                "QTime": 3
            }
        },
        "10.9.21.42:8983_solr": {
            "responseHeader": {
                "status": 0,
                "QTime": 0
            },
            "STATUS": "completed",
            "Response": "TaskId: backup120415121643240950269884 webapp=null path=/admin/cores params={core=collectionName_shard2_replica_n4&async=backup120415121643240950269884&qt=/admin/cores&name=shard2&shardBackupId=md_shard2_1&action=BACKUPCORE&location=s3:/b39587e3-c296-4634-b8e2-7ff1e94e6a69/index/backup.1/collectionName/&incremental=true&repository=s3&prevShardBackupId=md_shard2_0&wt=javabin&version=2} status=0 QTime=2"
        },
        "10.9.21.42:8983_solr": {
            "responseHeader": {
                "status": 0,
                "QTime": 0
            },
            "STATUS": "completed",
            "Response": "TaskId: backup120415121643240950730312 webapp=null path=/admin/cores params={core=collectionName_shard1_replica_n1&async=backup120415121643240950730312&qt=/admin/cores&name=shard1&shardBackupId=md_shard1_1&action=BACKUPCORE&location=s3:/b39587e3-c296-4634-b8e2-7ff1e94e6a69/index/backup.1/collectionName/&incremental=true&repository=s3&prevShardBackupId=md_shard1_0&wt=javabin&version=2} status=0 QTime=3"
        }
    },
    "backup120415121643240950269884": {
        "responseHeader": {
            "status": 0,
            "QTime": 0
        },
        "STATUS": "completed",
        "Response": "TaskId: backup120415121643240950269884 webapp=null path=/admin/cores params={core=collectionName_shard2_replica_n4&async=backup120415121643240950269884&qt=/admin/cores&name=shard2&shardBackupId=md_shard2_1&action=BACKUPCORE&location=s3:/b39587e3-c296-4634-b8e2-7ff1e94e6a69/index/backup.1/collectionName/&incremental=true&repository=s3&prevShardBackupId=md_shard2_0&wt=javabin&version=2} status=0 QTime=2"
    },
    "backup120415121643240950730312": {
        "responseHeader": {
            "status": 0,
            "QTime": 0
        },
        "STATUS": "completed",
        "Response": "TaskId: backup120415121643240950730312 webapp=null path=/admin/cores params={core=collectionName_shard1_replica_n1&async=backup120415121643240950730312&qt=/admin/cores&name=shard1&shardBackupId=md_shard1_1&action=BACKUPCORE&location=s3:/b39587e3-c296-4634-b8e2-7ff1e94e6a69/index/backup.1/collectionName/&incremental=true&repository=s3&prevShardBackupId=md_shard1_0&wt=javabin&version=2} status=0 QTime=3"
    },
    "response": [
        "collection",
        "collectionName",
        "numShards",
        2,
        "backupId",
        1,
        "indexVersion",
        "8.10.1",
        "startTime",
        "2021-12-04T06:12:52.540773Z",
        "indexSizeMB",
        0.0
    ],
    "status": {
        "state": "completed",
        "msg": "found [backup12041512] in completed tasks"
    }
}

Issue Links

• relates to: SOLR-15884 unexpected response format in Collection Backup Status
• links to: GitHub Pull Request #479