Did this duplicity backup fail or not?

Discussion in 'Server Operation' started by Ovidiu, Aug 19, 2014.

  1. Ovidiu

    Ovidiu Active Member

    I am running two duplicity profiles. Both logs show similar errors but one says success the other one fail. Anyone able to help me make sense of the errors?

    It seems my full backup always fails and why would they both be running the same day? The full one is scheduled for Sundays yet both ran yesterday on a Monday...

    apparent failure:
    Code:
    Start duply v1.5.5.5, time is 2014-08-17 21:00:01.
    Using profile '/root/.duply/proxmox'.
    Using installed duplicity version 0.6.18, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'GNU Awk 4.0.1', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
    Autoset found secret key of first GPG_KEY entry '27A8AEC1' for signing.
    Test - Encrypt to 27A8AEC1 & Sign with 27A8AEC1 (OK)
    Test - Decrypt (OK)
    Test - Compare (OK)
    Cleanup - Delete '/tmp/duply.382146.1408302001_*'(OK)
    
    --- Start running command FULL at 21:00:01.920 ---
    Import of duplicity.backends.giobackend Failed: No module named gio
    Reading globbing filelist /root/.duply/proxmox/exclude
    Local and Remote metadata are synchronized, no sync needed.
    Last full backup date: Sun Aug 10 21:00:03 2014
    Reuse configured PASSPHRASE as SIGN_PASSPHRASE
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    gaierror: [Errno -2] Name or service not known
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 104] Connection reset by peer
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 104] Connection reset by peer
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 104] Connection reset by peer
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    gaierror: [Errno -2] Name or service not known
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 104] Connection reset by peer
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 733, in _mexe
        request.body, request.headers)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 622, in sender
        response.status, response.reason, body)
    S3ResponseError: S3ResponseError: 400 Bad Request
    <?xml version="1.0" encoding="UTF-8"?>
    <Error><Code>BadDigest</Code><Message>The Content-MD5 you specified did not match what we received.</Message><ExpectedDigest>OUaJDsZ5YNUN3mDFqERpkQ==</ExpectedDigest><CalculatedDigest>2zka+wVsnRjxn2EEmI+xdg==</CalculatedDigest><RequestId>15893E5688913B33</RequestId><HostId>AwXCq54Zw/nEaS1pH0Rf1Qdai/EIASecvkXvUJJQ2gHN/lPqsYQjyBZ277+8dUc36CxWKvxCk60=</HostId></Error>
    Traceback (most recent call last):
      File "/usr/bin/duplicity", line 1404, in <module>
        with_tempdir(main)
      File "/usr/bin/duplicity", line 1397, in with_tempdir
        fn()
      File "/usr/bin/duplicity", line 1367, in main
        full_backup(col_stats)
      File "/usr/bin/duplicity", line 506, in full_backup
        sig_outfp.to_remote()
      File "/usr/lib/python2.7/dist-packages/duplicity/dup_temp.py", line 184, in to_remote
        globals.backend.move(tgt) [email protected]
      File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 364, in move
        source_path.delete()
      File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 567, in delete
        util.ignore_missing(os.unlink, self.name)
      File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 116, in ignore_missing
        fn(filename)
    OSError: [Errno 2] No such file or directory: '/root/.cache/duplicity/duply_proxmox/duplicity-full-signatures.20140817T190012Z.sigtar.gpg'
    
    10:51:19.990 Task 'FULL' failed with exit code '30'.
    --- Finished state FAILED 'code 30' at 10:51:19.990 - Runtime 37:51:18.069 ---
    apparent success:
    Code:
    Start duply v1.5.5.5, time is 2014-08-18 17:00:01.
    Using profile '/root/.duply/proxmox'.
    Using installed duplicity version 0.6.18, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'GNU Awk 4.0.1', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
    Autoset found secret key of first GPG_KEY entry '27A8AEC1' for signing.
    Test - Encrypt to 27A8AEC1 & Sign with 27A8AEC1 (OK)
    Test - Decrypt (OK)
    Test - Compare (OK)
    Cleanup - Delete '/tmp/duply.413187.1408374002_*'(OK)
    
    --- Start running command PRE at 17:00:02.192 ---
    Skipping n/a script '/root/.duply/proxmox/pre'.
    --- Finished state OK at 17:00:02.202 - Runtime 00:00:00.009 ---
    
    --- Start running command BKP at 17:00:02.210 ---
    Import of duplicity.backends.giobackend Failed: No module named gio
    Reading globbing filelist /root/.duply/proxmox/exclude
    Local and Remote metadata are synchronized, no sync needed.
    Last full backup left a partial set, restarting.
    Last full backup date: Sun Aug 17 21:00:12 2014
    Reuse configured PASSPHRASE as SIGN_PASSPHRASE
    RESTART: Volumes 28 to 29 failed to upload before termination.
             Restarting backup at volume 28.
    Restarting after volume 27, file root/.cache/duplicity/duply_diladele/duplicity-new-signatures.20140807T160149Z.to.20140808T160144Z.sigtar.gz, block 380
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 104] Connection reset by peer
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 104] Connection reset by peer
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    gaierror: [Errno -2] Name or service not known
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    gaierror: [Errno -2] Name or service not known
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_multi.py", line 398, in _upload
        mp.upload_part_from_file(fd, offset + 1, cb=_upload_callback)
      File "/usr/lib/python2.7/dist-packages/boto/s3/multipart.py", line 236, in upload_part_from_file
        query_args=query_args, size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 905, in set_contents_from_file
        size=size)
      File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 663, in send_file
        query_args=query_args)
      File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 450, in make_request
        override_num_retries=override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 829, in make_request
        return self._mexe(http_request, sender, override_num_retries)
      File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 794, in _mexe
        raise e
    error: [Errno 32] Broken pipe
    --------------[ Backup Statistics ]--------------
    StartTime 1408374028.85 (Mon Aug 18 17:00:28 2014)
    EndTime 1408434944.44 (Tue Aug 19 09:55:44 2014)
    ElapsedTime 60915.59 (16 hours 55 minutes 15.59 seconds)
    SourceFiles 38770
    SourceFileSize 1896702162 (1.77 GB)
    NewFiles 38770
    NewFileSize 1896699602 (1.77 GB)
    DeletedFiles 0
    ChangedFiles 0
    ChangedFileSize 0 (0 bytes)
    ChangedDeltaSize 0 (0 bytes)
    DeltaEntries 38770
    RawDeltaSize 1880825350 (1.75 GB)
    TotalDestinationSizeChange 597846932 (570 MB)
    Errors 0
    -------------------------------------------------
    
    --- Finished state OK at 10:44:04.833 - Runtime 17:44:02.623 ---
    
    --- Start running command POST at 10:44:04.859 ---
    Skipping n/a script '/root/.duply/proxmox/post'.
    --- Finished state OK at 10:44:04.882 - Runtime 00:00:00.023 ---
    Here are the relevant crontab entries:
    Code:
    #run proxmox backup every day except Sunday
    0 17 * * 1-6 /usr/bin/duply proxmox backup
    #run full proxmox backup every Sunday
    0 21 * * 0 /usr/bin/duply proxmox full
    
    I checked the backup status and these two are both missing WTF!?

    Code:
    [email protected]:~# [B]duply proxmox status[/B]
    Start duply v1.5.5.5, time is 2014-08-19 18:40:44.
    Using profile '/root/.duply/proxmox'.
    Using installed duplicity version 0.6.18, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'GNU Awk 4.0.1', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
    Autoset found secret key of first GPG_KEY entry '27A8AEC1' for signing.
    Test - Encrypt to 27A8AEC1 & Sign with 27A8AEC1 (OK)
    Test - Decrypt (OK)
    Test - Compare (OK)
    Cleanup - Delete '/tmp/duply.453181.1408466444_*'(OK)
    
    --- Start running command STATUS at 18:40:45.068 ---
    
    Import of duplicity.backends.giobackend Failed: No module named gio
    Local and Remote metadata are synchronized, no sync needed.
    Last full backup date: Sun Aug 17 21:00:12 2014
    Collection Status
    -----------------
    Connecting with backend: BotoBackend
    Archive dir: /root/.cache/duplicity/duply_proxmox
    
    Found 4 secondary backup chains.
    Secondary chain 1 of 4:
    -------------------------
    Chain start time: Fri Jul 18 17:09:24 2014
    Chain end time: Fri Jul 25 17:00:04 2014
    Number of contained backup sets: 6
    Total number of contained volumes: 116
     Type of backup set:                            Time:      Num volumes:
                    Full         Fri Jul 18 17:09:24 2014                15
    
             Incremental         Mon Jul 21 17:00:03 2014                 3
             Incremental         Tue Jul 22 17:00:03 2014                 1
             Incremental         Wed Jul 23 17:00:08 2014                 1
             Incremental         Thu Jul 24 17:00:04 2014                63
             Incremental         Fri Jul 25 17:00:04 2014                33
    -------------------------
    
    Secondary chain 2 of 4:
    -------------------------
    Chain start time: Sun Jul 27 21:00:03 2014
    Chain end time: Sat Aug  2 17:00:04 2014
    Number of contained backup sets: 7
    Total number of contained volumes: 35
     Type of backup set:                            Time:      Num volumes:
                    Full         Sun Jul 27 21:00:03 2014                24
             Incremental         Mon Jul 28 17:00:04 2014                 2
             Incremental         Tue Jul 29 17:00:04 2014                 1
             Incremental         Wed Jul 30 17:00:04 2014                 2
             Incremental         Thu Jul 31 17:00:04 2014                 2
             Incremental         Fri Aug  1 17:00:04 2014                 2
             Incremental         Sat Aug  2 17:00:04 2014                 2
    -------------------------
    
    Secondary chain 3 of 4:
    -------------------------
    Chain start time: Sun Aug  3 21:00:03 2014
    Chain end time: Sat Aug  9 17:00:04 2014
    Number of contained backup sets: 7
    Total number of contained volumes: 44
     Type of backup set:                            Time:      Num volumes:
                    Full         Sun Aug  3 21:00:03 2014                32
             Incremental         Mon Aug  4 17:00:03 2014                 2
             Incremental         Tue Aug  5 17:00:04 2014                 2
             Incremental         Wed Aug  6 17:00:08 2014                 2
             Incremental         Thu Aug  7 17:00:05 2014                 2
             Incremental         Fri Aug  8 17:00:04 2014                 2
             Incremental         Sat Aug  9 17:00:04 2014                 2
    -------------------------
    
    Secondary chain 4 of 4:
    -------------------------
    Chain start time: Sun Aug 10 21:00:03 2014
    Chain end time: Sat Aug 16 17:00:06 2014
    Number of contained backup sets: 7
    Total number of contained volumes: 53
     Type of backup set:                            Time:      Num volumes:
                    Full         Sun Aug 10 21:00:03 2014                40
             Incremental         Mon Aug 11 17:00:03 2014                 3
             Incremental         Tue Aug 12 17:00:12 2014                 2
             Incremental         Wed Aug 13 17:00:12 2014                 2
             Incremental         Thu Aug 14 17:00:12 2014                 2
             Incremental         Fri Aug 15 17:00:14 2014                 2
             Incremental         Sat Aug 16 17:00:06 2014                 2
    -------------------------
    
    
    Found primary backup chain with matching signature chain:
    -------------------------
    Chain start time: Sun Aug 17 21:00:12 2014
    Chain end time: Tue Aug 19 17:00:12 2014
    Number of contained backup sets: 2
    Total number of contained volumes: 52
     Type of backup set:                            Time:      Num volumes:
                    Full         Sun Aug 17 21:00:12 2014                50
             Incremental         Tue Aug 19 17:00:12 2014                 2
    -------------------------
    No orphaned or incomplete backup sets found.
    --- Finished state OK at 18:41:07.966 - Runtime 00:00:22.897 ---
    
     
  2. Ovidiu

    Ovidiu Active Member

    I'm about to email to the duplicity mailing list too, posting here just in case.

    And here is another log from another backup:

    Code:
    Start duply v1.5.5.5, time is 2014-08-19 18:00:01.
    Using profile '/root/.duply/diladele'.
    Using installed duplicity version 0.6.18, python 2.7.3, gpg 1.4.12 (Home: ~/.gnupg), awk 'GNU Awk 4.0.1', bash '4.2.37(1)-release (x86_64-pc-linux-gnu)'.
    Autoset found secret key of first GPG_KEY entry '27A8AEC1' for signing.
    Test - Encrypt to 27A8AEC1 & Sign with 27A8AEC1 (OK)
    Test - Decrypt (OK)
    Test - Compare (OK)
    Cleanup - Delete '/tmp/duply.451590.1408464002_*'(OK)
    
    --- Start running command PRE at 18:00:02.221 ---
    Running '/root/.duply/diladele/pre' - OK
    Output: INFO: starting new backup job: vzdump 105 --dumpdir /var/lib/vz/backups --compress 0
    INFO: Starting Backup of VM 105 (qemu)
    INFO: status = running
    INFO: update VM 105: -lock backup
    INFO: backup mode: snapshot
    INFO: ionice priority: 7
    INFO: creating archive '/var/lib/vz/backups/vzdump-qemu-105-2014_08_19-18_00_03.vma'
    INFO: started backup task '171c8cd8-2acc-4ca3-b960-38b4920a9e7f'
    INFO: status: 1% (639827968/53687091200), sparse 0% (336482304), duration 3, 213/101 MB/s
    INFO: status: 2% (1115619328/53687091200), sparse 0% (369917952), duration 9, 79/73 MB/s
    INFO: status: 3% (1634729984/53687091200), sparse 0% (420761600), duration 16, 74/66 MB/s
    INFO: status: 4% (2171535360/53687091200), sparse 0% (455528448), duration 23, 76/71 MB/s
    INFO: status: 5% (2724462592/53687091200), sparse 1% (635297792), duration 29, 92/62 MB/s
    INFO: status: 6% (3231252480/53687091200), sparse 1% (664129536), duration 39, 50/47 MB/s
    INFO: status: 11% (6294863872/53687091200), sparse 6% (3240710144), duration 49, 306/48 MB/s
    INFO: status: 13% (7009337344/53687091200), sparse 7% (3778846720), duration 52, 238/58 MB/s
    INFO: status: 20% (11035017216/53687091200), sparse 13% (7423401984), duration 59, 575/54 MB/s
    INFO: status: 28% (15438446592/53687091200), sparse 21% (11628494848), duration 62, 1467/66 MB/s
    INFO: status: 32% (17662803968/53687091200), sparse 25% (13704683520), duration 65, 741/49 MB/s
    INFO: status: 33% (17838505984/53687091200), sparse 25% (13704691712), duration 68, 58/58 MB/s
    INFO: status: 34% (18257543168/53687091200), sparse 25% (13798801408), duration 75, 59/46 MB/s
    INFO: status: 37% (19903086592/53687091200), sparse 28% (15221899264), duration 78, 548/74 MB/s
    INFO: status: 38% (20773076992/53687091200), sparse 29% (15811874816), duration 83, 173/56 MB/s
    INFO: status: 43% (23379640320/53687091200), sparse 34% (18266951680), duration 86, 868/50 MB/s
    INFO: status: 52% (28316925952/53687091200), sparse 42% (23016411136), duration 89, 1645/62 MB/s
    INFO: status: 77% (41520332800/53687091200), sparse 67% (36123357184), duration 92, 4401/32 MB/s
    INFO: status: 87% (47092531200/53687091200), sparse 77% (41530245120), duration 95, 1857/55 MB/s
    INFO: status: 94% (50601328640/53687091200), sparse 83% (44829790208), duration 98, 1169/69 MB/s
    INFO: status: 100% (53687091200/53687091200), sparse 89% (47915552768), duration 99, 3085/0 MB/s
    INFO: transferred 53687 MB in 99 seconds (542 MB/s)
    INFO: archive file size: 5.38GB
    INFO: Finished Backup of VM 105 (00:01:41)
    INFO: Backup job finished successfully
    /var/lib/vz/backups/vzdump-qemu-105-2014_08_19-18_00_03.log not renamed: /var/lib/vz/backups/vzdump-qemu-105.log already exists
    /var/lib/vz/backups/vzdump-qemu-105-2014_08_19-18_00_03.vma not renamed: /var/lib/vz/backups/vzdump-qemu-105.vma already exists
    --- Finished state OK at 18:01:44.833 - Runtime 00:01:42.611 ---
    
    --- Start running command BKP at 18:01:44.873 ---
    Import of duplicity.backends.giobackend Failed: No module named gio
    Reading globbing filelist /root/.duply/diladele/exclude
    Local and Remote metadata are synchronized, no sync needed.
    Last full backup left a partial set, restarting.
    Last full backup date: Sun Aug 17 23:00:14 2014
    Reuse configured PASSPHRASE as SIGN_PASSPHRASE
    RESTART: Volumes 22 to 22 failed to upload before termination.
             Restarting backup at volume 22.
    Restarting after volume 21, file vm-105-disk-1.qcow2, block 89067
    Traceback (most recent call last):
      File "/usr/bin/duplicity", line 1404, in <module>
        with_tempdir(main)
      File "/usr/bin/duplicity", line 1397, in with_tempdir
        fn()
      File "/usr/bin/duplicity", line 1367, in main
        full_backup(col_stats)
      File "/usr/bin/duplicity", line 506, in full_backup
        sig_outfp.to_remote()
      File "/usr/lib/python2.7/dist-packages/duplicity/dup_temp.py", line 184, in to_remote
        globals.backend.move(tgt) [email protected]
      File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 364, in move
        source_path.delete()
      File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 567, in delete
        util.ignore_missing(os.unlink, self.name)
      File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 116, in ignore_missing
        fn(filename)
    OSError: [Errno 2] No such file or directory: '/root/.cache/duplicity/duply_diladele/duplicity-full-signatures.20140817T210014Z.sigtar.gpg'
    
    19:22:54.708 Task 'BKP' failed with exit code '30'.
    --- Finished state FAILED 'code 30' at 19:22:54.708 - Runtime 01:21:09.835 ---
    
    --- Start running command POST at 19:22:54.722 ---
    Running '/root/.duply/diladele/post' - OK
    --- Finished state OK at 19:23:03.207 - Runtime 00:00:08.485 ---
     
  3. Ovidiu

    Ovidiu Active Member

    Anyone?

    My differential backups run just fine but all full backups fail as can be seen above. Any ideas?
     

Share This Page