How to resume a large SCP file transfer on Linux

Question: I was downloading a large file using SCP, but the download transfer failed in the middle because my laptop got disconnected from the network. Is there a way to resume the interrupted SCP transfer where I left off, instead of downloading the file all over again?

Originally based on BSD RCP protocol, SCP (Secure copy) is a mechanism that allows you to transfer a file between two end points over a secure SSH connection. However, as a simple secure copy protocol, SCP does not understand range-request or partial transfer like HTTP does. As such, popular SCP implementations like the scp command line tool cannot resume aborted downloads from lost network connections.

If you want to resume an interrupted SCP transfer, you need to rely on other programs which support range requests. One popular such program is rsync. Similar to scp, rsync can also transfer files over SSH.

Suppose you were trying to download a file (bigdata.tgz) from a remote host remotehost.com using scp, but the SCP transfer was stopped in the middle due to a stalled SSH connection. You can use the following rsync command to easily resume the stopped transfer. Note that the remote server must have rsync installed as well.

$ cd /path/to/directory/of/partially_downloaded_file
$ rsync -P --rsh=ssh userid@remotehost.com:bigdata.tgz ./bigdata.tgz

The "-P" option is the same as "--partial --progress", allowing rsync to work with partially downloaded files. The "--rsh=ssh" option tells rsync to use ssh as a remote shell.

Once the command is invoked, rsync processes on local and remote hosts compare a local file (./bigdata.tgz) and a remote file (userid@remotehost.com:bigdata.tgz), determine among themselves what portion of the file is not the same, and transfer the discrepancy to either end. In this case, missing bytes in the partially downloaded local file is downloaded from a remote host.

If the above rsync session itself gets interrupted, you can resume it as many time as you want by typing the same command. rsync will automatically restart the transfer where it left off.

Download this article as ad-free PDF (made possible by your kind donation): 
Download PDF

Subscribe to Ask Xmodulo

Do you want to receive Linux related questions & answers published at Ask Xmodulo? Enter your email address below, and we will deliver our Linux Q&A straight to your email box, for free. Delivery powered by Google Feedburner.


Support Xmodulo

Did you find this tutorial helpful? Then please be generous and support Xmodulo!

4 thoughts on “How to resume a large SCP file transfer on Linux

  1. Alternatively, one can use the sftp command. The underlying SFTP protocol, like SCP built on top of SSH, but far more sophisticated and better documented, does support transfer resumption.

  2. The title here is a little confusing. Although rsync works over ssh, it isn't guaranteed to be available whenever scp is available, and indeed is much less available than the sftp solution offered by KLR.

    I would agree that WHEN available, rsync is an excellent solution, though.

  3. I could not get this to run without changing '-rsh=ssh' to '--rsh=ssh'; that is, with two dashes instead of one. It was otherwise failing with "-rsh=ssh: unknown option"

    • Thanks. I don't recall exactly on what platform I tested it, but looks like with the latest rsync, it should be "--rsh". I fixed it.

Leave a Reply to Devon P Cancel reply

Your email address will not be published. Required fields are marked *