I am currently working with an SFTP server and aiming to synchronize its folder with a corresponding local directory. It's crucial to note that I lack control over the SFTP server, and files within it are consistently added and removed autonomously. Consequently, my objective is to retrieve these files without overwriting those already transferred.
Initially, I contemplated two approaches:
Develop a script manually, incorporating a caching mechanism to track already transferred files and initiate synchronization for new additions.
Discover a tool that could streamline this process, leading me to consider rsync.
However, I encountered a challenge as shell access is restricted, and my assumption that rsync could operate over SFTP was proven incorrect. Below is an example of the rsync command I attempted:
rsync -av --ignore-existing \
-e 'sshpass -p$($password) ssh -o StrictHostKeyChecking=no' \
$username@$username:$remoteBaseDirectory/ $localBaseDirectory"
Unfortunately, this command won't work because rsync is not installed on the SFTP server.
Now, my question is: How can I establish a reliable synchronization process, with a particular interest in the "--ignore-existing" option, without having to build it from scratch?
I usually do powershell in Ubuntu, but is not a problem if the enviromment needs to be changed.
Thank you very much, and have a great day.
Best regards,
