FTP: copy, check integrity and delete

Question

I am looking for a way to connect to a remote server with ftp or lftp and make sure the following steps:

1. Copy files from FTP serve

Answer 1

First, make sure your remote server supports the checksum calculation at all. Many do not. I believe there's even no standard FTP command to calculate a checksum of a remote file. There were many proposals and there are many proprietary solutions.

The latest proposal is:
https://tools.ietf.org/html/draft-bryan-ftpext-hash-02

So even if your server supports checksum calculation, you have to find a client that supports the same command.

Some of the commands that can be used to calculate checksum are: XSHA1, XSHA256, XSHA512, XMD5, MD5, XCRC and HASH.

You can test that with WinSCP. The WinSCP supports all the previously mentioned commands. Test its checksum calculation function or the checksum scripting command. If they work, enable logging and check, what command and what syntax WinSCP uses against your server.

---

Neither the ftp (neither Windows nor *nix version) nor the lftp support checksum calculation, let only automatic verification of downloaded file.

I'm not even aware of any other client that can automatically verify downloaded file.

You can definitely script it with a help of some feature-rich client.

---

I've wrote this answer before OP specified that he/she is on Linux. I'm keeping the Windows solution in case it helps someone else.

On Windows, you could script it with PowerShell using WinSCP .NET assembly.

param (
    $sessionUrl = "ftp://username:password@example.com/",
    [Parameter(Mandatory)]
    $localPath,
    [Parameter(Mandatory)]
    $remotePath,
    [Switch]
    $pause = $False
)

try
{
    # Load WinSCP .NET assembly
    Add-Type -Path (Join-Path $PSScriptRoot "WinSCPnet.dll")

    # Setup session options
    $sessionOptions = New-Object WinSCP.SessionOptions
    $sessionOptions.ParseUrl($sessionUrl);

    $session = New-Object WinSCP.Session

    try
    {
        # Connect
        $session.Open($sessionOptions)

        Write-Host "Downloading $remotePath to $localPath..."
        $session.GetFiles($remotePath, $localPath).Check();

        # Calculate remote file checksum
        $buf = $session.CalculateFileChecksum("sha-1", $remotePath)
        $remoteChecksum = [BitConverter]::ToString($buf)
        Write-Host "Remote file checksum:     $remoteChecksum"

        # Calculate local file checksum
        $sha1 = [System.Security.Cryptography.SHA1]::Create()
        $localStream = [System.IO.File]::OpenRead($localPath)
        $localChecksum = [BitConverter]::ToString($sha1.ComputeHash($localStream))
        Write-Host "Downloaded file checksum: $localChecksum"

        # Compare cheksums
        if ($localChecksum -eq $remoteChecksum)
        {
            Write-Host "Match, deleting remote file"

            $session.RemoveFiles($remotePath).Check();
            $result = 0
        }
        else
        {
            Write-Host "Does NOT match"
            $result = 1
        }
    }
    finally
    {
        # Disconnect, clean up
        $session.Dispose()
    }
}
catch [Exception]
{
    Write-Host "Error: $($_.Exception.Message)"
    $result = 1
}

# Pause if -pause switch was used
if ($pause)
{
    Write-Host "Press any key to exit..."
    [System.Console]::ReadKey() | Out-Null
}

exit $result

You can run it like:

powershell -file checksum.ps1 -remotePath ./file.dat -localPath C:\path\file.dat

This is partially based on WinSCP example for Verifying checksum of a remote file against a local file over SFTP/FTP protocol.

(I'm the author on WinSCP)

Answer 2

That's a long shot, but if the server supports php, you can exploit that.

Save the following as a php file (say, check.php), in the same folder as your name_of_file.txt file:

<? php
echo md5_file('name_of_file.txt');
php>

Then, visit the page check.php, and you should get the md5 hash of your file.

Related questions:

• Calculate file checksum in FTP server using Apache FtpClient
• How to perform checksums during a SFTP file transfer for data integrity?
• https://serverfault.com/q/98597/401691