How to delete files with a Python script from a FTP server which are older than 7 days?

Question

I would like to write a Python script which allows me to delete files from a FTP Server after they have reached a certain age. I prepared the scipt below but it throws the e

Answer 1

What OS are you running on? The file path /test123/*.* is Unix-style yet the message says WindowsError. Are you taking the output of an ftp LIST command, which is in Unix-style, and trying to use it verbatim in a Windows script?

Answer 2

OK. Assuming your FTP server supports the MLSD command, make a module with the following code (this is code from a script I use to sync a remote FTP site with a local directory):

module code

# for python ≥ 2.6
import sys, os, time, ftplib
import collections
FTPDir= collections.namedtuple("FTPDir", "name size mtime tree")
FTPFile= collections.namedtuple("FTPFile", "name size mtime")

class FTPDirectory(object):
    def __init__(self, path='.'):
        self.dirs= []
        self.files= []
        self.path= path

    def getdata(self, ftpobj):
        ftpobj.retrlines('MLSD', self.addline)

    def addline(self, line):
        data, _, name= line.partition('; ')
        fields= data.split(';')
        for field in fields:
            field_name, _, field_value= field.partition('=')
            if field_name == 'type':
                target= self.dirs if field_value == 'dir' else self.files
            elif field_name in ('sizd', 'size'):
                size= int(field_value)
            elif field_name == 'modify':
                mtime= time.mktime(time.strptime(field_value, "%Y%m%d%H%M%S"))
        if target is self.files:
            target.append(FTPFile(name, size, mtime))
        else:
            target.append(FTPDir(name, size, mtime, self.__class__(os.path.join(self.path, name))))

    def walk(self):
        for ftpfile in self.files:
            yield self.path, ftpfile
        for ftpdir in self.dirs:
            for path, ftpfile in ftpdir.tree.walk():
                yield path, ftpfile

class FTPTree(FTPDirectory):
    def getdata(self, ftpobj):
        super(FTPTree, self).getdata(ftpobj)
        for dirname in self.dirs:
            ftpobj.cwd(dirname.name)
            dirname.tree.getdata(ftpobj)
            ftpobj.cwd('..')

single directory case

If you want to work on the files of a directory, you can:

import ftplib, time

quite_old= time.time() - 7*86400 # seven days

site= ftplib.FTP(hostname, username, password)
site.cwd(the_directory_to_work_on) # if it's '.', you can skip this line
folder= FTPDirectory()
folder.getdata(site) # get the filenames
for path, ftpfile in folder.walk():
    if ftpfile.mtime < quite_old:
        site.delete(ftpfile.name)

This should do what you want.

a directory and its descendants

Now, if this should work recursively, you'll have to do the following two changes in the code for “single directory case”:

folder= FTPTree()

and

site.delete(os.path.join(path, ftpfile.name))

Possible caveat

The servers I've worked with didn't have any issues with relative paths in the STOR and DELE commands, so site.delete with a relative path worked too. If your FTP server requires pathless filenames, you should first .cwd to the path provided, .delete the plain ftpfile.name and then .cwd back to the base folder.

Answer 3

OK, well rather than analyze the code you have posted any further, here's an example instead that might put you on the right track.

from ftplib import FTP
import re

pattern = r'.* ([A-Z|a-z].. .. .....) (.*)'

def callback(line):
    found = re.match(pattern, line)
    if (found is not None):
        print found.groups()

ftp = FTP('myserver.wherever.com')
ftp.login('elvis','presley')
ftp.cwd('testing123')
ftp.retrlines('LIST',callback)

ftp.close()
del ftp

Run it and you'll get output something like this, which should be a start towards what you're trying to achieve. To finish it out you'd need to parse the first result into a datetime, compare it with "now" and use ftp.delete() to get rid of the remote file if it's too old.

>>> 
('May 16 13:47', 'Thumbs.db')
('Feb 16 17:47', 'docs')
('Feb 23  2007', 'marvin')
('May 08  2009', 'notes')
('Aug 04  2009', 'other')
('Feb 11 18:24', 'ppp.xml')
('Jan 20  2010', 'reports')
('Oct 10  2005', 'transition')
>>> 

Answer 4

I had to do this and it took a while, thought I could save someones time here. We are using python with ftputil module installed:

#! /usr/bin/python
import time
import ftputil
host = ftputil.FTPHost('ftphost.com', 'username', 'password')
mypath = 'ftp_dir'
now = time.time()
host.chdir(mypath)
names = host.listdir(host.curdir)
for name in names:
    if host.path.getmtime(name) < (now - (7 * 86400)):
      if host.path.isfile(name):
         host.remove(name)


print 'Closing FTP connection'
host.close()

Answer 5

Well, it looks like the error you are seeing has to do with the fact that you are trying to remove the 'test123' directory from your local machine, not the FTP site. The FTP docs have a method called delete, and that's what you'd want to use to remove the file. As far as testing whether or not something is 7 days old or not, you might actually have to pull those files down from the FTP temporarily then check the modify times before using FTP.delete.