How to archive files older than 7 days with creating one archive for all files with same date?

Question

I am looking for someone who can help me make a scheduled task to automatically move log files into RAR archives.

It does not have to be a batch file solution, if you have other ideas please share.

I got the basic code for it. This is the batch file code I have so far:

"C:\Program Files\WinRAR\rar.exe" a -ag -ms "D:\tet\Web3811\Web3811\LogsBackup\" @backup.txt

That line in the batch file runs RAR to create an archive with all files in the folder specified in list file backup.txt containing:

D:\tet\Web3811\Web3811\log

The RAR archive is created in D:\tet\Web3811\Web3811\LogsBackup\ with yyyy-mm-dd.rar as file name.

I need help with:

1. The RAR archives should have date in format dd-mm-yyyy in name instead of yyyy-mm-dd.
2. Only log files should be archived which are older than 7 days according to last modification date in comparison to current date whereby time does not matter, just date. All files with a date and time before 27-07-2014 00:00:00 should be added to the RAR archives if current date and time is 02-08-2014 12:30:00.
3. Each RAR archive to create should contain only files with same last modification date.
4. All archived log files should be deleted once the RAR compression is completed without errors.

The reason for being a batch file is the requirement of being executable as scheduled task.

An example for third requirement:

The folder contains 5 log files with following last modification dates:

Oldest.log    23-07-2014 02:20:54
AlsoOld.log   23-07-2014 23:52:26
Sample1.log   25-07-2014 09:08:46
Sample2.log   25-07-2014 12:59:02
Newest.log    26-07-2014 18:32:48

The scheduled task needs to create 3 archives with following names and files:

1. 23-07-2014_Logs.rar with Oldest.log and AlsoOld.log.
2. 25-07-2014_Logs.rar with Sample1.log and Sample2.log.
3. 26-07-2014_Logs.rar with just Newest.log.

No log file was created on 24-07-2014 and therefore also no RAR archive to create for this day.

Answer 1

I suggest to use in the batch file:

"C:\Program Files\WinRAR\rar.exe" mf -ac -ao -agDD-MM-YYYY-NN -ep1 -idq -m5 -to7d -y "D:\tet\Web3811\Web3811\LogsBackup\Logs_" @backup.txt

Above command moves all files older than 7 days according to last modification date into a RAR archive with name starting with Logs_ and current date in requested format and an additional incrementing number starting with number 1 after a hyphen in case of running this command line several times on one day.

Only files with archive attribute are moved into an archive. The archive attribute is cleared after archiving a file even if deletion is not possible for example when another application has opened the file with a write lock. RAR does not delete files on which reading and compressing the data failed at all (read lock).

See text file Rar.txt in program files folder of WinRAR for a description of all the switches used in this command line.

---

After some requirements have been explained better, here is a batch file to create the archive files as finally requested.

@echo off
setlocal EnableExtensions EnableDelayedExpansion
rem Define the directories to use for backup task.
set "LogDirectory=D:\tet\Web3811\Web3811\log"
set "BakDirectory=D:\tet\Web3811\Web3811\LogsBackup"

rem Get all file names in log directory into a list file sorted by last
rem modification date with oldest file at top and newest at bottom.
rem Note: /S is important to get the file names with complete path.
dir "%LogDirectory%\*" /A-D /B /OD /S /TW 1>"%BakDirectory%\LogFiles.lst" 2>nul
rem Jump to clean up stage if no file found in the log directory.
if errorlevel 1 goto :CleanUp

rem Delete list file for all files with same day if file exists
rem for example from a previous execution of this batch file
rem which was terminated manually by a user during execution.
if exist "%BakDirectory%\DayFiles.lst" del "%BakDirectory%\DayFiles.lst"

set LastDate=none
for /F "usebackq delims=" %%F in ( "%BakDirectory%\LogFiles.lst" ) do (

   set FileTime=%%~tF

   rem Get just file date from file time in format DD-MM-YYYY.
   rem The file time string format depends on date and time
   rem format definition in Windows language settings.
   rem Therefore the line below must be adapted if date format
   rem is whether DD.MM.YYYY nor DD-MM-YYYY nor DD/MM/YYYY.
   set FileDate=!FileTime:~0,2!-!FileTime:~3,2!-!FileTime:~6,4!

   rem Is the last modification date of this file different
   rem to last modification date of the previous file?
   if not "!FileDate!"=="!LastDate!" (
      rem Nothing to archive on first difference.
      if not "!LastDate!"=="none" call :ArchiveLogs
      rem Exit loop if RAR has not archived any file which means
      rem all other files are modified within the last 7 days.
      if "!LastDate!"=="ExitLoop" goto CleanUp
      rem Start creating a new list.
      set LastDate=!FileDate!
   )
   rem Append name of this file with path to current day list.
   echo %%F>>"%BakDirectory%\DayFiles.lst"
)

rem Jump to clean up stage if no list file with files to archive.
if not exist "%BakDirectory%\DayFiles.lst" goto CleanUp

rem Otherwise with no log file created or modified within
rem the last 7 days, but at least one older file exists
rem nevertheless, archive all those files in list file.
call :ArchiveLogs

:CleanUp
del "%BakDirectory%\LogFiles.lst"
endlocal
goto :EOF

:ArchiveLogs
rem Move all files in the list file older than 7 days without
rem path using best compression into a RAR archive with last
rem modification date of archived file(s) in RAR file name.
"C:\Program Files\WinRAR\Rar.exe" mf -ep1 -idq -m5 -to7d -y "%BakDirectory%\!LastDate!_Logs.rar" "@%BakDirectory%\DayFiles.lst"
rem Exit FOR loop above if no file archived because
rem no file in the list file is older than 7 days.
if errorlevel 10 set LastDate=ExitLoop
del "%BakDirectory%\DayFiles.lst"

I first thought, it is not possible to do this without coding a small console application to create the file lists per date and ignore files not modified within the last 7 days. But then I had an idea on how to solve this main problem using just a batch file and RAR as it can be seen above.

It is best to run this batch file with a scheduled task short after midnight as RAR takes also current time into account for "older than 7 days" and not just the date.

But it would be no problem if batch file is executed for example at 18:00 and there are log files created respectively modified at 23:00. In this case log files with last modification date before 18:00 and with a date exactly before 7 days in comparison to current date are moved first into a RAR archive, and on next day the other log files last modified after 18:00 from same date are moved also to the RAR archive for this date.

Example with batch task executed always at 18:00 and what happens.

There are the log files

FirstSundayAugust2014_1.log   03/08/2014 15:23
FirstSundayAugust2014_2.log   03/08/2014 23:48

and the scheduled task runs on Sunday, 10th August 2014 at 18:00.

The batch file moves FirstSundayAugust2014_1.log into RAR archive 03-08-2014_Logs.rar, but the other log file FirstSundayAugust2014_2.log also from last Sunday remains in the directory.

On Monday, 11th August 2014 at 18:00 the batch file moves also FirstSundayAugust2014_2.log into the RAR archive 03-08-2014_Logs.rar and this archive contains now both log files created respectively last modified on first Sunday in August 2014.

One more note:

RAR file names with date in format DD-MM-YYYY are not really good in my point of view. Better would be YYYY-MM-DD as this results in *.rar files where those RAR files listed alphabetically according to file name in Windows Explorer would result in same list as when those RAR files are listed according to file date/time in Windows Explorer.

To get RAR files with format YYYY-MM-DD for the date in file name the line

set FileDate=!FileTime:~0,2!-!FileTime:~3,2!-!FileTime:~6,4!

needs to be modified to

set FileDate=!FileTime:~6,4!-!FileTime:~3,2!-!FileTime:~0,2!