ftp automation

ScriptFTP

The professional tool to automate FTP, SFTP, FTPS and schedule FTP batch jobs

The forum is now read only. Please, go to the the main ScriptFTP website if you need help.
Need help writing a script? Have any question about ScriptFTP?
Hi

I am copying backup files from one server to another. Works fine what I am doing however I need to refine it. Otherwise I have to keep deleting the files on the backup server.

I want to limit the number and start overwriting the files if there are more than say 10 files on the backup server.

So far I have this:
FTP Script
  1. # Web site 1:
  2. SYNC("C:\websites\web1\backup","/backups/web1",UPLOAD,"*.zip")
Using SYNC i am able to leave the old backups on the server and add the news ones each night by schedule. But I just want to keep the most recent 10 files and then start overwriting.

Can this be done?

thanks
Gus
perhaps this could be achieved by deleting the oldest file on the target server before syncing?
This would then simply upload the latest backup.

Unfortunately this is complicated by there being two files for each backup. e.g.

web1-120111.files.bak.zip
web2.120111.db-bak.zip

But the file name does have a date. How could I then delete the two oldest files on the ftp directory?
perhaps this could be achieved by deleting the oldest file on the target server before syncing?
How could I then delete the two oldest files on the ftp directory?
FTP Script
  1. SYNC("C:\websites\web1\backup","/backups/web1",UPLOAD_DELETE,"*.zip")
I don't think that works because it only deletes from the target location when files have been removed from the source location. It would delete all the backup files from the backup server and then copy across the new file.

Let me try to explain again.

1> web application creates automatic backup files three times a week. There are two .zip files, one for DB and one for files. These files are overwritten by the web application on the next backup. Space is v tight on the server so I cannot keep more than one backup set on the web server.
2> Script FTP runs every night after the backups have completed and copies them over to a backup server.
3> Backup server gets filled up OR I have to keep deleting old files.

So, I want my script to be clever enough to look at the backup server before copying over the new files and
delete only the two oldest files before copying across the new set.

Perhpas I'm going to have to write something to do this..
Can it is necessary was ask simply at once how to delete the oldest files?

Removal of all files on the oldest date:
FTP Script
  1. GETLIST($aList, REMOTE_FILES, "*.zip")
  2. $aMinTime = GETTIME(FORMAT0)
  3.  
  4. FOREACH $aItem IN $aList
  5.   $aTime = GETFILETIME(REMOTE, $aItem)
  6.   IF($aTime < $aMinTime)
  7.     $aMinTime = $aTime
  8.   END IF
  9.  
  10. $aMinDate = TEXTCUT($aMinTime, 1, 10)
  11.  
  12. FOREACH $aItem IN $aList
  13.   $aTime = GETFILETIME(REMOTE, $aItem)
  14.   $aDate = TEXTCUT($aTime, 1, 10)
  15.   IF($aDate == $aMinDate)
  16.     DELETEFILE($aItem)
  17.   END IF
Removal of two oldest files on date and time:

FTP Script
  1. GETLIST($aList, REMOTE_FILES, "*.zip")
  2. $aMinTime = GETTIME(FORMAT0)
  3. $aLastMinTime = $aMinTime
  4.  
  5. FOREACH $aItem IN $aList
  6.   $aTime = GETFILETIME(REMOTE, $aItem)
  7.   IF($aTime < $aMinTime)
  8.     $aLastMinTime = $aMinTime
  9.     $aMinTime = $aTime
  10.   END IF
  11.  
  12. FOREACH $aItem IN $aList
  13.   $aTime = GETFILETIME(REMOTE, $aItem)
  14.   IF(($aTime == $aMinTime) OR ($aTime == $aLastMinTime))
  15.     DELETEFILE($aItem)
  16.   END IF
Working capacity did not check, wrote exclusively on section the

http://www.scriptftp.com/reference.php

I am sorry for my English, I use the machine translator.
many thanks!

I will give that a try
Hi again

I'm afraid I can't get this to work at all.

I am using the second script to try to remove the last two (oldest files) returned by GETLIST.
But I'm having no luck. It doesn't work as it should and I can't work out what you are trying to go with the date variables.

If GETLIST returned the files in order based on datetime then I could simply remove the last two but it appears to return a random list.

How do I loop the list and find the oldest two files? I just can't seem to work it out with this script language.

Gus
oh and I tried the first one.

It just deletes ALL the files regardless of their date.

:cry:
All the same in what order getlist returns files - we select the minimum dates from the first cycle, in the second - we delete files.
Give we will try to add output of operations.
FTP Script
  1. OPENHOST("10.5.6.253")
  2. CHDIR("/Tools")
  3. GETLIST($aList, REMOTE_FILES, "*.zip")
  4.  
  5. $aMinTime = GETTIME(FORMAT0)
  6. $aLastMinTime = $aMinTime
  7.  
  8. PRINT("Search 2 oldest data and time")
  9. FOREACH $aItem IN $aList
  10.   $aTime = GETFILETIME(REMOTE, $aItem)
  11.   PRINT($aItem." - ".$aTime)
  12.   IF($aTime < $aMinTime)
  13.     $aLastMinTime = $aMinTime
  14.     $aMinTime = $aTime
  15.   END IF
  16.  
  17. PRINT("Found:")
  18. PRINT("aLastMinTime - ".$aLastMinTime)
  19. PRINT("aMinTime - ".$aMinTime)
  20.  
  21. PRINT("Delete 2 oldest file")
  22.  
  23. FOREACH $aItem IN $aList
  24.   $aTime = GETFILETIME(REMOTE, $aItem)
  25.   IF(($aTime == $aMinTime) OR ($aTime == $aLastMinTime))
  26.     PRINT("DELETEFILE(".$aItem.")")
  27.   END IF
This is an unregistered copy of ScriptFTP. You have 30 days of trial left.
(Press Control-C to abort)
Running 2.ftp

OPENHOST("10.5.6.253")
Connecting to 10.5.6.253 as anonymous
Connected.

CHDIR("/Tools")
Changing current remote directory to /Tools

GETLIST(REMOTE_FILES,"*.zip")
Getting file listing of current remote directory
Found 9 files.

GETTIME(FORMAT0)
Retrieving current time.
Search 2 oldest data and time

GETFILETIME(REMOTE,"asciipic.zip")
asciipic.zip - 2000_07_29-05_10_28

GETFILETIME(REMOTE,"cdslow22.zip")
cdslow22.zip - 2003_06_14-13_41_14

GETFILETIME(REMOTE,"fe122se.zip")
fe122se.zip - 2002_01_07-22_00_00

GETFILETIME(REMOTE,"genmat.zip")
genmat.zip - 2000_08_22-05_18_20

GETFILETIME(REMOTE,"nlcalc3.zip")
nlcalc3.zip - 2000_04_10-12_55_44

GETFILETIME(REMOTE,"openpass.zip")
openpass.zip - 2006_04_26-14_07_57

GETFILETIME(REMOTE,"OpenpassXP.zip")
OpenpassXP.zip - 2006_12_14-08_55_10

GETFILETIME(REMOTE,"Poledit.zip")
Poledit.zip - 2001_12_13-14_35_32

GETFILETIME(REMOTE,"unpass.zip")
unpass.zip - 2002_03_20-13_11_24
Found:
aLastMinTime - 2000_07_29-05_10_28
aMinTime - 2000_04_10-12_55_44
Delete 2 oldest file

GETFILETIME(REMOTE,"asciipic.zip")
DELETEFILE(asciipic.zip)

GETFILETIME(REMOTE,"cdslow22.zip")

GETFILETIME(REMOTE,"fe122se.zip")

GETFILETIME(REMOTE,"genmat.zip")

GETFILETIME(REMOTE,"nlcalc3.zip")
DELETEFILE(nlcalc3.zip)

GETFILETIME(REMOTE,"openpass.zip")

GETFILETIME(REMOTE,"OpenpassXP.zip")

GETFILETIME(REMOTE,"Poledit.zip")

GETFILETIME(REMOTE,"unpass.zip")
Work fine...

Excuse me for my English, I use a machine translator
Last edited by WAndrey on 05 Jul 2011, 18:37, edited 1 time in total.
And fist script
FTP Script
  1. OPENHOST("10.5.6.253")
  2. CHDIR("/Tools")
  3. GETLIST($aList, REMOTE_FILES, "*.zip")
  4.  
  5. $aMinTime = GETTIME(FORMAT0)
  6.  
  7. PRINT("Search oldest data")
  8. FOREACH $aItem IN $aList
  9.   $aTime = GETFILETIME(REMOTE, $aItem)
  10.   PRINT($aItem." - ".$aTime)
  11.   IF($aTime < $aMinTime)
  12.     $aMinTime = $aTime
  13.   END IF
  14.  
  15. $aMinDate = TEXTCUT($aMinTime, 1, 10)
  16. PRINT("Found: ".$aMinDate)
  17.  
  18. PRINT("Delete old files by date")
  19. FOREACH $aItem IN $aList
  20.   $aTime = GETFILETIME(REMOTE, $aItem)
  21.   $aDate = TEXTCUT($aTime, 1, 10)
  22.   IF($aDate == $aMinDate)
  23.     PRINT("DELETEFILE(".$aItem.")")
  24.   END IF
This is an unregistered copy of ScriptFTP. You have 30 days of trial left.
(Press Control-C to abort)
Running 1.ftp

OPENHOST("10.5.6.253")
Connecting to 10.5.6.253 as anonymous
Connected.

CHDIR("/Tools")
Changing current remote directory to /Tools

GETLIST(REMOTE_FILES,"*.zip")
Getting file listing of current remote directory
Found 9 files.

GETTIME(FORMAT0)
Retrieving current time.
Search oldest data

GETFILETIME(REMOTE,"asciipic.zip")
asciipic.zip - 2000_07_29-05_10_28

GETFILETIME(REMOTE,"cdslow22.zip")
cdslow22.zip - 2003_06_14-13_41_14

GETFILETIME(REMOTE,"fe122se.zip")
fe122se.zip - 2002_01_07-22_00_00

GETFILETIME(REMOTE,"genmat.zip")
genmat.zip - 2000_08_22-05_18_20

GETFILETIME(REMOTE,"nlcalc3.zip")
nlcalc3.zip - 2000_04_10-12_55_44

GETFILETIME(REMOTE,"openpass.zip")
openpass.zip - 2006_04_26-14_07_57

GETFILETIME(REMOTE,"OpenpassXP.zip")
OpenpassXP.zip - 2006_12_14-08_55_10

GETFILETIME(REMOTE,"Poledit.zip")
Poledit.zip - 2001_12_13-14_35_32

GETFILETIME(REMOTE,"unpass.zip")
unpass.zip - 2002_03_20-13_11_24
Found: 2000_04_10
Delete old files by date

GETFILETIME(REMOTE,"asciipic.zip")

GETFILETIME(REMOTE,"cdslow22.zip")

GETFILETIME(REMOTE,"fe122se.zip")

GETFILETIME(REMOTE,"genmat.zip")

GETFILETIME(REMOTE,"nlcalc3.zip")
DELETEFILE(nlcalc3.zip)

GETFILETIME(REMOTE,"openpass.zip")

GETFILETIME(REMOTE,"OpenpassXP.zip")

GETFILETIME(REMOTE,"Poledit.zip")

GETFILETIME(REMOTE,"unpass.zip")

Work fine again... Except that there were no two files with one date :)
Write was more concretely that is impossible to you.

Excuse me for my English, I use a machine translator
thanks for taking the time and trouble to test the scripts.

I will try again

:)
Hi
I tried again with the first script but it didn't work. It tries to delete the oldest file and another one which is not the second oldest:

"Deleteing File(Euroace-Database.2011.07.09.194957_Native.bak.zip)"
"Deleteing File(Euroace-Database.2011.07.06.193823_Native.bak.zip)"

As you can see it is deleting the one file on the 6th and another on the 9th. Both the files should be dated the 6th July.

See attached image for the file list.

Here is is log output:

CHDIR("/httpdocs/backups/euroace")
Changing current remote directory to /httpdocs/backups/euroace

GETLIST(REMOTE_FILES,"*.zip")
Getting file listing of current remote directory
Found 6 files.

GETTIME(FORMAT0)
Retrieving current time.
Found:
Server time: aMinTime - 2011_07_10-12_18_26
Search 2 oldest data and time

GETFILETIME(REMOTE,"Euroace-Database.2011.07.09.194957_Native.bak.zip")
Euroace-Database.2011.07.09.194957_Native.bak.zip - 2011_07_09-22_31_57

GETFILETIME(REMOTE,"Euroace-Database.2011.07.06.193823_Native.bak.zip")
Euroace-Database.2011.07.06.193823_Native.bak.zip - 2011_07_06-21_00_31

GETFILETIME(REMOTE,"Euroace-Database.2011.07.08.193739_Native.bak.zip")
Euroace-Database.2011.07.08.193739_Native.bak.zip - 2011_07_08-22_02_07

GETFILETIME(REMOTE,"Euroace-files.2011.07.08.193739_Files.zip")
Euroace-files.2011.07.08.193739_Files.zip - 2011_07_08-22_08_03

GETFILETIME(REMOTE,"Euroace-files.2011.07.06.193823_Files.zip")
Euroace-files.2011.07.06.193823_Files.zip - 2011_07_06-21_06_19

GETFILETIME(REMOTE,"Euroace-files.2011.07.09.194957_Files.zip")
Euroace-files.2011.07.09.194957_Files.zip - 2011_07_09-22_37_26
Found:
aLastMinTime - 2011_07_09-22_31_57
aMinTime - 2011_07_06-21_00_31
Delete 2 oldest file

GETFILETIME(REMOTE,"Euroace-Database.2011.07.09.194957_Native.bak.zip")
Deleteing File(Euroace-Database.2011.07.09.194957_Native.bak.zip)

GETFILETIME(REMOTE,"Euroace-Database.2011.07.06.193823_Native.bak.zip")
Deleteing File(Euroace-Database.2011.07.06.193823_Native.bak.zip)

GETFILETIME(REMOTE,"Euroace-Database.2011.07.08.193739_Native.bak.zip")

GETFILETIME(REMOTE,"Euroace-files.2011.07.08.193739_Files.zip")

GETFILETIME(REMOTE,"Euroace-files.2011.07.06.193823_Files.zip")

GETFILETIME(REMOTE,"Euroace-files.2011.07.09.194957_Files.zip")

SYNC("C:\websites\euroace\backup","/httpdocs/backups/euroace",UPLOAD,"*.zip")
Uploading temporary file to calculate server-client clock time difference.
Clock time difference is -7196 seconds.
Synchronizing remote directory /httpdocs/backups/euroace from C:\websites\euroace\backup. Ignoring subdirectories.
Skipping (remote file is up to date) Euroace-Database.2011.07.09.194957_Native.bak.zip
Skipping (remote file is up to date) Euroace-files.2011.07.09.194957_Files.zip

CLOSEHOST
Disconnected.
Attachments
Capture.JPG
Capture.JPG (41.26KiB)Viewed 3424 times
I think perhaps the script assumes the files are on a different day? They are usually but not always on the same day. Backups may run into the next day?

Does the script find the oldest one based on day and then the next one using time but misses the one on the same day with a different time?

The script simply needs to delete the two oldest files from the list.

thanks
Angus
No, it is an error at me - aLastMinTime it was selected incorrectly - in that implementation really depends on in what order GETLIST produces files. Try such variant:
[ScriptFTP]GETLIST($aList, REMOTE_FILES, "*.zip")

$aMinTime = GETTIME(FORMAT0)
$aLastMinTime = $aMinTime

FOREACH $aItem IN $aList
$aTime = GETFILETIME(REMOTE, $aItem)
IF($aTime < $aMinTime)
IF($aMinTime < $aLastMinTime)
$aLastMinTime = $aMinTime
END IF
$aMinTime = $aTime
END IF
END FOREACH

FOREACH $aItem IN $aList
$aTime = GETFILETIME(REMOTE, $aItem)
IF(($aTime == $aMinTime) OR ($aTime == $aLastMinTime))
DELETEFILE($aItem)
END IF
END FOREACH[/ScriptFTP]Excuse for my English, I use the machine translator.
Sorry, that don't work either.

Here is the ouput:

CHDIR("/httpdocs/backups/euroace")
Changing current remote directory to /httpdocs/backups/euroace

GETLIST(REMOTE_FILES,"*.zip")
Getting file listing of current remote directory
Found 8 files.

GETTIME(FORMAT0)
Retrieving current time.

GETFILETIME(REMOTE,"Euroace-Database.2011.07.09.194957_Native.bak.zip")

GETFILETIME(REMOTE,"Euroace-Database.2011.07.10.195747_Native.bak.zip")

GETFILETIME(REMOTE,"Euroace-Database.2011.07.06.193823_Native.bak.zip")

GETFILETIME(REMOTE,"Euroace-files.2011.07.10.195747_Files.zip")

GETFILETIME(REMOTE,"Euroace-Database.2011.07.08.193739_Native.bak.zip")

GETFILETIME(REMOTE,"Euroace-files.2011.07.08.193739_Files.zip")

GETFILETIME(REMOTE,"Euroace-files.2011.07.06.193823_Files.zip")

GETFILETIME(REMOTE,"Euroace-files.2011.07.09.194957_Files.zip")

GETFILETIME(REMOTE,"Euroace-Database.2011.07.09.194957_Native.bak.zip")
Deleteing File(Euroace-Database.2011.07.09.194957_Native.bak.zip)

GETFILETIME(REMOTE,"Euroace-Database.2011.07.10.195747_Native.bak.zip")

GETFILETIME(REMOTE,"Euroace-Database.2011.07.06.193823_Native.bak.zip")
Deleteing File(Euroace-Database.2011.07.06.193823_Native.bak.zip)

GETFILETIME(REMOTE,"Euroace-files.2011.07.10.195747_Files.zip")

GETFILETIME(REMOTE,"Euroace-Database.2011.07.08.193739_Native.bak.zip")

GETFILETIME(REMOTE,"Euroace-files.2011.07.08.193739_Files.zip")

GETFILETIME(REMOTE,"Euroace-files.2011.07.06.193823_Files.zip")

GETFILETIME(REMOTE,"Euroace-files.2011.07.09.194957_Files.zip")

SYNC("C:\websites\euroace\backup","/httpdocs/backups/euroace",UPLOAD,"*.zip")
Uploading temporary file to calculate server-client clock time difference.
Clock time difference is -7197 seconds.
Synchronizing remote directory /httpdocs/backups/euroace from C:\websites\euroace\backup. Ignoring subdirectories.
Skipping (remote file is up to date) Euroace-Database.2011.07.10.195747_Native.bak.zip
Skipping (remote file is up to date) Euroace-files.2011.07.10.195747_Files.zip

CLOSEHOST
Disconnected.



Again it's supposed to delete these two files from the 6th July.

Euroace-Database.2011.07.06.193823_Native.bak.zip
Euroace-files.2011.07.06.193823_Files.zip

But skips the second file and deletes this instead:

Deleteing File(Euroace-Database.2011.07.09.194957_Native.bak.zip)

Files list now looks like the attached:
Attachments
Capture.JPG
Capture.JPG (53.73KiB)Viewed 3421 times
It is not enough information for the analysis. GETFILETIME "get the last modification time of a local or remote file". And dates and time for your picture are not signed. Show result of work of it:[ScriptFTP]SILENT(ON)

OPENHOST("xx.xx.xx.xx")
CHDIR("/httpdocs/backups/euroace")
GETLIST($aList, REMOTE_FILES, "*.zip")

$aMinTime = GETTIME(FORMAT0)
$aLastMinTime = $aMinTime

PRINT("Search 2 oldest data and time")
FOREACH $aItem IN $aList
$aTime = GETFILETIME(REMOTE, $aItem)
IF($aTime < $aMinTime)
IF($aMinTime < $aLastMinTime)
$aLastMinTime = $aMinTime
END IF
$aMinTime = $aTime
END IF
PRINT($aItem." - ".$aTime." ".$aMinTime." ".$aLastMinTime)
END FOREACH

PRINT("Found:")
PRINT("aLastMinTime - ".$aLastMinTime)
PRINT("aMinTime - ".$aMinTime)

PRINT("Delete 2 oldest file")

FOREACH $aItem IN $aList
$aTime = GETFILETIME(REMOTE, $aItem)
IF(($aTime == $aMinTime) OR ($aTime == $aLastMinTime))
PRINT("DELETEFILE(".$aItem.")")
END IF
END FOREACH

CLOSEHOST()[/ScriptFTP]
Excuse me for my English, I use a machine translator
Here is the output from the new version:
The files to delete should both have date 6th July 2011.

Search 2 oldest data and time
Euroace-Database.2011.07.09.194957_Native.bak.zip - 2011_07_09-22_31_57 2011_07_09-22_31_57 2011_07_12-09_49_42
Euroace-Database.2011.07.10.195747_Native.bak.zip - 2011_07_10-21_00_38 2011_07_09-22_31_57 2011_07_12-09_49_42
Euroace-Database.2011.07.06.193823_Native.bak.zip - 2011_07_06-21_00_31 2011_07_06-21_00_31 2011_07_09-22_31_57
Euroace-files.2011.07.10.195747_Files.zip - 2011_07_10-21_06_15 2011_07_06-21_00_31 2011_07_09-22_31_57
Euroace-Database.2011.07.08.193739_Native.bak.zip - 2011_07_08-22_02_07 2011_07_06-21_00_31 2011_07_09-22_31_57
Euroace-Database.2011.07.11.195119_Native.bak.zip - 2011_07_11-22_32_46 2011_07_06-21_00_31 2011_07_09-22_31_57
Euroace-files.2011.07.08.193739_Files.zip - 2011_07_08-22_08_03 2011_07_06-21_00_31 2011_07_09-22_31_57
Euroace-files.2011.07.11.195119_Files.zip - 2011_07_11-22_39_09 2011_07_06-21_00_31 2011_07_09-22_31_57
Euroace-files.2011.07.06.193823_Files.zip - 2011_07_06-21_06_19 2011_07_06-21_00_31 2011_07_09-22_31_57
Euroace-files.2011.07.09.194957_Files.zip - 2011_07_09-22_37_26 2011_07_06-21_00_31 2011_07_09-22_31_57
Found:
aLastMinTime - 2011_07_09-22_31_57
aMinTime - 2011_07_06-21_00_31
Delete 2 oldest file
DELETEFILE(Euroace-Database.2011.07.09.194957_Native.bak.zip)
DELETEFILE(Euroace-Database.2011.07.06.193823_Native.bak.zip)
***** SYNC Error #18001: Not connected.
***** CLOSEHOST Error #18001: Not connected.
Attachments
Capture.JPG
Capture.JPG (85.23KiB)Viewed 3421 times
Aha, again my error... Absolutely brains have become numb already :( Well this variant precisely correct :)
[ScriptFTP]$aHost = "xx.xx.xx.xx"
$aLocDir = "C:\websites\euroace\backup"
$aRemDir = "/httpdocs/backups/euroace"

OPENHOST($aHost)
CHDIR($aRemDir)
GETLIST($aList, REMOTE_FILES, "*.zip")

$aMinTime = GETTIME(FORMAT0)
$aLastMinTime = $aMinTime

FOREACH $aItem IN $aList
$aTime = GETFILETIME(REMOTE, $aItem)
IF($aTime < $aMinTime)
$aLastMinTime = $aMinTime
$aMinTime = $aTime
ELSE
IF($aTime < $aLastMinTime)
$aLastMinTime = $aTime
END IF
END IF
END FOREACH

FOREACH $aItem IN $aList
$aTime = GETFILETIME(REMOTE, $aItem)
IF(($aTime == $aMinTime) OR ($aTime == $aLastMinTime))
DELETEFILE($aItem)
END IF
END FOREACH

SYNC($aLocDir,$aRemDir,UPLOAD,"*.zip")

CLOSEHOST()[/ScriptFTP]
Excuse for my English, I use the machine translator
:D
works!! thanks so much! I will do some thorough testing to be sure but it seems just fine.

thanks again
:D
Well I know a situation when it will delete not two files :) If will be more than two files with identical date and time. But at your setting of the task such should not to happen.
Excuse for my English, I use a machine translator.
yes, that will never happen because the backups are done one after the other. So they will never have identical timestamps.

Seems to be working fine.

It would be nice if the scripting language had more powerful array processing commands like Ruby.

$alist.order
$alist.reverse
$x=$alist.find("yyy")

anyway.. for now the job is done

thanks