ftp automation

ScriptFTP

The professional tool to automate FTP, SFTP, FTPS and schedule FTP batch jobs

The forum is now read only. Please, go to the the main ScriptFTP website if you need help.
Post here if you experience file transfer problems or unexpected errors.
I've made a script to take backup of the most important files on my laptop. This works well when I'm connected to the same physical net as my FTP-server but when SYNCing thru internet, I get errors like these:
  • ***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/Websider/StoresundBF/j15/components/com_search/views/search.
    ***** The server said: Unknown command
  • ***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/Retrospect Catalog Files.
    ***** The server said: Unknown command
  • ***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/Min musikk/iTunes/Album Artwork/Cache/2153593DDCD6FC07/05/15/14.
    ***** The server said: Unknown command
  • ***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/Pinnacle Studio/Render/T1 40BB000B/DVD/StudioSequence.
    ***** The server said: Unknown command
  • ***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/Websider/StoresundBF/j1012/mambots/editors/jce/jscripts/tiny_mce/plugins/searchreplace/langs.
    ***** The server said: Unknown command
  • ***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/C702memstick/Picture.
    ***** The server said: Unknown command
The script runs 3 SYNC sessions and it is always the last one that fails. As you can see from the error messages, the SYNC fails on different files each time.

The first SYNC only affect one file but it is >400MB. Works OK.
Second SYNC is a few hundred files and also work OK.
The third SYNC is the largest syncing 13.000 files at a total about 20GB. This one fails every time I sync over internet.

Seems to me that some more error tolerance is needed and I believe you are already working on it.

All the above messages come from ver 3.2 (got a dev.version link from Lars H. 8-) ). Had more trouble with ver 3.1...
Thanks for the information SveinHa.

In your case there is no timeout, at least none is shown in your message. The strange thing is the response from your FTP server:
***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/Websider/StoresundBF/j15/components/com_search/views/search.
***** The server said: Unknown command
Could you put VERBOSE(ON) on the top of your script and send me the 10 (or so) lines before the "Unknown command" error? It would be very useful.

What your FTP server states here is that it didn't understand what ScriptFTP said. Are you using always the same FTP server or this one is different?
I'm always using the same FTP server (D-Link DNS-323).

I'll try the VERBOSE(ON) next week.
Thanks SveinHa. I'm curious about your log.
Done a couple of new tests with VERBOSE(ON). The end of the logfile looks like this:
Skipping (remote file is up to date) Båt\Bavaria 37 AC 2001\6_1154859025773.jpg
Remote file time: 11/27/2008 16:15:38
Adjusted remote file time: 11/27/2008 17:13:19
Local file time: 08/14/2006 11:54:16
Skipping (remote file is up to date) Båt\Bavaria 37 AC 2001\FINN båt - Norges ledende båtmarked. Søk etter båter, båtmotorer og båtutstyr..htm
Remote file time: 11/27/2008 16:15:39
Adjusted remote file time: 11/27/2008 17:13:20
Local file time: 06/20/2008 19:05:51
Skipping (remote file is up to date) Båt\Bavaria 37 AC 2001\Thumbs.db
CWD ..
250 OK. Current directory is /Volume_1/D810/Båt
PWD
257 "/Volume_1/D810/Båt" is your current location
CWD Bavaria 37 Exclusive 1996
250 OK. Current directory is /Volume_1/D810/Båt/Bavaria 37 Exclusive 1996
PWD
257 "/Volume_1/D810/Båt/Bavaria 37 Exclusive 1996" is your current location
PORT 172,24,8,222,18,244
200 PORT command successful
MLSD
500 ?
500 Unknown command
***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/Båt/Bavaria 37 Exclusive 1996.
***** The server said: Unknown command
CWD ..
250 OK. Current directory is /Volume_1/D810/Båt
PWD
257 "/Volume_1/D810/Båt" is your current location
CWD ..
250 OK. Current directory is /Volume_1/D810
PWD
257 "/Volume_1/D810" is your current location
CWD /
250 OK. Current directory is /
PWD
257 "/" is your current location
Synchronization error. Exiting
And the second test:
Remote file time: 11/27/2008 19:49:38
Adjusted remote file time: 11/27/2008 20:47:20
Local file time: 04/24/2007 23:00:00
Skipping (remote file is up to date) Websider\StoresundBF\j1012\administrator\components\com_jce\jscripts\scriptaculous.js
CWD ..
250 OK. Current directory is /Volume_1/D810/Websider/StoresundBF/j1012/administrator/components/com_jce
PWD
257 "/Volume_1/D810/Websider/StoresundBF/j1012/administrator/components/com_jce" is your current location
CWD language
250 OK. Current directory is /Volume_1/D810/Websider/StoresundBF/j1012/administrator/components/com_jce/language
PWD
257 "/Volume_1/D810/Websider/StoresundBF/j1012/administrator/components/com_jce/language" is your current location
PORT 172,24,8,222,7,179
200 PORT command successful
MLSD
150 Connecting to port 1971
226-ASCII
226-Options: -l
226 3 matches total
335 bytes transferred. (285 byte/s) (00:00:01)
PORT 172,24,8,222,7,180
Timeout (30s).
Timeout
Client closed the connection.
***** SYNC Error #4: Operation timed out.
Synchronization error. Exiting
Third:
Skipping (remote file is up to date) C702memstick\Music\Alarm-Rooster.wav
Remote file time: 11/27/2008 16:26:10
Adjusted remote file time: 11/27/2008 17:23:51
Local file time: 06/25/2008 16:22:50
Skipping (remote file is up to date) C702memstick\Music\bohemianrhapsod.mid
CWD podcast
250 OK. Current directory is /Volume_1/D810/C702memstick/Music/podcast
PWD
257 "/Volume_1/D810/C702memstick/Music/podcast" is your current location
PORT 172,24,8,222,8,142
200 PORT command successful
MLSD
500 ?
500 Unknown command
***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/C702memstick/Music/podcast.
***** The server said: Unknown command
CWD ..
250 OK. Current directory is /Volume_1/D810/C702memstick/Music
PWD
257 "/Volume_1/D810/C702memstick/Music" is your current location
CWD ..
250 OK. Current directory is /Volume_1/D810/C702memstick
PWD
257 "/Volume_1/D810/C702memstick" is your current location
CWD ..
250 OK. Current directory is /Volume_1/D810
PWD
257 "/Volume_1/D810" is your current location
CWD /
250 OK. Current directory is /
PWD
257 "/" is your current location
Synchronization error. Exiting
The SYNC command is (modified from one of your sample scripts):
IF(SYNC("C:\Documents and Settings\SveinHa\Mine dokumenter","/Volume_1/D810",UPLOAD, SUBDIRS)!="OK")
Thanks for the log Sveinha.

I'm working to make ScriptFTP more tolerable to timeouts errors. In the next version ScriptFTP will try to reestabilish the connection to the FTP server, if possible, without throwing an error.

But I found also a very strange error in your script output:
MLSD
500 ?
500 Unknown command

***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/Båt/Bavaria 37 Exclusive 1996.
***** The server said: Unknown command
Here, ScriptFTP request a remote file listing with the FTP protocol command MLSD and your FTP server replies:
500 ?
500 Unknown command
That's very very rare. MLSD is part of the FTP protocol standard and this is the first time I find a FTP server replying "unknown command" for such a common command... Try updating, if you are able to, the FTP server software or change it. I think that it's not a ScriptFTP fault.

I really don't understand why this error (the unknown command error, not the timeout) appears only when ScriptFTP is used over internet instead a local network.
I find the MLSD 45 times in one of my logfiles but it is only the last one that fails. My FTP-server was updated some months ago and when checking a few days ago, it still have the latest released SW.

I can email you the logfile if it may help.
Yes, please, this way I can see all the details, FTP server version etc. Remember to remove the user name and password as they appear in the complete log file.
Sendt right now :P
I really don't know why your FTP server answers "unknown command" to
ScriptFTP. I read your log and I don't see anything wrong with the
ScriptFTP behavior. Regarding to the timout, as you know, it is
something I'm working on. In the meantime I suggest you to retry the
synchronization if it fails using GOTO:
Code: Select all:syncstart IF(SYNC("C:\Documents and Settings\SveinHa\Minedokumenter","/Volume_1/D810",UPLOAD, SUBDIRS)!="OK")  PRINT("Synchronization error. Exiting")  GOTO :syncstart END IF
Have tried a slightly different approach but ended up with the same result
Code: Select allLOCALCHDIR("C:\Documents and Settings\SveinHa\Mine dokumenter") GETLIST($list,LOCAL_DIRECTORIES) FOREACH $item IN $list  IF(SYNC("C:\Documents and Settings\SveinHa\Mine dokumenter\".$item,"/Volume_1/D810/".$item,UPLOAD,SUBDIRS)!="OK")  PRINT("Synchronization error. Exiting")  SLEEP("60")  EXIT(2) ELSE  PRINT("All files downloaded. Waiting 1 second")  SLEEP(1) END IF END FOREACH
Usually ends with error #500 but also an occasional timeout.
Here are several of the logfiles (only the last part). The SYNC ends on very different places, sometimes after a few hundred files and other times after thousands (logfile size varies from 50kb to 2 Mb).
Local file time: 10/11/2008 17:42:51
Skipping (remote file is up to date) RR 2.4TDI Tromsø\Thumbs.db
CWD ..
250 OK. Current directory is /Volume_1/D810/Biler
PWD
257 "/Volume_1/D810/Biler" is your current location
CWD RR 78 rød
250 OK. Current directory is /Volume_1/D810/Biler/RR 78 rød
PWD
257 "/Volume_1/D810/Biler/RR 78 rød" is your current location
PORT 172,24,8,222,7,179
200 PORT command successful
MLSD
500 ?
500 Unknown command
***** SYNC Error #500: Cannot read remote file listing of /Volume_1/D810/Biler/RR 78 rød.
***** The server said: Unknown command
CWD ..
250 OK. Current directory is /Volume_1/D810/Biler
PWD
257 "/Volume_1/D810/Biler" is your current location
CWD /
250 OK. Current directory is /
PWD
257 "/" is your current location
Synchronization error. Exiting
SYNC("C:\Documents and Settings\SveinHa\Mine dokumenter\DreamUp","/Volume_1/D810/DreamUp",UPLOAD,SUBDIRS)
CWD /Volume_1/D810/DreamUp
250 OK. Current directory is /Volume_1/D810/DreamUp
PWD
257 "/Volume_1/D810/DreamUp" is your current location
Clock time difference is -3488 seconds.
Synchronizing remote directory /Volume_1/D810/DreamUp from C:\Documents and Settings\SveinHa\Mine dokumenter\DreamUp.
PORT 172,24,8,222,8,84
200 PORT command successful
MLSD
150 Connecting to port 2132
226-ASCII
226-Options: -l
226 1 matches total
118 bytes transferred. (7,68 kB/s) (15 ms)
PORT 172,24,8,222,8,85
Timeout (30s).
Timeout
Client closed the connection.
***** SYNC Error #4: Operation timed out.
Synchronization error. Exiting
Local file time: 03/06/2007 23:00:00
Skipping (remote file is up to date) StoresundBF\j1012\administrator\modules\mod_unread.php
CWD ..
250 OK. Current directory is /Volume_1/D810/Websider/StoresundBF/j1012/administrator
PWD
257 "/Volume_1/D810/Websider/StoresundBF/j1012/administrator" is your current location
CWD popups
250 OK. Current directory is /Volume_1/D810/Websider/StoresundBF/j1012/administrator/popups
PWD
257 "/Volume_1/D810/Websider/StoresundBF/j1012/administrator/popups" is your current location
PORT 172,24,8,222,13,107
200 PORT command successful
MLSD
150 Connecting to port 3435
226-ASCII
226-Options: -l
226 5 matches total
595 bytes transferred. (36,3 kB/s) (16 ms)
PORT 172,24,8,222,13,108
Timeout (30s).
Timeout
Client closed the connection.
***** SYNC Error #4: Operation timed out.
Synchronization error. Exiting
250 OK. Current directory is /Volume_1/D810/Websider/StoresundBF/j1012/mambots/editors/tinymce/jscripts/tiny_mce/plugins/directionality
PWD
257 "/Volume_1/D810/Websider/StoresundBF/j1012/mambots/editors/tinymce/jscripts/tiny_mce/plugins/directionality" is your current location
PORT 172,24,8,222,4,188
200 PORT command successful
MLSD
150 Connecting to port 1212
578 bytes transferred. (18,2 kB/s) (31 ms)
226-ASCII
226-Options: -l
226 5 matches total
PORT 172,24,8,222,4,189
Timeout (30s).
Timeout
Client closed the connection.
***** SYNC Error #4: Operation timed out.
Synchronization error. Exiting
250 OK. Current directory is /Volume_1/D810/Min musikk/iTunes/Album Artwork/Cache/2153593DDCD6FC07/15/12/14
PWD
257 "/Volume_1/D810/Min musikk/iTunes/Album Artwork/Cache/2153593DDCD6FC07/15/12/14" is your current location
PORT 172,24,8,222,6,40
200 PORT command successful
MLSD
150 Connecting to port 1576
226-ASCII
226-Options: -l
226 1 matches total
145 bytes transferred. (N/A/s) (0 ms)
PORT 172,24,8,222,6,41
Timeout (30s).
Timeout
Client closed the connection.
***** SYNC Error #4: Operation timed out.
Synchronization error. Exiting
Hi Sveinha,
Usually ends with error #500 but also an occasional timeout.
I think that the error 500 is a mistake from your FTP server. ScriptFTP is no doing nothing strange here, just requesting a file listing.

Regarding to the occasional timouts, the only solution at this moment is to relaunch the SYNC command if it returned an error.

I'm sorry I cannot help you further. If you have any other idea feel free to post here again.