Hi,
I have version 3.1 and I want to sync via FTPS_IMPLICIT.
This works great. I have the command:
It wants to sync a directory with over 2800 files (130Mb).
I did a verbose ON and this is a part of the result:
CWD 200804
250 CWD successful. "/vmu/Rapportages/PMS-rapportage/monthly/200804" is current directory.
PWD
257 "/vmu/Rapportages/PMS-rapportage/monthly/200804" is current directory.
PORT 10,5,102,54,4,151
200 Port command successful
MLSD
150 Opening data channel for directory list.
252362 bytes transferred. (1.39 MB/s) (172 ms)
226 Transfer OK
Server closed connection
***** SYNC Error #3: Not connected and not logged in to remote server.
CLOSEHOST
***** CLOSEHOST Error #12001: Not connected.
In the working situation, starting the sync of this directory takes some time, but it worked.
Now it takes a lot more time and the FTP server closes the connection (timeout). It was set to 120 seconds.
I changed the timeout setting to 600 seconds on the ftp server, but still it is not working correctly for this folder.
I enlarged the memory with 1Gb to 2Gb and still the timeout...
Why is it not working correctly?
Regards
Erik
I have version 3.1 and I want to sync via FTPS_IMPLICIT.
This works great. I have the command:
Code: Select all
Now I'm moving the script to an other server (VMWARE) and halfway syncing, the script stops...SYNC($local_dir,$remote_dir,UPLOAD_DELETE,SUBDIRS)It wants to sync a directory with over 2800 files (130Mb).
I did a verbose ON and this is a part of the result:
CWD 200804
250 CWD successful. "/vmu/Rapportages/PMS-rapportage/monthly/200804" is current directory.
PWD
257 "/vmu/Rapportages/PMS-rapportage/monthly/200804" is current directory.
PORT 10,5,102,54,4,151
200 Port command successful
MLSD
150 Opening data channel for directory list.
252362 bytes transferred. (1.39 MB/s) (172 ms)
226 Transfer OK
Server closed connection
***** SYNC Error #3: Not connected and not logged in to remote server.
CLOSEHOST
***** CLOSEHOST Error #12001: Not connected.
In the working situation, starting the sync of this directory takes some time, but it worked.
Now it takes a lot more time and the FTP server closes the connection (timeout). It was set to 120 seconds.
I changed the timeout setting to 600 seconds on the ftp server, but still it is not working correctly for this folder.
I enlarged the memory with 1Gb to 2Gb and still the timeout...
Why is it not working correctly?
Regards
Erik
I reduced the file amount in the folder back to multiple subfolders with 250 files and than it is working again. This is not what I want though... I want one folder with all the files synchronized.
I just want it to work like on the other server.
Regards
Erik
I just want it to work like on the other server.
Regards
Erik
Hello Erik
252362 bytes transferred. (1.39 MB/s) (172 ms)It says that it takes only 172 ms (far less than a second) to retrieve the file listing. Is it true? The ScriptFTP timeout is 30 seconds and in the case the timeout expires you should see a message. Could you post a log of the FTP server itself? it would be very useful as it seems that it was the server who closed the connection (although you increased the server timeout...)
liked Script FTP? Try our other product Email Parser
Yesterday I looked at the admin panel of the ftp server (filezilla) during my sysnc session.
There is nothing happening after the dir list is send back to the client.
In the client (scriptftp) I see the line telling me that the dirlist is received (the one you mentioned also). This is al done with no delay. After the 'Transfer OK' message the silence begins.
It looks to me scriptftp is processing the dirlist and this takes some time. It could be that the time needed for scriptftp is bigger than the server timeout setting (but 10 minutes??)
On the other server is takes some time also, but it runs after lets say a minute or two.
Regards,
Erik
N.B. I'm not at the office today to check anything. (Back tomorrow)
There is nothing happening after the dir list is send back to the client.
In the client (scriptftp) I see the line telling me that the dirlist is received (the one you mentioned also). This is al done with no delay. After the 'Transfer OK' message the silence begins.
It looks to me scriptftp is processing the dirlist and this takes some time. It could be that the time needed for scriptftp is bigger than the server timeout setting (but 10 minutes??)
On the other server is takes some time also, but it runs after lets say a minute or two.
Regards,
Erik
N.B. I'm not at the office today to check anything. (Back tomorrow)
It looks to me scriptftp is processing the dirlist and this takes some time. It could be that the time needed for scriptftp is bigger than the server timeout setting (but 10 minutes??)Am I right?
Regards
Erik
Hello Erik,
Yes, you are right, it seems that the time is consumed by ScriptFTP to process the file list. But I tried to test it here synchronizing 2500 empty txt files (with UPLOAD_DELETE and SUBDIRS) and worked fine... May I try using test account on your FTP server? How is the file listing? how many levels of subdirs? Do you use very long names or special characters in the file names?
Yes, you are right, it seems that the time is consumed by ScriptFTP to process the file list. But I tried to test it here synchronizing 2500 empty txt files (with UPLOAD_DELETE and SUBDIRS) and worked fine... May I try using test account on your FTP server? How is the file listing? how many levels of subdirs? Do you use very long names or special characters in the file names?
liked Script FTP? Try our other product Email Parser
Hi, The FTP server is not available from the internet, it's an internal one. I can not give you access to look at it your self. The filesnames in the directory are long, I have to check this again and get back on you, but they are over 30 characters I think. The file type is PDF, but this should not bother the sync I think.
I'll post later this week the directory structure.
Regards
Erik
I'll post later this week the directory structure.
Regards
Erik
This is my script
"G:\26-------- --------- -----\15-------- ----\3--\11-------- \24-------- --------- ---\7------\6-----"
The filename length is like this:
46-------- --------- --------- --------- -----.pdf
The numbers in the names above are the length of the directories/filename.
The size of the files are around 60 - 100 KB.
The amount of files could be more than 3000.
If I create more subdirs into the latest one and place 250 files in each, the sync is running again. The subdirs have a length of 6:
"G:\26-------- --------- -----\15-------- ----\3--\11-------- \24-------- --------- ---\7------\6-----\6-----"
This is all I can give you.
Regards
Erik
Code: Select all
The place where the problems occur is a few directories deeper:SETPROTOCOL(FTPS_IMPLICIT)
SETPORT(990)
SETPASSIVE(DISABLED)
$server = GETPARAM(3)
$user = GETPARAM(4)
$password = GETPARAM(5)
$local_dir = "G:\26-------- --------- -----\15-------- ----"
$remote_dir = "/"
# Connect to FTP server
OPENHOST($server,$user,$password)
$result = SYNC($local_dir,$remote_dir,UPLOAD_DELETE,SUBDIRS)
# Transfer finished, close the connection
CLOSEHOST
 "G:\26-------- --------- -----\15-------- ----\3--\11-------- \24-------- --------- ---\7------\6-----"
The filename length is like this:
46-------- --------- --------- --------- -----.pdf
The numbers in the names above are the length of the directories/filename.
The size of the files are around 60 - 100 KB.
The amount of files could be more than 3000.
If I create more subdirs into the latest one and place 250 files in each, the sync is running again. The subdirs have a length of 6:
"G:\26-------- --------- -----\15-------- ----\3--\11-------- \24-------- --------- ---\7------\6-----\6-----"
This is all I can give you.
Regards
Erik
Hello Erik,
Thanks for the detailed information. I was finally able to get the same error here and I've fixed it partially.
I've modified the routines of ScriptFTP that handle file listings to make them work faster. Please, download this new build of ScriptFTP:
http://www.ScriptFTP.com/ScriptFTP_devel_setup.exe
Now you won't get that delay between retrieving the file listing and the file synchronization itself. But you might find that, if most of the files are up to date, the FTP server may disconnect ScriptFTP because it does not make any transfer (upload or download) for a long time. It takes to ScriptFTP too many minutes to realize that the ~6000 files stored in that directory are up to date (normal!). This FTP server behavior can be changed via a configuration file or file.
Please, let me know what you get.
Thanks for the detailed information. I was finally able to get the same error here and I've fixed it partially.
I've modified the routines of ScriptFTP that handle file listings to make them work faster. Please, download this new build of ScriptFTP:
http://www.ScriptFTP.com/ScriptFTP_devel_setup.exe
Now you won't get that delay between retrieving the file listing and the file synchronization itself. But you might find that, if most of the files are up to date, the FTP server may disconnect ScriptFTP because it does not make any transfer (upload or download) for a long time. It takes to ScriptFTP too many minutes to realize that the ~6000 files stored in that directory are up to date (normal!). This FTP server behavior can be changed via a configuration file or file.
Please, let me know what you get.
liked Script FTP? Try our other product Email Parser
Hi,
I will test this asap. One other solution is to send a noop command after x seconds to keep the connection alive.
Regards
Erik
NOOP
No Operation
This command does not affect anything at all. It performs no action other than having the server send an OK reply. This command is used to keep connections with servers "alive" (connected) while nothing is being done.
I will test this asap. One other solution is to send a noop command after x seconds to keep the connection alive.
Regards
Erik
NOOP
No Operation
This command does not affect anything at all. It performs no action other than having the server send an OK reply. This command is used to keep connections with servers "alive" (connected) while nothing is being done.
Hi Erik,
One other solution is to send a noop command after x seconds to keep the connection alive.Yes, but FileZilla (the FTP server you use) drops the connection if no file transfer is done for 3 minutes or so. Even if you send NOPs, request file listing etc. At least this is the behavior of the test machine I have with FileZilla... Very strange but this is what I found.
liked Script FTP? Try our other product Email Parser
Is it possible to have two different versions on one machine by installing them in seperate folders?
Hello Erik,
Yes, to do it install one version and rename the folder C:\Program Files\ScriptFTP. Then install the other version. The file associations and start menu shortcuts will point to the second installation.
Did you find that an older version doesn't get timeouts?
Yes, to do it install one version and rename the folder C:\Program Files\ScriptFTP. Then install the other version. The file associations and start menu shortcuts will point to the second installation.
Did you find that an older version doesn't get timeouts?
liked Script FTP? Try our other product Email Parser
OK, I'll try that.
Before I had verison 2.1 and this worked fine (I thought). Now I have version 3.1 and I'm running it from a different computer.
I did not have the time yet to test the other setup. I hope I can do this tomorrow.
Regards
Erik
Before I had verison 2.1 and this worked fine (I thought). Now I have version 3.1 and I'm running it from a different computer.
I did not have the time yet to test the other setup. I hope I can do this tomorrow.
Regards
Erik
- All times are UTC -
