ftp automation

ScriptFTP

The professional tool to automate FTP, SFTP, FTPS and schedule FTP batch jobs

The forum is now read only. Please, go to the the main ScriptFTP website if you need help.
Missing any feature or command? Post your ideas here. Suggestions are welcome.
I know that I can set the transfer speed via SETSPEED. However I want to be able to detect how busy the network is, then adjust the transfer speed based on that.

So if the network is busy I want the transfer speed to be low (the other traffic gets priority), but when the network is idle the transfer speed should not be throttled.

Maybe a BUSY() function which returns true if the network is busy, or GETTRAFFIC() which would return some value.

* My ISP upload speed is 512Kbps. If I am connected to my camera, then the upload connection is almost saturated. Adding an FTP transfer really interferes with the video.
Hello Bucket,

It would be great to have this feature as most of FTP transfers are not time-critical (such as the transfer of your camera). Many users would welcome it.

But it is not possible because Windows does not have a mechanism to give the applications information about how much saturated the network is. I also think that this information is not even available inside the operating system itself. look for example at the network load graphic in the windows task manager. It estimates how busy is your network based on the maximum bandwidth of your network card (usually 10Mbps or higher depending on your computer) but you know that most users' internet connection is not that fast. There is a bottleneck in the DSL router or whatever the user has, and the operating system does not know the size of it, it simply sends information packets for delivery to the network card.

Some routers have something called QoS (quality of service) that gives priority to some kind of traffic over another. For example give the Skype calls a higher priority than web browsing.

If you have a clue about how to accomplish this please let me know. I am curious about if there is a way or not.
But it is not possible because Windows does not have a mechanism to give the applications information about how much saturated the network is.
Something like Background Intelligent Transfer Service?
Some routers have something called QoS (quality of service) that gives priority to some kind of traffic over another.
I always turn off QoS as it is not entirely stable (yes, yes, Microsoft SAYS it is...).
I am curious about if there is a way or not.
I was thinking more of timing the transfer of the first file (or using a small test file), then using that as an indicator. ScripfFTP could store the time and use it as a running indicator. Then if the test transfer is slow, it could return a "busy" flag, or it could throttle itself, probably based on some user set value.

Then ScriptFTP could time each file and if the transfer speeds up it could go faster, or if it slows down, then it could throttle again.
Something like Background Intelligent Transfer Service?
Thanks for the info Wandrey, I didn't know that BITS existed. I googled a bit and found that it only supports HTTP/HTTPS and SMB transfers. From http://msdn.microsoft.com/en-us/library ... S.85).aspx :
BITS supports the HTTP, HTTPS, and SMB protocols for RemoteName.
From Bucket:
I was thinking more of timing the transfer of the first file (or using a small test file), then using that as an indicator. ScripfFTP could store the time and use it as a running indicator. Then if the test transfer is slow, it could return a "busy" flag, or it could throttle itself, probably based on some user set value.
Yes, good idea, that could be the way to go. I will create a small script that uploads a test file and calculates the elapsed time. Based on that it will set the transfer speed (with SETSPEED). As soon as I reply the emails and do some coding I will go on it.
Well, here it is the quick and dirty way to set the transfer speed based on the current network load. This script uploads a 10Mb file and calculates the time it takes. Based on that value, if it is more than 5 seconds (this value can be adjusted) sets the maximum upload speed to only 8Kb/s.


FTP Script
  1. OPENHOST("127.0.0.1","carl","12345")
  2. # Get the current time
  3. $StartTimestamp=GETTIME(HOUR).GETTIME(MIN).GETTIME(SEC)
  4.  
  5. # Upload a file for testing how much time it takes
  6. $result=PUTFILE("C:\path_to\10MB_file.zip")
  7.  
  8. IF($result!="OK")
  9.     STOP
  10.  
  11.  
  12.  
  13. # Calculate the elapsed time in seconds
  14. $startSeconds =  (textcut($StartTimestamp,1,2) * (60 * 60)) + (textcut($StartTimestamp,3,2) * (60)) + textcut($StartTimestamp,5,2)
  15.  
  16. #endSeconds
  17. $endTimeStamp = GETTIME(HOUR).GETTIME(MIN).GETTIME(SEC)
  18. $endSeconds =  (textcut($endTimeStamp,1,2) * (60 * 60)) + (textcut($endTimeStamp,3,2) * (60)) + textcut($endTimeStamp,5,2)
  19.  
  20.  
  21. # Elapsed time is
  22. $elapsed_time_in_seconds= $endSeconds - $startSeconds
  23. Print("The upload hastaken ".$elapsed_time_in_seconds." seconds")
  24.  
  25.  
  26. # If the upload has taken more than 5 seconds means
  27. # that the connection is being actively used by another
  28. # program (The 5 seconds value vary depending on your network connection
  29. # adjust it accordingly)
  30. IF($elapsed_time_in_seconds>5)  
  31.    # This FTP transfer should have much less priority
  32.    # limit the ScriptFTP upload speed to 8KB/s
  33.    SETSPEED(8)
  34.    PRINT("The upload has taken more than 5 seconds. Limiting the upload speed to 8KB/s")
  35.  
  36.  
  37. # Delete the file uploaded for testing
  38. DELETEFILE("10MB_file.zip")
  39.  
  40.  
  41. #
  42. #
  43. #
  44. # Having adjusted the speed of ScriptFTP
  45. # in the case the connection is being heavily
  46. # used, put the rest of tour script here
  47. #
  48. #
  49. #
  50.  
  51.  
  52.  

The algorithm to calculate the elapsed time is the one posted by Bucket in another forum Post.
$StartTimestamp=GETTIME(HOUR).GETTIME(MIN).GETTIME(SEC)
Careful. I ran into problems building a timestamp... :-)

Regarding the SETSPEED() command, how do I reset it?
SETSPEED(-1)
or
SETSPEED(10000000000)
or ?
Last edited by Bucket on 01 Apr 2011, 00:37, edited 1 time in total.
Most programs which transfer files around a network show the realtime transfer speed as a file is written/read. Most of them also show the final transfer speed achieved.

So maybe a command to retrieve the speed of the last transfer?

PUTFILE("file")
$xferSpeed = GetLastSpeed()

So rather how long it took I get the actual speed. Since I know what the ISP gives me for upload bandwidth I can determine if some other piece of equipment is active (such as a realtime camera feed).

I can use this within the loop which uploads a long list of files. Since each file is a different size using just the elapsed time is tricky. I would need to get the filesize, do some arithmetic, test it, then possibly adjust the speed.

Also, using the OS to get the LAN activity would not work as the computer is on a 100Mbps LAN which is connected to the ISP via a router. I can transfer files on a 100Mbps LAN all day long and not impact other equipment. But the ISP upload is capped at 500Kbps, which is what I need to get information on.