I have a remote camera which will FTP videos to a server (motion detection). However the up connection is slow. So if there is a lot of video files to upload, some of them get lost as the camera does not buffer. I can see this in the camera's log files as failed FTP uploads.
So I would like to set up a laptop which will have an FTP server installed (probably FileZilla's server). The camera will then FTP the files over a 100M connection to the laptop. I then want the laptop to FTP the files to my Internet server and delete them locally.
I know I can set up a script to wake up every so often and upload files, then delete them locally, but what if the camera is still busy sending a file which the script wants to upload? I would then get a partial file which could potentially be deleted.
Is there a way to tell if a file is in the process of being written to? So I can iterate over a list of files and only upload ones which are complete?
So I would like to set up a laptop which will have an FTP server installed (probably FileZilla's server). The camera will then FTP the files over a 100M connection to the laptop. I then want the laptop to FTP the files to my Internet server and delete them locally.
I know I can set up a script to wake up every so often and upload files, then delete them locally, but what if the camera is still busy sending a file which the script wants to upload? I would then get a partial file which could potentially be deleted.
Is there a way to tell if a file is in the process of being written to? So I can iterate over a list of files and only upload ones which are complete?
You can start with a very commonly used script called upload_and_delete.ftp available in the script library section of this website. I will paste it the most interesting part here (with some modifications):
The problem could be, as you already point, that if the video file being uploaded by ScriptFTP is at the same time being sent from the camera... a corrupt or incomplete video file will be uploaded. Fortunately, I think that FileZilla server only actually stores the uploaded files once they are complately received, so ScriptFTP will not find incomplete files in the directory where the camera uploads its videos.
FTP Script
- # Change current lcoal directory to where the videos are stored
- # Get the list of video files in that folder
- # For each video file in $list...
- # Upload the file
- # If the video file has been succesfully uploaded
- # delete the local copy of the file
The problem could be, as you already point, that if the video file being uploaded by ScriptFTP is at the same time being sent from the camera... a corrupt or incomplete video file will be uploaded. Fortunately, I think that FileZilla server only actually stores the uploaded files once they are complately received, so ScriptFTP will not find incomplete files in the directory where the camera uploads its videos.
liked Script FTP? Try our other product Email Parser
It is entirely possible to upload a partial file.
But I have come up with a script which should work. It relies on the OS file system "move" command. Since the file is open for a write operation is has an active file handle. So you cannot move it.
The script tries to move any files to a transfer directory, then uploads those files in the transfer directory. Any partial files will be handled during the next pass.
I have the script if you are willing to review it, plus you can put it into your script library if you wish.
But I have come up with a script which should work. It relies on the OS file system "move" command. Since the file is open for a write operation is has an active file handle. So you cannot move it.
The script tries to move any files to a transfer directory, then uploads those files in the transfer directory. Any partial files will be handled during the next pass.
I have the script if you are willing to review it, plus you can put it into your script library if you wish.
liked Script FTP? Try our other product Email Parser
wow, nice script Bucket. I will post it here, people usually visit the script library section only for very basic things and this one is a bit specific:
FTP Script
- # This script will upload video and picture files
- # Global Variables
- $UploadServer = "xxxx"
- $LocalFTPDir = "D:\FTP"
- $LocalUploadDir = "D:\Upload"
- # in seconds
- $CycleDelay = 20
- $LogDate = ""
- # processing loop
- :aprocessingLoop
- # process GH
- $return = 1
- $localDir = "GH"
- $userName = "xxxx"
- $password = "xxxx"
- goto :aprocessFiles
- :Return1
- # process Cabin
- $return = 2
- $localDir = "Cabin"
- $userName = "xxxx"
- $password = "xxxx"
- goto :aprocessFiles
- :Return2
- :Cycle
- goto :aprocessingLoop
- # ########################################################
- # ########################################################
- # the basic operation is:
- # 1. Move any files from the camera inbound FTP directory to the
- # associated upload directory
- # 2. Upload any files in the upload directory to the remote server
- # 3. Delete uploaded files in the upload directory
- # 4. Rinse, wash, repeat
- :aprocessFiles
- $sourceDir = $LocalFTPDir."\".$localDir
- $targetDir = $LocalUploadDir."\".$localDir
- # ------------------
- #first move any file from the FTP dir to the upload dir
- goto :ReturnBlock
- goto :ReturnBlock
- $moveCount = 0
- $totalCount = 0
- $fullFileName = $sourceDir."\".$fileName
- $totalCount = $totalCount + 1
- # if this fails, then the file is still being written to
- $moveCount = $moveCount + 1
- # ------------------
- # Upload the moved files to the remote server
- goto :ReturnBlock
- goto :ReturnBlock
- goto :ReturnBlock
- goto :ReturnBlock
- $uploadCount = 0
- $deleteCount = 0
- $totalCount = 0
- $fullFileName = $targetDir."\".$fileName
- $totalCount = $totalCount + 1
- $uploadCount = $uploadCount + 1
- $deleteCount = $deleteCount + 1
- # ------------------
- # Now go back
- :ReturnBlock
- # just in case it is open when we get here (ie: via an error)
- goto :Return1
- goto :Return2
- goto :Cycle
- # ########################################################
- # ########################################################
liked Script FTP? Try our other product Email Parser
:ProcessingLoop as <img src="./images/smilies/icon_razz.gif" alt=":P" title="Razz" />rocessingLoop has pleased. In section [ScriptFTP] it is necessary smilies and other options should not work.
Hi Wandrey,
I've just fixed it replacing the labels in the script.
The script syntax highlighter that runs behind the forum still looks for smilies in the scripts...
I've just fixed it replacing the labels in the script.
The script syntax highlighter that runs behind the forum still looks for smilies in the scripts...
liked Script FTP? Try our other product Email Parser
Just as a point of interest, the camera takes less than a second to xfer a file to the computer over the 100MBps connection. The script then moves the file from the FTP directory to the Upload directory, and this also takes less than a second.
And I already had a conflict between the two operations, that split second timing when the stars aligned. The script worked as designed and the offending file was uploaded during the next cycle.
Secondly there was a connection failure (timeout) connecting to the Internet FTP server. Normally this would have resulted in a lost file, but the script re-connected at the next cycle and uploaded the file.
And I already had a conflict between the two operations, that split second timing when the stars aligned. The script worked as designed and the offending file was uploaded during the next cycle.
Secondly there was a connection failure (timeout) connecting to the Internet FTP server. Normally this would have resulted in a lost file, but the script re-connected at the next cycle and uploaded the file.
Thanks for the update and your collaboration Bucket
liked Script FTP? Try our other product Email Parser
- All times are UTC -
