delete_source not deleting files #28

kdvlr · 2019-05-13T19:58:26Z

Is delete_source designed to be used in conjunction with monitor or monitor_once ? When I use that on my NAS, it does not delete the files that have been merged.

ehendrix23 · 2019-05-13T20:10:42Z

It can be used without monitor or monitor_once.
delete_source would delete the front, left,right camera files and the folder when it was able to create the merged file for the folder successfully.

By default the intermediate files (these are the video files created that have the merged cameras on 1 minute intervals) are deleted from the moment the merged file for the folder was created. Using --keep-intermediate those files will be kept.

kdvlr · 2019-05-14T03:50:49Z

So, this is what I am using

set DIRNAME=%DATE:~-4%-%DATE:~-7,2%-%DATE:~-10,2%
mkdir E:\Tesla\TeslaCam\Merged\%DIRNAME%
E:\Tesla\tesla_dashcam\tesla_dashcam.exe --merge --compression ultrafast --encoding x265 --quality HIGH --delete_source --keep-intermediate --output E:\Tesla\TeslaCam\Merged\%DIRNAME%\ E:\Tesla\TeslaCam\SavedClips

Without keep-intermediate, I don't get a single merged file. This is understandable because the per-folder intermediate files are deleted (which is used to create the complete merged file)

However, at the end of the process, I have my original front, right and left files, the intermediate file and the merged file.

All I want is the merged file. Is there a way of achieving this ?

Here's the output from the script

E:\Tesla\tesla_dashcam>E:\Tesla\tesla_dashcam\tesla_dashcam.exe --merge --compression ultrafast --encoding x265 --quality HIGH --delete_source --keep-intermediate --output E:\Tesla\TeslaCam\Merged\2019-13-05\ E:\Tesla\TeslaCam\SavedClips
Discovered 2 folders with 2 clips to process.
        Processing 1 clips in folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-16 (1/2)
                Processing clip 1/1 from 04/24/19 21:16:27 and 59 seconds long.
                Creating movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-04-24_21-16.mp4, please be patient.
        Movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-04-24_21-16.mp4 for folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-16 is ready.
        Processing 1 clips in folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-19 (2/2)
                Processing clip 1/1 from 04/24/19 21:19:29 and 59 seconds long.
                Creating movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-04-24_21-19.mp4, please be patient.
        Movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-04-24_21-19.mp4 for folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-19 is ready.
        Creating movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-13-05.mp4, please be patient.
Movie E:\Tesla\TeslaCam\Merged\2019-13-05\2019-13-05.mp4 has been created, enjoy.
Total processing time: 0:08:42

ehendrix23 · 2019-05-15T22:49:18Z

I just checked the code, --delete_source indeed only removes the source file using --monitor or --monitor_once; will change this in next release.

Not specifying --keep_intermediate should result in the merged minute files that are created to be removed. They are then removed when the 10-minute clip (all 1-minute clips from the folder) is created.
The final merged file (with --merge) uses the 10-minute clip ones and not the 1-minute clips.

The merged files on the folder level (normally the 10 minute long clip then) currently would always be there as there is nothing in the code to remove that.

Ran similar like yours, running it without --keep_intermediate does result in the final merged file being created for me. The 10-minute clip files are still there but the 1-minute clip files are deleted. Hence not sure why you are not getting the final merged file then. Can you run it without --keep_intermediate and provide me the output?

kdvlr · 2019-05-16T17:03:58Z

Here's the output

E:\Tesla\tesla_dashcam>merge_files.bat

E:\Tesla\tesla_dashcam>set DIRNAME=2019-16-05

E:\Tesla\tesla_dashcam>mkdir E:\Tesla\TeslaCam\Merged\2019-16-05

E:\Tesla\tesla_dashcam>E:\Tesla\tesla_dashcam\tesla_dashcam.exe --merge --compression ultrafast --encoding x265 --quality HIGH --delete_source --output E:\Tesla\TeslaCam\Merged\2019-16-05\ E:\Tesla\TeslaCam\SavedClips
Discovered 2 folders with 2 clips to process.
        Processing 1 clips in folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-16 (1/2)
                Processing clip 1/1 from 04/24/19 21:16:27 and 59 seconds long.
                Creating movie E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-16.mp4, please be patient.
        Movie E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-16.mp4 for folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-16 is ready.
        Processing 1 clips in folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-19 (2/2)
                Processing clip 1/1 from 04/24/19 21:19:29 and 59 seconds long.
                Creating movie E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-19.mp4, please be patient.
        Movie E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-19.mp4 for folder E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-19 is ready.
        Creating movie E:\Tesla\TeslaCam\Merged\2019-16-05\2019-16-05.mp4, please be patient.
                File E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-16.mp4 does not exist anymore, skipping.
                File E:\Tesla\TeslaCam\Merged\2019-16-05\2019-04-24_21-19.mp4 does not exist anymore, skipping.
                Error: No valid clips to merge found.
All folders have been processed, resulting movie files are located in E:\Tesla\TeslaCam\Merged\2019-16-05\
Total processing time: 0:04:22

Here's the resulting directory structure.

E:\Tesla\tesla_dashcam>dir /s E:\Tesla\TeslaCam
 Volume in drive E is NAS
 Volume Serial Number is 8091-F15A

 Directory of E:\Tesla\TeslaCam

05/11/2019  10:55 PM    <DIR>          .
05/11/2019  10:55 PM    <DIR>          ..
05/16/2019  11:55 AM    <DIR>          Merged
05/15/2019  04:36 AM    <DIR>          SavedClips
               0 File(s)              0 bytes

 Directory of E:\Tesla\TeslaCam\Merged

05/16/2019  11:55 AM    <DIR>          .
05/16/2019  11:55 AM    <DIR>          ..
05/16/2019  12:00 PM    <DIR>          2019-16-05
               0 File(s)              0 bytes

 Directory of E:\Tesla\TeslaCam\Merged\2019-16-05

05/16/2019  12:00 PM    <DIR>          .
05/16/2019  12:00 PM    <DIR>          ..
               0 File(s)              0 bytes

 Directory of E:\Tesla\TeslaCam\SavedClips

05/15/2019  04:36 AM    <DIR>          .
05/15/2019  04:36 AM    <DIR>          ..
05/13/2019  10:11 PM    <DIR>          2019-04-24_21-16
05/13/2019  10:31 PM    <DIR>          2019-04-24_21-19
               0 File(s)              0 bytes

 Directory of E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-16

05/13/2019  10:11 PM    <DIR>          .
05/13/2019  10:11 PM    <DIR>          ..
04/24/2019  09:17 PM        29,972,998 2019-04-24_21-16-front.mp4
04/24/2019  09:17 PM        29,966,497 2019-04-24_21-16-left_repeater.mp4
04/24/2019  09:17 PM        29,952,832 2019-04-24_21-16-right_repeater.mp4
               3 File(s)     89,892,327 bytes

 Directory of E:\Tesla\TeslaCam\SavedClips\2019-04-24_21-19

05/13/2019  10:31 PM    <DIR>          .
05/13/2019  10:31 PM    <DIR>          ..
04/24/2019  09:20 PM        30,024,504 2019-04-24_21-19-front.mp4
04/24/2019  09:20 PM        30,027,339 2019-04-24_21-19-left_repeater.mp4
04/24/2019  09:20 PM        29,987,456 2019-04-24_21-19-right_repeater.mp4
               3 File(s)     90,039,299 bytes

     Total Files Listed:
               6 File(s)    179,931,626 bytes
              17 Dir(s)  4,675,108,347,904 bytes free

ehendrix23 · 2019-05-23T15:36:32Z

FYI, not ignoring this. Just haven't had time with end of school stuff for the kids etc.. :-)

I will be looking into this and ensuring it is fixed in the next release however.

kdvlr · 2019-05-25T17:25:52Z

No worries. It's a long weekend; Have fun!

I have just created a quick batch file to help me sort it all out. It's a quick hack, but just putting it here in case it helps someone.

@echo off
SETLOCAL ENABLEDELAYEDEXPANSION
set YEAR=%DATE:~-4%
set MONTH=%DATE:~-10,2%
set DAY=%DATE:~-7,2%
for /L %%i IN (%DAY%,-1,1) DO (
    set DIRNAME=%YEAR%-%MONTH%-%%i
    echo !DIRNAME!
    for /r "E:\Tesla\TeslaCam\SavedClips\" %%x in (*!DIRNAME!*) do (
        mkdir "E:\Tesla\TeslaCam\Merged\!DIRNAME!"
        move "%%x" "E:\Tesla\TeslaCam\Merged\!DIRNAME!\"
    )
)
for /f "delims=" %%d in ('dir /s /b /ad E:\Tesla\TeslaCam\SavedClips\ ^| sort /r') do rd "%%d"
for /f "usebackq tokens=*" %%D in (`DIR /S /B /A:D /O:N E:\Tesla\TeslaCam\Merged`) do (
   # E:\Tesla\tesla_dashcam\tesla_dashcam.exe --merge --compression ultrafast --encoding x265 --quality HIGH --delete_source --output E:\Tesla\TeslaCam\Merged\%%~nxD\%%~nxD.mp4 E:\Tesla\TeslaCam\Merged\%%~nxD
    )
)

I know you mentioned you'll take care of delete_source. I think that should be good enough for me. Even now, I can get rid of all files with _front _repeater and it will do it, but I'd rather have it be included in your script to guarantee that the files have been processed

exime · 2019-05-26T17:57:43Z

Semi-related to this issue - I am also using this script with a NAS, which my Raspberry Pi copies to automatically upon connecting to WiFi (https://github.com/marcone/teslausb).

Would it be possible to have a different flavor of --monitor that periodically (I'm thinking in the middle of the night to avoid starting to process files while my Raspberry Pi is still copying files to the share) checks for new folders, instead of looking for TeslaCam to be mounted?

ehendrix23 · 2019-05-27T16:10:03Z

Semi-related to this issue - I am also using this script with a NAS, which my Raspberry Pi copies to automatically upon connecting to WiFi (https://github.com/marcone/teslausb).

Would it be possible to have a different flavor of --monitor that periodically (I'm thinking in the middle of the night to avoid starting to process files while my Raspberry Pi is still copying files to the share) checks for new folders, instead of looking for TeslaCam to be mounted?

This is something I've already been thinking off to be able to do. My thoughts on this was for example when someone is using it with a cloud service, they could copy the files up there and then it would start doing the merging etc.. But in the end solution would work for your case as well then. :-)

Would you mind just opening a new issue for this with the request? Won't be in next release (going on vacation so want to have the next release out before then as it also includes some fixes).

exime · 2019-05-27T16:21:30Z

Done, thanks.

ehendrix23 · 2019-05-29T01:09:30Z

FYI,

Just released test version that includes option for --delete_source when not using --monitor and fix for reported issue:
https://github.com/ehendrix23/tesla_dashcam/releases/tag/v0.1.10b0

ehendrix23 · 2019-05-30T17:20:33Z

Release v.0.1.10 has been made available with the fix:
https://github.com/ehendrix23/tesla_dashcam/releases/tag/v0.1.10

ehendrix23 closed this as completed May 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

delete_source not deleting files #28

delete_source not deleting files #28

kdvlr commented May 13, 2019

ehendrix23 commented May 13, 2019

kdvlr commented May 14, 2019 •

edited

Loading

ehendrix23 commented May 15, 2019 •

edited

Loading

kdvlr commented May 16, 2019

ehendrix23 commented May 23, 2019

kdvlr commented May 25, 2019

exime commented May 26, 2019

ehendrix23 commented May 27, 2019

exime commented May 27, 2019

ehendrix23 commented May 29, 2019

ehendrix23 commented May 30, 2019

delete_source not deleting files #28

delete_source not deleting files #28

Comments

kdvlr commented May 13, 2019

ehendrix23 commented May 13, 2019

kdvlr commented May 14, 2019 • edited Loading

ehendrix23 commented May 15, 2019 • edited Loading

kdvlr commented May 16, 2019

ehendrix23 commented May 23, 2019

kdvlr commented May 25, 2019

exime commented May 26, 2019

ehendrix23 commented May 27, 2019

exime commented May 27, 2019

ehendrix23 commented May 29, 2019

ehendrix23 commented May 30, 2019

kdvlr commented May 14, 2019 •

edited

Loading

ehendrix23 commented May 15, 2019 •

edited

Loading