[<<]Message[>>] [<<]Author[>>] [<<]Subject[>>] [<<]Thread[>>]
Number : 3229 Date : 2002-12-11 Author : freedom41ca Subject : Re: Preserving Generation Backups Size(KB) : 8
--- In xxcopy@yahoogroups.com, Kan Yabumoto wrote: > > OK, let me answer your question about the "Generation backup". > WOW! Thank you very much for taking the time to respond with this invaluable information. I see and like the logic behind what you're saying, and I'm almost there! However, what happens if the last access date of the source file (or any of the three timestamps, for that matter) is also not modified for extended periods of time? The /db# switch would still inadvertently delete a file in the destination folder that I would otherwise have liked to remain (for whatever independent and arbitrary period I've chosen for the generation backups). ---(That is unless the first run of xxcopy for that day's backup modifies the last access date of the file being backed-up, thereby now providing an updated access date from which the second clean-up command can now base its /db#/fa on, but as you said, I believe this is not the case with xxcopy)--- ---(I guess you could include a command in your batch files that automatically modifies the last access date just before your run your clean-up command, but there must be a simpler way)--- Also, what happens when the source files happen to be in different source folders? You would then need to create multiple lines in a batch file testing for each individual file - which becomes very messy and time consuming. I suppose that you can create a single command using wildcards, but that too would get very complicated. Up to now, my batch file simply created a daily backup of the files required and put them into a folder that has a timestamp for a name (a "generation" backup, in other words), and then goes on to remove any generation back-ups older than x days so as to not end with 365 generation back-ups at the end of the year (maintaining 5 - 10 generations at any one time is plenty sufficient). What I was hoping for is a scheme that triggers the delete function based solely on the timestamp of the backup_dir "folder", and not on any particular file "in the folder". If the timestamp "of the folder" is within the /db# parameter, xxcopy moves on and leave the folder in its entirety. On the other hand, if the folder is outside the parameter identified by the /db# switch, xxcopy then removes the folder using the /rmdir switch (thereby also wiping out all contents within that folder). An all-or-none function, so to speak. ---(I believe that this is the way I have the command setup in my batch file, using the /rmdir and /db# switches in combination, but the /db# also goes on to delete any files in the folder that is older than the /db# switch)--- In programming terms, which would probably put us on the right direction, can the folder itself be treated as an "object" by xxcopy? Most Appreciated, Martin If a man says something in the woods and there are no women there to hear it, is he still wrong? >The LastWrite timestamp must be treated as what it is. As > you pointed out, your scheme of using /DB#30 will delete > files that were made over a month regardless of how important > that file might be on a daily basis. Therefore, it is > clear that the LastWrite timestamp is not useful for that > purpose. > > That is, let us realize that there are two types of files; > > 1) files that you create and update from time to time --- > most user-generated .DOC files are in this category. > > 2) files that are meant to be semi-permanent --- nearly all > vendor-supplied files and some user-generated files (such > as document template files) are in this category. > > When you write an xxcopy script such as; > > xxcopy \backup_dir\ /rsy/pd0/s /db#30 > > This script will not distinguish the two types and will > delete any file that was created 30 days ago or earlier. > And, this is how the LastWrite timestamp is and should be. > > So, the question is how can we make the distinction of > the two types? If you examine the \backup_dir\ directory > in isolation, unless you carry out certain extra steps, > none of the three timestamps (Creation, LastWrite, LastAccess) > suffices it. Therefore, we need to come up with a scheme > which will somehow achieve your original goal of deleting > the Type 1 files when they age inside the \backup_dir\ > (Or even from the source directory, for that matter), > and keeping the Type 2 files. > > Let us then, see what distinguishes the Type 2 files > from the Type 1 files in the source directory? > The answer is the LastAccess timestamp in the source > files. What else? As you may know (see XXTB #15), > the LastAccess timestamps is not all that reliable > if you casually use various file management tools that > defeat the intents of LastAccess timestamps (unfortunately, > many tools behave that way). But, let suppose that you > refrain yourself from using those "defective" tools > as much as possible, and stay with XXCOPY for most > file management activities. At least that way, the > LastAccess timestamps are mostly usable. > > What we now need is to somehow reflect the LastAccess > value in the source directory to handle the removal > of the unwanted files in the destination. > > xxcopy \src_dir\ \backup_dir\ /rdy/pd0/s/h/r/DB#30/FA > > This technique utilizes the relatively rare switch, > /RD (the Y suffix suppresses the confirmation on each > file) which deletes file that would be overwritten > from the destination if it were a copy command. > The file selection mechanism (such as /S, /H, and > /DB#30) works on the source side. And that is exactly > what we need here. /DB#30/FA select files in the > source whose LastAccess is 30 days ago or earlier. > > Please remember that this command works exactly like > the one without /RDY/PD0 for selecting files > in the source. Yet, the actual files that will be > deleted are in the destination. Note that this > method does not remove the extra files that are no > longer present in the source. But, that should be > done as part of the backup copy steps anyway. > > So, the overall two-step script should be like: > > > mybackup.bat > ---------------------------------------------------------- > xxcopy \src_dir\ \backup_dir\ /clone > xxcopy \src=dir\ \backup_dir\ /rdy/pd0/s/h/r/db#30/fa > ---------------------------------------------------------- > > Note: since this scheme involves the LastAccess (/FA) > timestamps which is a controversial one, you should > play with the common tools that you use and make sure > none of them inadvertently sets the LastAccess timestamps > to today's date. If you use XXCOPY, even opening the > file and accessing the file contents will not change > the LastAccess timestamps. But, many programs do. > So, you need to determine what files to be avoided. > Usually, a program that only looks at the directory > without actually reading the file contents will not > alter the LastAccess value. The good news is that > unless you use a switch like /DA#30, an inadvertent > update of the LastAccess value by other programs > will not affect the safety of this operation in any > adverse way. That is, when thing does not work as > intended, the worst thing that will happen to you > is extra files in the backup directory. But, that > will not be the end of the world. > > > Kan Yabumoto > ===================================================== > > At 2002-12-10 06:48, you wrote: > >I do a "generation backup" of certain files every day using the date macro > >switch. This creates a folder with the date that the backup was run, and > >then backs-up the selected files to that folder. > > > >The same batch file also runs a "clean-up" (so that the generation folders > >don't eventually take over the disk), and deletes older generations using > >the /DB# switch. Up to now, any folder older than the date specified by > >the date switch would be deleted, thereby always maintaining a > >predetermined number of generations on the disk (10 generations, in this > >case). However, I just noticed that the /db# switch also removes any > >files within the created folder that are older than the switch date. Duh, > >common sense, but what caught me off guard was that some of my files > >remain static over long periods of time (longer than the "older than" date > >value I've chosen for the switch). Therefore, when the daily backup ran > >this morning, I noticed that some of my files were deleted from each of > >the 10 generations backed-up on the disk. > > > >What I would like to be able to do is just delete the folder itself if the > >"folder date" is older than the switch date, and not touch any of the > >files within the folder if one or more files should fall outside the range > >specified by the date switch. Is there therefore any way around this so > >that I can preserve each "generation backup" in its entirety until it > >comes time to delete it? > > > >Martin > >Last night I played a blank tape at full blast. The mime next door went nuts.
This message if part of XXCOPY's message Archive. The archive contains all the messages posted at Yahoo!Groups: XXCOPY.