[<<]Message[>>]    [<<]Author[>>]    [<<]Subject[>>]    [<<]Thread[>>]

Number   :  3485
Date     :  2003-01-08
Author   :  Kan Yabumoto
Subject  :  Re: EXTREME alarm
Size(KB) :  13


Dear Yuki:

At 2003-01-08 02:17, you wrote:
>Hi Kan,
>
>KY> Yuku:
>
>Yuki ^_-

I felt funny typing "Yuku", but I simply repeated your signature line 
:-) I thought I was talking to a Japanese guy and then,
I started to wonder if this guy was from eastern Europe 

At any rate, I'm very much relieved that the loss was of the
backup, not the real data --- a big sigh.

I cannot figure it out, either Kan. But I can tell you that after
>initially losing the contents of t:, I continued to experiment, and
>every time I ran the command, a command I had been running countless
>times in the past few months, it wiped out anything on t:. Every
>time.

I'm glad that we can chase this problem methodically.

>Now, the key point that I was explaining to Dan Anderson, that you
>might not have read yet:

I read it and dismissed it at that time.

>1) Yesterday, I did what I thought was a safe upgrade. I went from
>2.82.7 to the 2.83.4. I did not expect the behavior to be any
>different with this simple batch file between the two versions of
>xxcopy, but I was wrong. Finally deciding I *must* be xxcopy, and it
>must be the upgrade, I pulled all of xxcopy, and reinstalled the
>2.82.7 version. Immediately, I had no problem, and the command ran
>as expected, with safe and sound output. So it was version 2.83.4,
>and that is where you need to look.

In this particular instance, the jump is rather significant.
The versions 2.82.x was a "dead-end thread" in the XXCOPY
evolution. In order to make our next major update, we had
to redo a lot of things differently. So, substantial changes
were made in the code and we spent almost six months on this
version. So, nothing's surprise me except with a lot of
disappointments (and embarrassments).

>I will take a moment here to digress, and mention something I have
>always felt was amiss with the xxcopy install.bat routine. You
>recommend dumping all the files in the Windows system folder, and
>that comes up default on the install, with your recommendation. I
>have *never* felt comfortable about this, and I don't allow it. I
>have a discreet directory for xxcopy, and that is where it lives.
>This makes uninstalls, or rollbacks, very easy. I know I got
>*everything*, whereas I cannot be sure with all the stuff in the
>system directory. I suggest you change this default recommendation,
>for this reason. I am sure you do it because the directory needs to
>be in the PATH, but I am also sure that 99.99 percent of the people
>who use xxcopy know how to add its home to their PATH statements. ^_-

About the XXCOPY INSTALL -----------------------------------------

As to the way XXCOPY installs, we believe what XXCOPY is doing is
the best compromise. As you say, for experienced users, where
to place XXCOPY is not a major problem. It is the novices that
causes a lot of headache. While I was using Win9X, I never
felt comfortable when files are written to the C:\windows\command\
directory by an application. I wanted to keep that directory
as pure as what it was when the Win9X was installed. But, then,
where should we put XXCOPY files? An alternative to what the
current scheme works is to create a directory like:

c:\Program Files\Pixelab\xxcopy\xxcopy.exe

Most people hate the extra level --- I do, so, alternatively,

c:\Program Files\xxcopy\xxcopy.exe

But, unlike most other GUI programs which need not be in the
PATH=, being a pure command line program, XXCOPY.EXE need to
be in the PATH=. That is already a major headache right there.
And, we don't want XXCOPY to be a set of many files. That means,
XXCOPY need to be a self-contained program including the
installer. So, we don't want to consume a lot of bytes for this.

XXCOPY may be installed anywhere you like. For example:

xxcopy /install:c:\bin\

The rest will be taken care by XXCOPY. One nice thing about
the /INSTALL command as opposed to you put it by yourself
(which is acceptable) is that XXCOPY will look for existing
XXCOPY.EXE file in the target directory and when one is found,
it will be renamed based upon the version number. So, if
you had installed at c:\bin\ and your previous version
(v.2.82.7) is found, it will be renamed to xx2827.exe and
the new one will be placed right there. This allows that
you will always be able to go back to the older version by
typing xx2827 rather than xxcopy.

XXCOPY never insists on putting itself at c:\windows\command\
(in case of Win9x) or c:\windows\system32\ (in XP).
It is only a suggestion. One thing XXCOPY install does not
do is to patch the PATH= value. It is just a hassle that
we don't want to get into.

----------------------------------------------------------------


>KY> Let me start with the source (as you will see, this matter is not
>KY> very significant to the subject of how you lost your data).
>
>KY> Your command line had the "built-in" ambiguity as to the source
>KY> specifier.
>
>KY> Source: m:\2000ap~1\equis72
>KY> Destination: "t:\Daily Essential\equis72"
>
>KY> Here, the source can be interpreted in two ways. This
>KY> not-so-optimal characteristics of XXCOPY command line
>KY> is a direct result of making XXCOPY fully compatible
>KY> with Microsoft's XCOPY. That is, your source specifier
>KY> is ambiguous and I cannot tell which of the following
>KY> two ways to interpret it (nor can XXCOPY until it examines
>KY> the directory at m:\2000ap~1\). If, there is a directory
>KY> named "equis72" in m:\2000ap~1\, then, you are trying
>KY> to copy everything inside the m:\2000ap~1\equis72\ directory
>KY> (including all subdirectories) to the destination.
>
>Correct. I was trying to copy the directory m:\2000ap~1\equis72 and
>EVERYTHING in it to the destination. Something I've done probably
>1,000 times without a problem.
>
>KY> The second possible interpretation of your command line is when
>KY> there is no such directory as m:\2000ap~1\equis72\. In that case,
>KY> XXCOPY will interpret (just as XCOPY would) as the following:
>
>KY> Source base directory: m:\2000ap~1\
>KY> Lastname pattern: equis72
>
>KY> In this scenario, XXCOPY will try to copy a file named "equis72"
>KY> in every subdirectory under the m:\2000ap~1\ directory.
>KY> You should avoid such an ambiguous command line if you mean
>KY> the source specifier is the name of the directory by adding
>KY> the trailing backslash.
>
>Kan, I believe you are saying above that the *only* way this can be
>ambiguous is if the complete source, as specified, does not exist. Am
>I correct? In any case, that was not a problem here. The complete
>source existed, and still exists. Simply rolling back to the old
>version of xxcopy allowed the batch to run perfectly.

The ambiguity is inherent to the command line that you create.
It is only you who can say it is not ambiguous. Moreover,
even you cannot guarantee that the m:\2000ap~1\equis72\ directory
really exists at all time. What if a virus wipes out that directory
prior to the XXCOPY execution? Or, someone came to your computer
and did something. One way or another, the fact remains true
that the command line itself does not resolve the ambiguity
until the contents of the volume is examined and the presence
of the intended directory is verified. That is why we strongly
suggest that the trailing backslash is almost "mandatory" and
I encourage this every time I have occasion for it. If you
browse the technical bulletins, you will find nearly all examples
follow this recommendation. There is no advantage in keeping
the deliberate command ambiguity when it is easily avoidable
(and accidental massive deletion at the destination side
due to the missing directory --- regardless of how that may
happen).

KY> here's the heart of my analysis.

>It was present, I can assure you. This is a daily essential backup
>to a USB drive. Takes only a few minutes, and is done *every* day.
>This is the first problem I've ever had with it. The destination
>directory was there, for sure.

Your word of assurance cannot be expressed in the command line.
Please accept my advice that the trailing backslash should be
placed when you use the name of the source directory name.
Even though XXCOPY is an extension of XCOPY, when it comes to
the syntactic ambiguity, Robocopy's command syntax which
avoid this ambiguity by not allowing the "illogical" concatenation
of two distinct items into one (the source base directory +
the lastname-pattern) as if they were a valid "pathspec".

If we find sufficient number of defiant users on this issue,
We could one day abandon the XCOPY-compatible syntax in favor
of Robocopy-like syntax for the source specifier. At present,
we recognize the weakness of XCOPY-syntax and we strongly
advocate the use of the trailing backslash. Please swallow it.

>KY> So, let us study the two possible scenarios:
>
>KY> 1. If the "T:\Daily Essential\equis72\" directory existed:
>
>KY> Depending on what was in the source directory, some or
>KY> nearly *ALL* of the contents in the destination directory
>KY> could be "wiped out". That is by design and that is what
>KY> /CLONE operation is all about. Nothing to panic except if
>KY> you did not have a valid directory in the source as
>KY> expected, you may see XXCOPY do its job correctly by
>KY> removing all files and directories from the destination
>KY> directory. But, that's nothing to complain here.
>
>KY> 2. If "T:\Daily Essential\equis72\" directory did *NOT* exist.
>
>KY> In this case, xxcopy will not perform any delete operation.
>KY> Since there was nothing in the destination, there was
>KY> nothing to delete from the destination, nor there was any
>KY> file to be overwritten. Therefore, in this case, there
>KY> were absolutely no files to lose.
>
>KY> ----------
>
>KY> By this analysis, the worst case scenario is that you may
>KY> lose some or *ALL* of the files that were already present
>KY> in the T:\Daily Essential\equis72\ directory. But, under
>KY> no circumstances, XXCOPY will touch anything outside of
>KY> the destination directory that you have specified (this
>KY> fact is based on the assumption that there is no serious
>KY> bug with respect to the destination --- we are not aware
>KY> of any bug of this sort).
>
>KY> In short, the damage should have been confined to inside
>KY> the destination directory. If you had misspelled the
>KY> destination directory (and the wrong directory did not
>KY> exist), then, no harm would be done (this would be
>KY> similar to Case 2 above).
>
>I understand this, Kan. And I agree with you about what *should*
>have happened, and what we would expect to have happened based on
>these switches and how they work.
>
>However, I can tell you that the drive was wiped. I had about 30 GB
>of partition images there, outside the "daily essential\" folder, and
>they were gone, as was every other folder or piece of data outside
>that folder.
>
>What was there, was "daily essential\equis72\the entire root
>structure of m:\2000ap~1. Actually, the entire structure was not
>there, because I stopped the batch from completing when I realized
>there was a problem.
>
>I tried this several times before rolling back xxcopy versions, and
>the behavior was always the same.
>
>KY> So far, we have no confirmed behavior ever that the
>KY> /Z function ever harmed directory contents other than
>KY> the specified destination directory. I was running
>KY> quite a few tests today on both WinXP and Win98SE
>KY> environments and I have not observed any XXCOPY
>KY> behavior that would alter contents outside the
>KY> destination directory.
>
>KY> This is the end of my analysis on the XXCOPY itself.
>
>KY> =========================================================
>
>KY> Let me add my thoughts on other issues that are
>KY> outside XXCOPY's activities.
>
>KY> I'm not sure how to interpret your report. First of all,
>KY> what I don't understand is what procedure you took to
>KY> "slammed it shut". The proper way to terminate an ongoing
>KY> XXCOPY operation is to hit Ctrl-Break. This is the best
>KY> way to end a session because XXCOPY will gracefully
>KY> terminate the action immediately after the handling of
>KY> the current file is completed. Under no circumstances,
>KY> XXCOPY will abort a file I/O operation in the middle and
>KY> keep an open file open. Alternatively, you may click
>KY> the Window-Close button (the top-right corner [X]), and you
>KY> click [YES] to the confirming dialog. The second method
>KY> is much inferior to the first (Ctrl-Break) method, because
>KY> you are forcing an active program to be aborted.
>
>I was not very graceful about this, I admit. We ladies sometimes
>panic, and when I realized there was a big problem, I hit the panic
>button. I clicked the close box on the DOS window and terminated.

Wow, up to this point I thought I was talking to a man.
Sorry for my stereotyping. In this business, 95% of
our customers are male and I have tendency to assume as
such. I usually do not like the wording like "he or she".
Not that I don't have respect to female professionals in
this field. Melissa in this group is one whose feedback
over the years has helped us tremendously. Actually,
you are the first "yamato-nadeshiko" that I encountered
who can talk the file management subject in great detail.

>I did read to the bottom, and I apologize for not being more clear
>about this. I didn't yank the cable or throw the power switch. ^_^
>I'm not THAT bad. ^^_^^
>
>Well, I guess I've said about all I can say regarding this, Kan.
>Essentially, what we have here is a batch file that has worked for
>months upon months. Never a problem. Big problem after the install
>of the new xxcopy version yesterday. Problem disappeared like magic
>after the rollback. I don't know what else to say.
>
>Best,
>
>Yuki

==========

OK, Yuki, I'm convinced that we are dealing with something
quite serious here. We are now withdrawing the latest
beta test version v.2.83.x from our server until we find
out more about this incident.

Kan Yabumoto



This message if part of XXCOPY's message Archive.
The archive contains all the messages posted at Yahoo!Groups: XXCOPY.