From: Kan Yabumoto tech@xxcopy.com
To: XXCOPY user
Subject: The Exclusion specifier in XXCOPY
Date: 2010-12-07 (revised)
===============================================================================
Much of the mostly hidden power of XXCOPY lies in the exclusion
mechanism. Starting with v.2.97.7 (2009-01-13), the exclusion
specifier is fully compatible with the "Wild-Wildcard" feature.
That is, you may use wildcard characters (* and ?) anywhere in
the exclusion specifier in any number of times. (Earlier
versions of XXCOPY had complicated rules on the wildcard usage).
XXCOPY Exclusion switch syntax
/X<xspec> excludes the file or directory item given by
<xspec> which is an exclusion specifier.
If the specifier contains an embedded space,
the specifier must be surrounded by a pair
of double-quotes (").
/EX<xfile> specifies a text file whose name is <xfile>
which contains a list of <xspec> separated by space.
You may specify as many exclusion specifiers as you like.
Some examples of the /X switch
The exclusion parameter must be specified immeidately after /X
(without a space).
/Xc:\mydir\myfile.txt // specifies just a single file
/X*.tmp // all files that end with ".tmp"
/Xabc* // all files that start with "abc"
To improve readability, we suggest inserting a colon after X.
Here are more examples:
/X:mydir\ // the entire directory, "mydir" in the source
/X:mydir\*\* // same as /Xmydir\ which is a shortcut
/X:mydir\*\*.tmp // inside mydir, all files matching "*.tmp"
/X:my*xyz\*\abc*.c // in directories that match my*xyz, all "abc*.c"
/X:*\cache\ // multiple-level subdirectories
/X:*\cache\*\* // same as above with a trailing backslash
/X:*\cach?\*\* // multiple-level subdir spec may have wildcards
/X:"\Program Files\" // use quotes (") with a pattern with a space
/X:C:\windows\ // it's OK. Two colons with a full path name
Here, you may see the glimpse of the powerful syntax in the exclusion
specifier. The first example seems the most straight forward. The
fourth example which ends with a backslash is a shorthand of for the
common case of excluding a directory (it abbreviates "*\*" which follows).
Therefore, all of the above examples except the first one contain
or imply at least one wildcard specifier. The last example includes
one asterisks in each of the three parts.
Don't worry about the complexity yet. At least the first example shows
a case which you can use it immediately without any further reading.
Yes, if you have energy to list all of the files you want to exclude,
you may painstakingly list all of such files by giving the full
file specification of each file. Since you will soon run out of the
command line space, you will probably want to set up a text file using
the /EX switch.
E.g., /EXmyexcl.lst
and myexcl.lst contains the following specifiers:
// this is a comment line
c:\win386.swp // comment may start like this
c:\autoexec.bat myfile.tmp // one line may have multiple items
"c:\program files" // use quotes (") for embedded space
mydir\myfile.txt // pathspec relative to the source dir
yourdir\ // entire yourdir\*\*
Characteristics of exclusion specifier (a short explanation):
1. The Wild-Wildcard feature fully supported for exclusion.
2. A pattern for direcotires always ends with a backslash.
3. A pattern for files has no trailing backslash.
4. A relative specifier is relative to the source base directory.
Syntactic rules of exclusion specifier:
1. Add the exclusion item immediately (without space) after /X.
/Xmyfile.txt // exclude "myfile.txt" anywhere
2. You may insert an optional colon (:) between /X and the rest.
/X:xyz.doc // the colon improves the readability
3. The trailing backslash distinguishes directory from file.
/X:Windows\ // specifies the "Windows" directory
/X:pagefile.sys // specifies the "pagefile.sys" file
4. Use a pair of quotes (") for an exclusion item with embeded space.
/X:"\Program Files\" // the space would split the string
5. Absolute exclusion specifiers start with a backslash
(with an optional drive letter or UNC pattern).
/X:c:\mydir\myfile.txt // an absolute pattern
/X:\mydir\myrecord.doc // you may omit the drive letter
/X:\\john\c\windows\ // UNC always starts with \\
6. Relative exclusion specifiers (without leading backslash)
are relative to the source base specifier.
/X:dir_abc\myfile.txt // relative to the source base dir
7. A bare exclusion item (without any backslash) is treated
as a file name pattern that applies to all directories.
/X:desktop.ini // equivalent to /X:*\desktop.ini
/X:*.mp3 // equivalent to /X:*\*.mp3
This feature allows for a filename pattern to apply for all
directories. On the other hand, you may want to specify a
file pattern that explicitly apply only to the files in the
source base directory. In such a case, use a ".\" prefix.
/X:.\*.tmp // for files in the top directory
8. A bare directory pattern (the only backslash is at the end)
is treated a pattern that applies only to the source base
directory, unlike the bare file name case exlained above.
/X:dir_xyz\ // applies only the top directory
/X:.\dir_xyz\ // the same as the one above
The Wild-Wildcard feature in exclusion specifier:
Starting with v.2.97.7 (2009-01-01), you may use wildcard character
anywhere in an exclusion specifier for any number of times.
Astrisk (*) denotes a string of zero or more of any character.
Question Mark (?) denotes exactly one character.
/X:*.mp3 // any file that ends with ".mp3"
/X:abc*\ // any directory that starts with "abc"
/X:xyz??? // starts with xyx followed with 3 more chars
/X:*a* // a filename that has at least one letter "a"
The Special "wild subdirectory" (..\*\..) notation:
The \*\ (an asterisk sandwiched by backslashes) sequence
denotes a pathname pattern with zero or more of directories.
/X:c:\mydir\*\abc\ excludes the following directories
c:\mydir\abc\
c:\mydir\xyz\abc\
c:\mydir\klm\xyz\abc\
Such a pattern does not always match the first one.
/X:c:\mydir\*\a\b\ excludes the following directories
c:\mydir\a\b\ // the first instance
c:\mydir\a\xyz\a\b\ // 1st \a\ does not but 2nd one does
Syntax rule for the Exclusion List File (/EX).
An "Exclusion List File" specified in the /EX switch is a plain
text file which contains a list of exclusion specifiers.
You may list as many exclusion specifiers in one line.
Exclusion specifiers are separated by one or more blank, tab,
and/or newline character. An exclusion specifier cannot be
broken into two or more lines. When a space character is
embedded, the exclusion specifier must be surrounded by a
pair of double-quotes ("). A line may contain a comment field
which will be ignored by XXCOPY. A comment field starts with
two consecutive slashes (//) or two consecutive colons (::)
and ends at the end of the line. You may also use another
comment style (starts with /* and ends with */ that can span
over multiple lines).
A few examples of common exclusion specifiers.
The directory template may have the following four variations
in the ending.
dir_pattern\ // full directory
dir_pattern\*\* // full directory (same as above)
dir_pattern\* // files in the directory (one-level)
dir_pattern\?*\ // all subdirectories but not files
Even though the second example is to specify files (as opposed
to directories) without the mandatory trailing backslash,
it behaves exactly like the first one because it covers
all files in all subdirectories which is equivalent to the
whole directory contents that is the directory itself.
Examples:
/X:dirtmpl\*\* // excludes all matching directories in the
// base (src) directory and its contents
/X:dirtmpl\ // same as above (the triling backslash
// denotes everything inside the directory)
/X:.\dirtmpl\ // in the case of the directory template,
// it applies to one directory relative to
// the base (src) directory (1-Level)
/x:*\dirtmp\ // you may make a directory template apply
// to many instances (Multi-Level)
/x:c:\windows\* // specifies all the files in the first
// level of the c:\Windows directory such
// as, EXPLORER.EXE, WIN.INI,
/x:c:\windows\?*\ // this does not includes the first level
// files but all subdirectories in it such
// as \WINDOWS\SYSTEM\ \WINDOWS\DESKTOP\, etc.
Note that the following two are distinct:
/Xdir_spec\* // one layer only (subdirectories not excluded)
/Xdir_spec\*\* // the entire dir_spec directory is excluded
Beware of a few pitfalls:
xxcopy mydir\ x:\mydst\ /S /X:mydir\a*\
The exclusion item specifies a pattern (directory starting with
letter a) as a relative path. As you explained above, it is
relative to the source base directory. Therefore, this command
excludes directories that matches mydir\mydir\a*\ not
the subdirectories immediately in the mydir\. Rather, the
exclusion specifier stipulate mydir inside the source base
directory which happens to spell the same.
xxcopy \src\ \dst\ /S /X:*.tmp
The exclusion specifier is equivalent to /X:*\*.tmp that applies
to all subdirectories.
xxcopy "C:\Program Files\" \dst\ /S /X:Microsoft*\
The exclusion specifier deos not implicitly applies to all
subdirectories as the filename template does. In this case, only
directories whose name starts with Microsoft immediately inside
the C:\Program Files\ directory, but not at other levels.
The optimization of exclusion matching.
In a very large scale backup operation, an XXCOPY job may encompass
an entire volume as the source directory (such as c:\*). To make
the matters worse, the more files the source directory contains,
the more the need for the exclusion specifiers grows. Therefore,
it is entirely possible that the entire C: drive may contain
70,000 files and the total number of exclusion items the user specify
in the exclusion list file with the /EX switch may contain literally
hundreds of various exclusion specifiers. If we were to test every
file against this very large number of exclusion list, the combination
will easily reach tens of millions which would slow down the entire
backup process. Therefore, XXCOPY performs preprocessing steps
to analyze the set of exclusion specifiers. First by classifying
them into the five classes, some redundant exclusion specifiers can
be removed.
Debug feature
Because a liberal use of wildcard characters in exclusion feature
could become fairly complex, sometimes the user may want to test
if a given set of exclusion items really achieve the intended
result.
The /oX switch generates a list of exclusion specifiers after
XXCOPY's initial exclusion parameter optimization step. The list
is organized by the internal classification of the wildcard usage.
It is most effective in combination with the /DEBUG function.
(This output is mostly for Pixelab's support activities.)
The /ZLX function provides a special test by listing all the files
and directories that would be excluded without actually making any
copy or delete operation. It suppresses the display of regular
(selected) files and generates a list of files and directories that
would be excluded by the /X and /EX switches. It is analogous to
the /L function that lists the files that would be selected by
a given set of command arguments.
Automatically excluded files.
Since the few output files (e.g., the error log files) which are
generated by the XXCOPY program itself could not be successfully
included in the current copying job if any of them happens to be
in the source directory (or its subdirectories), those files will
be automatically excluded.
Note on earlier (pre v.2.97.7) versions:
In earlier versions of XXCOPY (prior to v.2.97.7), the exclusion
feature was the hardest one to fully master by XXCOPY's users.
That was because the exclusion specifier in earlier versions
did not support the "Wild-Wildcard" capablitity to the fullest
extent. Rather, XXCOPY provided a total of 11 classes of wildcard
usages with a complex set of syntactic rules. So, the many users
had to stick with a few simplest ways to add wildcard characters
for the exclusion items. It was simply very difficult to implement
an efficent algorithm to support such a feature.
In the old (pre-v.2.97.7) versions, the exclusion feature defined
a special syntactic rule as follows:
dir_pattern\?\*
When an exclusion item with the \?\* ending, it excludes the
subdirectories of the directories that matches the pattern. The
above exclusion pattern can be written as
dir_pattern\?*\
which does not rely on the special syntactic rule (which is now
being abolished).
Use of the /GX switch:
In order to support the old syntax for the \?\* ending in the
exclusion specifier, we have added the /GX switch with which
a command line with the older exclusion specifier is treated
in the old way.
However, we urge all users to replace all usages of \?\* ending
with \?*\ that excludes exactly the same group of subdirectories.
[ Table of Contents ]
[ << ]
[ Show as Detached ]
[ >> ]
|
On-Line Manual 
XXTB #003