XXCOPY
[ Back to Table of Contents ] [ << ] [ >> ]

XXCOPY TECHNICAL BULLETIN #24


From:    Kan Yabumoto           tech@xxcopy.com
To:      XXCOPY user
Subject: XXCOPY's Macro processing feature
Date:    2010-01-01  (revised)
===============================================================================

Introduction:

    When you create a batch file which performs a daily backup, you
    may want to name the destination directory after the current
    date and/or time.  Using the standard batch file this is rather
    difficult without a help of some fancy batch enhancer.

    For example, you may want to name today's backup destination
    directory using the current time

       XXCOPY C:\   D:\BACKUP\DEC-20\                   /S
       XXCOPY C:\   D:\BU001225\                        /S
       XXCOPY C:\   D:\BACKUP\2000-12-25\               /S
       XXCOPY C:\   D:\BACKUP\DAILY\FEB.04.2000.BUP\    /S
       XXCOPY C:\   D:\BACKUP\WEEK\BKUP_WED.BUP\        /S

    What is needed is to synthesize certain names based on the current
    system time.


The XXCOPY macro processing feature.

    XXCOPY provides a set of predefined variables which can be entered
    in your command line (or the current value in the string of environment
    variable, XXCOPY).  The technique used here is sometimes called
    "Macro processing" albeit this is a very small implementation which
    allows you to reference (you cannot add new macro definitions) a
    predefined value through a simple rule of substitution.

    Macro reference may be used anywhere in your command line (or the
    XXCOPY environment variable (XXCOPY and XXCOPYX), or even in the
    exclusion item parameters.  That is, anywhere you normally supply
    a string of characters, you can place a macro reference.

    The trouble is, people's tastes are so different, providing just
    one way would always make someone unhappy.  Take the date for
    example, despite our crusade in advocating the ISO 8601 standard
    (yyyy-mm-dd), few would listen.  So, XXCOPY provides just a few
    building blocks and lets you combine them whichever way you want.


The macro reference examples

    Before showing the gory details, let me show you examples which give
    you a good start.  Just observe what is going on.  You will get a
    pretty good idea how XXCOPY processes the macros.

    Let's assume it is  Friday, July 4, 2003 01:05:30 pm.

    /$DATE$        --->  0704          // common 4-digit month/day
    /$TIME$        --->  1305          // common 4-digit hour/minute
    /$YYYY-MM-DD$  --->  2003-07-04    // hyphens may be added
    /$M-D-YY$      --->  7-4-03        // no leading zeros for mn dy
    /$DD.MM.YY$    --->  04.07.03      // Europeans may like this
    /$MON-DD$      --->  JUL-04        // 3-letter month may be used
    /$WWW$         --->  FRI           // 3-letter day of week
    /$IIII-IWK-K$  --->  2003-W17-5    // ISO-8601, Year, week no, day-of-week


What is a valid macro reference?

    /$xxxxxx$

    XXCOPY's macro reference always starts with a slash (/) which is
    immediately followed by a macro reference string which starts with
    a dollar sign ($) followed by zero or more macro keywords and ends
    by a dollar sign.

    Within the two dollar signs of a macro reference (xxxxxx shown above),
    zero or more of predefined macro keywords may be specified (multiple
    keywords are combined without any separating character).

    The following keywords are currently supported (we may add more
    later).

      Keyword     Value       comment
     -----------------------------------------------------------
      HOST        MYWINPC     up to 15 character machine name
      YYYY        2003        4-digit year (good for 10000 years)
      YY          03          2-digit year (good for a century)
      Y           3           1-digit year (good for a decade)
      MON         DEC         3-letter month name
      MM          07          2-digit month
      M           7           1 or 2 digit month (without leading 0)
      DD          04          2-digit day
      D           4           1 or 2 digit day (without leading 0)
      HH          13          2-digit hour (00-23)
      H           13          1 or 2 digit hour (0-23) (without leading 0)
      NN          05          2-digit minute
      N           5           1 or 2 digit minute  (without leading 0)
      SS          30          2-digit second
      S           30          1 or 2 digit second  (without leading 0)
      DATE        0704        same as MMDD
      TIME        1305        same as HHNN
      WWW         FRI         3-letter day of week name
      W           5           1-digit day of week (Sun:0 - Sat:6)
      IIII        2003        4-digit year (ISO week number notation)
      II          03          2-digit year (ISO, 2-digit-year value)
      IWK         W17         2-digit week number (always prefixed by W)
      K           5           1-digit day of week (ISO, Mon:1 - Sun:7)


Single-key macro reference

    The simplest macro reference is made of one keyword inside.

      /$DATE$    --->   0704

    You may combine as many single-key macros

      /$YYYY$/$MM$/$DD$   --->    20030704


Multiple-key macro reference

    You may chain two or more keywords within a macro reference
    without any other character.  A macro reference is parsed
    from left to right and the longest string which matches the
    pattern will be used first.

      /$YYYYMMDD$  --->   20030704     // same as /$YYYY$/$MM$/$DD$
      /$YDATE$     --->   30704

    For readability and punctuation, the following characters are
    also supported.

       T ! # % & ' ( ) , - . @ _ { } ~ `

    These characters function as a keyword and represent the character
    literally (after expansion, they appear as typed in the reference).

      /$YYYY-MM-DD$  --->   2003-07-04


Miscellaneous rules.

    1.  You may use a keyword within a macro reference more than once.
        (In nearly all cases, this makes little sense, but it's allowed).

         /$YYYYYYYY$  --->   20032003  ( YYYY  and YYYY )
         /$YYYYYYY$   --->   2003033   ( YYYY and YY and Y )
         /$WWWW$      --->   FRI5      ( WWW and W )

    2.  Macro reference is case-insensitive

         /$DATE$      --->   0704
         /$date$      --->   0704
         /$DaTe$      --->   0704

    3.  Macro reference is case-preserving

         /$MON$       --->   JUL
         /$mon$       --->   jul
         /$Www$       --->   Fri
         /$wWw$       --->   fRi

    4.  A dollar sign ($) cannot be inside a macro reference.
        If you need a dollar sign, you should do it outside.

         /$MM$$/$DD$  --->   07$04    // a macro, $, and another macro

    5.  Year, followed by week-number should be denoted by

         /$IIII-IWK$  --->   2003-W27 // See below for details


Where can a macro reference be placed?

    You may use a macro reference string ( /$xxxx$ )  anywhere in your
    command line as well as in the environment variables, XXCOPY and
    XXCOPYX (yes, even inside an exclusion specifier).  That means, you
    may place it in the external exclusion list file (specified by
    the /EX switch).  It is also permitted in alternate filename template
    (/IN) specifiers.  In short, practically anywhere you enter a string
    of text, XXCOPY will accept a macro reference and expand it
    accordingly.


Recommendation of macro usage.

    For DOS-compatibility, you should keep the directory names to
    the 8.3 format.  It will be much easier to access directories
    from a real mode (DOS) environment.

    For routine backup, a five- or six-digit date code is probably
    a good choice for most occasions.

     d:\bu\bu/$yymmdd$.c\    --->   d:\bu\bu030704.c\
     d:\bu\b/$ymmdd$_c\      --->   d:\bu\b30704_c\
     d:\bu\bu_/$MONYY$\c\    --->   d:\bu\bu_JUL03\c\

    Using two-digit year value as opposed to one-digit year value
    is all up to you.  In general, just 1-digit year format gives
    you values that are unique for ten years.  That's quite adequate
    for most backup operations.  In this context, using a four-digit
    year value (/$YYYY/) in the directory name is overkill especially
    if the directory name exceeds the 8.3 format.


Environment variable (/%xxxx%).

    Aside from the pre-defined macro references, environment variables
    (E-var for short) can be referenced with a similar syntactic rule.
    Instead of the use of a pair of dollar signs for macro reference
    (e.g., /$DATE$), an E-var is referenced by a slash followed by a pair
    of percent signs (%) (e.g., /%USERNAME%).  

    Note that the use of an E-var on command line (in the XXCOPY invocation
    line) is already supported by Windows system command processor (CMD.EXE)
    without XXCOPY's intervention.  In such a case, the syntax is a pair of
    percent sign surrounding the name of E-var (without the leading slash
    character).

    E.g., (on command line)   xxcopy c:\  d:\save\%USERNAME%\   /backup

    The usefulness of referencing an E-var is inside a command file (or
    an exclusion file) text which is beyond the reach of CMD.EXE.

    Here is an example of E-var used inside an XXCOPY command file:

       /x:c:\users\/%USERNAME%\favorites\

    If the E-var for USERNAME is assigned to the word "john", then,
    the line above will be substituted to /x:c:\users\john\favorites\.

    XXCOPY processes a reference to E-var not only inside its command file
    (via /CF or /EX), but also in the command line.  However, in a typical
    invocation of XXCOPY that is handled by Windows system command processor
    (CMD.EXE), the command line that is passed to the XXCOPY program is
    already processed by CMD.EXE that handles the use of E-vars.  Therefore,
    XXCOPY's ability to process E-vars in the command line is usually moot.
    (Note that the leading slash character is needed by XXCOPY's case).

       -----------------------------------------------------------------
        For advanced users with academic curiosity, the following two
        lines will produce the same result.

          xxcopy  c:\  d:\save\%USERNAME%\  /backup
          xxcopy  c:\  d:\save\/%%USERNAME%%\  /backup

        Hint: the %% sequence is an escape sequence that nullifies the
        built-in E-var substitution by Windows command processor.
        /%USERNAME% will be the character sequence XXCOPY actually sees.
       -----------------------------------------------------------------


The ISO 8601 standard.

    We take a very strong position in endorsing the ISO 8601 standard
    which provides logical and convenient guidelines in describing
    date and time values.  Visit http://www.iso.ch.

    The most prominent aspect of the ISO 8601 standard is of course,
    its date order in the YYYY-MM-DD format and the 24-hour notation,
    hh:nn:ss.  XXCOPY also supports ISO's lesser-known week date
    representation.

    According to the ISO definition, the week begins on Monday and
    ends on Sunday.  XXCOPY denotes the ISO's day-of-the-week
    value using /$K$.  The only difference between ISO's day-of-the-
    week value (/$K$) and the traditional day-of-the-week radix (/$W$)
    counterpart is the value for Sunday (/$K$ = 7 and /$W$ = 0);
    other days (Monday through Saturday) will have the same values.

    ISO 8601 also defines the "Week date" which assigns a number to
    each week of the year (More examples can be found here).
    Unfortunately, when the first day of the year does not coincide
    with the beginning of the week (Monday in ISO's specification),
    the week will straddle over two years for which special rules
    are made.  A week belongs to the year which is more dominant
    (has four or more days in the week).  So, the first week of
    the year (W01) is defined as the week which contains the 4th day
    of January.  With this rule, up to three days at the beginning
    of the year and also up to three days at the end of the year
    the date may become a part of the previous calendar year or the
    next calendar year, respectively.  This may be a bit confusing,
    but that's the rule and we understand why it is necessary.

    The Week-of-the-year value is always described with the prefix, W
    followed by a two-digit decimal value (starting 01) which is
    referenced by the /$IWK$ keyword.

    E.g.   Fri, 2003-07-04  --->  2003-W27-5
           Sun, 2003-12-28  --->  2003-W52-7
           Mon, 2003-12-29  --->  2004-W01-1  == next year value
           Thu, 2004-01-01  --->  2004-W01-4
           Sat, 2005-01-01  --->  2004-W53-6  == prev year value

    For this irregularity in the year values, XXCOPY assignes the
    /$IIII$ (and /$II$ for the two-digit notation) to be used in
    the week-of-the-year context to differentiate it from the
    calendar year value (/$YYYY$ and /$YY$, respectively).


Q and A:

   Q:  Why T is in the list of punctuation characters?

   A:  T is a special punctuation character as an exception since
       our favorite standard, the ISO 8601 stipulates the use of T
       to connect the date part and the time part.

         /$YYYY-MM-DDTHH.NN.SS$ ---> 2003-07-04T13.05.30

       (Alas, the colon (:) character is not a legal character
        for a filename and the dots are used instead.)

       You may use @ as an alternative separator:

         /$YYYY-MM-DD@HH.NN.SS$ ---> 2003-07-04@13.05.30

   Q:  Is there any way to test exactly how my macro usage is working?

   A:  If you are not sure of the effect of a macro, add the
       /EC (echo) switch that displays the full command line
       after the macro references are expanded.

       Example:

          XXCOPY  \src  \dst\/$DATE$\/$YYMMDD$\  /EC /W

          Here, the /EC gives you the line after the macro expansion,
          and /W will pause and let you terminate the command if you
          don't like the effect of the macro.


   Q:  Why the slash (/) is used in the macro prefix?

   A:  We wish we have more characters in the keyboard.  Seriously,
       the design goal of the XXCOPY Macro feature was to allow you
       to use it in as many places as possible (within the source and
       destination specifiers, as well as in other switch parameters).
       Since the macro reference should be anywhere (including a
       partial word), the macro syntax has to use a terminator (hence,
       the $xxxx$ sandwich).  Because the dollar sign ($) is a valid
       character for filename, we need a prefix character which is
       not permitted in filename.  Actually, there are only nine
       such characters in English alphabet ( " * ? : < > | \ / ) that
       are not permitted in Windows' filename.

       Every one of these character has some form of conflict with
       other usages.

         "  *  ?  \ are out of the question.
         <  >  |  are reserved by the command shell (COMMAND.COM).

         :  used only at the drive letter designator.
         /  used as the prefix for the command switch.

       So the choice was between the two (: and /).  We chose the
       two-character sequence, "/$" of the macro reference as a
       distinguishing feature from other switches.  Fortunately,
       XXCOPY's command syntax stipulates that the source and destination
       specifiers to be always separated from other switch arguments by
       a space, therefore, a macro reference embedded inside source or
       destination specifier can unambiguously parsed.  It's not pretty.
       It took us several months to settle on this syntax.  Now the
       journey is over.  Enjoy the macro.



© Copyright 2016 Pixelab All rights reserved.

[ XXCOPY Home ] [ Table of Contents ] [ << ] [ >> ]

Join the XXCOPY group