+ +Previous: Regexp Summary, +Up: Regexps +
The syntax and semantics of PCRE regular expressions, as used in +Monotone, are described in detail below. Regular expressions in +general are covered in a number of books, some of which have copious +examples. Jeffrey Friedl's “Mastering Regular Expressions,” +published by O'Reilly, covers regular expressions in great detail. +This description is intended as reference material. + +
A regular expression is a pattern that is matched against a subject +string from left to right. Most characters stand for themselves in a +pattern, and match the corresponding characters in the subject. As a +trivial example, the pattern + +
+ The quick brown fox ++ +
matches a portion of a subject string that is identical to +itself. When caseless matching is specified, letters are matched +independently of case. + +
The power of regular expressions comes from the ability to include +alternatives and repetitions in the pattern. These are encoded in the +pattern by the use of metacharacters, which do not stand for +themselves but instead are interpreted in some special way. + +
There are two different sets of metacharacters: those that are +recognized anywhere in the pattern except within square brackets, and +those that are recognized within square brackets. Outside square +brackets, the metacharacters are as follows: + +
\^$.[|()?*+{Part of a pattern that is in square brackets is called a "character +class". In a character class the only metacharacters are: + +
\^-[]The following sections describe the use of each of the metacharacters. + +
The backslash character has several uses. Firstly, if it is followed +by a non-alphanumeric character, it takes away any special meaning +that character may have. This use of backslash as an escape character +applies both inside and outside character classes. + +
For example, if you want to match a `*' character, you write +`\*' in the pattern. This escaping action applies whether or not +the following character would otherwise be interpreted as a +metacharacter, so it is always safe to precede a non-alphanumeric with +backslash to specify that it stands for itself. In particular, if you +want to match a backslash, you write `\\'. + +
If a pattern is compiled with the `(?x)' option, whitespace in +the pattern (other than in a character class) and characters between a +`#' outside a character class and the next newline are +ignored. An escaping backslash can be used to include a whitespace or +`#' character as part of the pattern. + +
If you want to remove the special meaning from a sequence of +characters, you can do so by putting them between `\Q' and +`\E'. The `\Q...\E' sequence is recognized both inside and +outside character classes. + +
A second use of backslash provides a way of encoding non-printing characters +in patterns in a visible manner. There is no restriction on the appearance of +non-printing characters, apart from the binary zero that terminates a pattern, +but when a pattern is being prepared by text editing, it is usually easier to +use one of the following escape sequences than the binary character it +represents: + +
\a\cx\e\f\n\r\t\ddd\xhh\x{hhh...}The precise effect of `\cx' is as follows: if x is a lower +case letter, it is converted to upper case. Then bit 6 of the +character (hex 40) is inverted. Thus `\cz' becomes hex 1A (the +<SUB> control character, in ASCII), but `\c{' becomes hex 3B +(`;'), and `\c;' becomes hex 7B (`{'). + + + +
After `\x', from zero to two hexadecimal digits are read (letters +can be in upper or lower case). Any number of hexadecimal digits may +appear between `\x{' and `}', but the value of the +character code must be less than 256 in non-UTF-8 mode, and less than +231in UTF-8 mode. That is, the maximum value in hexadecimal is +7FFFFFFF. Note that this is bigger than the largest Unicode code +point, which is 10FFFF. + +
If characters other than hexadecimal digits appear between `\x{' +and `}', or if there is no terminating `}', this form of +escape is not recognized. Instead, the initial `\x' will be +interpreted as a basic hexadecimal escape, with no following digits, +giving a character whose value is zero. + +
Characters whose value is less than 256 can be defined by either of +the two syntaxes for `\x'. There is no difference in the way they +are handled. For example, `\xdc' is exactly the same as +`\x{dc}'. + +
After `\0' up to two further octal digits are read. If there are +fewer than two digits, just those that are present are used. Thus the +sequence `\0\x\07' specifies two binary zeros followed by a +<BEL> character (octal 007). Make sure you supply two digits after +the initial zero if the pattern character that follows is itself an +octal digit. + +
The handling of a backslash followed by a digit other than 0 is +complicated. Outside a character class, PCRE reads it and any +following digits as a decimal number. If the number is less than 10, +or if there have been at least that many previous capturing left +parentheses in the expression, the entire sequence is taken as a +back reference. A description of how this works is given later, +following the discussion of parenthesized subpatterns. + +
Inside a character class, or if the decimal number is greater than 9 +and there have not been that many capturing subpatterns, PCRE re-reads +up to three octal digits following the backslash, and uses them to +generate a data character. Any subsequent digits stand for +themselves. In non-UTF-8 mode, the value of a character specified in +octal must be less than `\400'. In UTF-8 mode, values up to +`\777' are permitted. For example: + +
\040\40\7\11\011\0113\113\377\81Note that octal values of 100 or greater must not be introduced by a +leading zero, because no more than three octal digits are ever read. + +
All the sequences that define a single character value can be used +both inside and outside character classes. In addition, inside a +character class, the sequence `\b' is interpreted as the <BS> +character (hex 08), and the sequences `\R' and `\X' are +interpreted as the characters `R' and `X', +respectively. Outside a character class, these sequences have +different meanings (see below). + +
The sequence `\g' followed by an unsigned or a negative number, +optionally enclosed in braces, is an absolute or relative back +reference. A named back reference can be coded as +`\g{name}'. Back references are discussed later, following the +discussion of parenthesized subpatterns. + +
Another use of backslash is for specifying generic character types. The +following are always recognized: + +
\d\D\h\H\s\S\v\V\w\WEach pair of escape sequences partitions the complete set of +characters into two disjoint sets. Any given character matches one, +and only one, of each pair. + +
These character type sequences can appear both inside and outside +character classes. They each match one character of the appropriate +type. If the current matching point is at the end of the subject +string, all of them fail, since there is no character to match. + +
For compatibility with Perl, `\s' does not match the <VT> +character (code 11). This makes it different from the the POSIX +“space” class. The `\s' characters are <TAB> (9), <LF> +(10), <FF> (12), <CR> (13), and <SPACE> (32). + +
In UTF-8 mode, characters with values greater than 128 never match +`\d', `\s', or `\w', and always match `\D', +`\S', and `\W'. These sequences retain their original +meanings from before UTF-8 support was available, mainly for +efficiency reasons. + +
The sequences `\h', `\H', `\v', and `\V' are Perl +5.10 features. In contrast to the other sequences, these do match +certain high-valued codepoints in UTF-8 mode. The horizontal space +characters are: + +
U+0009U+0020U+00A0U+1680U+180EU+2000U+2001U+2002U+2003U+2004U+2005U+2006U+2007U+2008U+2009U+200AU+202FU+205FU+3000The vertical space characters are: + +
U+000AU+000BU+000CU+000DU+0085U+2028U+2029A “word” character is an underscore or any character less than 256 +that is a letter or digit. The definition of letters and digits is +that used for the “C” locale. + +
PCRE supports five different conventions for indicating line breaks in +strings: a single CR (carriage return) character, a single LF +(linefeed) character, the two-character sequence CRLF, any of the +three preceding, or any Unicode newline sequence. The default is to +match any Unicode newline sequence. It is possible to override the +default newline convention by starting a pattern string with one of +the following five sequences: + +
(*CR)(*LF)(*CRLF)(*ANYCRLF)(*ANY)For example, the pattern + +
+ (*CR)a.b ++ +
changes the convention to CR. That pattern matches `a\nb' because +LF is no longer a newline. Note that these special settings, which are +not Perl-compatible, are recognized only at the very start of a +pattern, and that they must be in upper case. If more than one of them +is present, the last one is used. + +
The newline convention does not affect what the `\R' escape +sequence matches. By default, this is any Unicode newline sequence, +for Perl compatibility. However, this can be changed; see the +description of `\R' below. A change of `\R' setting can be +combined with a change of newline convention. + +
Outside a character class, by default, the escape sequence `\R' matches +any Unicode newline sequence. This is a Perl 5.10 feature. In +non-UTF-8 mode `\R' is equivalent to the following: + +
+ (?>\r\n|\n|\x0b|\f|\r|\x85) ++ +
This is an example of an "atomic group", details of which are given
+below. This particular group matches either the two-character
+sequence <CR> followed by <LF>, or one of the single
+characters <LF> (linefeed, U+000A), <VT> (vertical tab,
+U+000B), <FF> (formfeed, U+000C), <CR> (carriage
+return, U+000D), or <NEL> (next line, U+0085). The
+two-character sequence is treated as a single unit that cannot be
+split. In UTF-8 mode, two additional characters whose codepoints are
+greater than 255 are added: <LS> (line separator, U+2028)
+and <PS> (paragraph separator, U+2029).
+
+
It is possible to change the meaning of `\R' by starting a +pattern string with one of the following sequences: + +
(*BSR_ANYCRLF)(*BSR_UNICODE)Note that these special settings, which are not Perl-compatible, are +recognized only at the very start of a pattern, and that they must be +in upper case. If more than one of them is present, the last one is +used. They can be combined with a change of newline convention, for +example, a pattern can start with: + +
+ (*ANY)(*BSR_ANYCRLF) ++ +
Inside a character class, `\R' matches the letter `R'. + +
Three additional escape sequences match characters with specific +Unicode properties. When not in UTF-8 mode, these sequences are of +course limited to testing characters whose codepoints are less than +256, but they do work in this mode. The extra escape sequences are: + +
\p{xx}\P{xx}\XThe property names represented by xx above are limited to the +Unicode script names, the general category properties, and `Any', +which matches any character (including newline). Other properties such +as `InMusicalSymbols' are not currently supported by PCRE. Note +that `\P{Any}' does not match any characters, so always causes +a match failure. + +
Sets of Unicode characters are defined as belonging to certain +scripts. A character from one of these sets can be matched using a +script name. For example: + +
+ \p{Greek}
+ \P{Han}
+
+
+Those that are not part of an identified script are lumped together as +“Common.” The current list of scripts is: + +
Arabic, +Armenian, +Balinese, +Bengali, +Bopomofo, +Braille, +Buginese, +Buhid, +Canadian_Aboriginal, +Cherokee, +Common, +Coptic, +Cuneiform, +Cypriot, +Cyrillic, +Deseret, +Devanagari, +Ethiopic, +Georgian, +Glagolitic, +Gothic, +Greek, +Gujarati, +Gurmukhi, +Han, +Hangul, +Hanunoo, +Hebrew, +Hiragana, +Inherited, +Kannada, +Katakana, +Kharoshthi, +Khmer, +Lao, +Latin, +Limbu, +Linear_B, +Malayalam, +Mongolian, +Myanmar, +New_Tai_Lue, +Nko, +Ogham, +Old_Italic, +Old_Persian, +Oriya, +Osmanya, +Phags_Pa, +Phoenician, +Runic, +Shavian, +Sinhala, +Syloti_Nagri, +Syriac, +Tagalog, +Tagbanwa, +Tai_Le, +Tamil, +Telugu, +Thaana, +Thai, +Tibetan, +Tifinagh, +Ugaritic, +Yi. + +
Each character has exactly one general category property, specified by a +two-letter abbreviation. For compatibility with Perl, negation can be specified +by including a circumflex between the opening brace and the property name. For +example, `\p{^Lu}' is the same as `\P{Lu}'. + +
If only one letter is specified with `\p' or `\P', it +includes all the general category properties that start with that +letter. In this case, in the absence of negation, the curly brackets +in the escape sequence are optional; these two examples have the same +effect: + +
+ \p{L}
+ \pL
+
+
+The following general category property codes are supported: + +
CCcCfCnCoCsLLlLmLoLtLuMMcMeMnNNdNlNoPPcPdPePfPiPoPsSScSkSmSoZZlZpZsThe special property `L&' is also supported: it matches a +character that has the `Lu', `Ll', or `Lt' property, in +other words, a letter that is not classified as a modifier or +“other.” + +
The `Cs' (Surrogate) property applies only to characters in the
+range U+D800 to U+DFFF. Such characters are not valid in
+UTF-8 strings (see RFC 3629) and so cannot be tested by PCRE.
+
+
The long synonyms for these properties that Perl supports (such as +`\p{Letter}') are not supported by PCRE, nor is it permitted to +prefix any of these properties with `Is'. + +
No character that is in the Unicode table has the `Cn' +(unassigned) property. Instead, this property is assumed for any code +point that is not in the Unicode table. + +
Specifying caseless matching does not affect these escape sequences. For +example, `\p{Lu}' always matches only upper case letters. + +
The `\X' escape matches any number of Unicode characters that +form an extended Unicode sequence. `\X' is equivalent to + +
+ (?>\PM\pM*) ++ +
That is, it matches a character without the “mark” property, +followed by zero or more characters with the “mark” property, and +treats the sequence as an atomic group (see below). Characters with +the “mark” property are typically accents that affect the preceding +character. None of them have codepoints less than 256, so in non-UTF-8 +mode `\X' matches any one character. + +
Matching characters by Unicode property is not fast, because PCRE has +to search a structure that contains data for over fifteen thousand +characters. That is why the traditional escape sequences such as +`\d' and `\w' do not use Unicode properties in PCRE. + +
The escape sequence `\K', which is a Perl 5.10 feature, causes +any previously matched characters not to be included in the final +matched sequence. For example, the pattern: + +
+ foo\Kbar ++ +
matches `foobar', but reports that it has matched +`bar'. This feature is similar to a lookbehind assertion +(described below). However, in this case, the part of the subject +before the real match does not have to be of fixed length, as +lookbehind assertions do. The use of `\K' does not interfere with the +setting of captured substrings. For example, when the pattern + +
+ (foo)\Kbar ++ +
matches `foobar', the first substring is still set to `foo'. + +
The final use of backslash is for certain simple assertions. An +assertion specifies a condition that has to be met at a particular +point in a match, without consuming any characters from the subject +string. The use of subpatterns for more complicated assertions is +described below. The backslashed assertions are: + +
\b\B\A\Z\z\GThese assertions may not appear in character classes (but note that +`\b' has a different meaning, namely the backspace character, +inside a character class). + +
A word boundary is a position in the subject string where the current +character and the previous character do not both match `\w' or +`\W' (i.e. one matches `\w' and the other matches +`\W'), or the start or end of the string if the first or last +character matches `\w', respectively. + +
The `\A', `\Z', and `\z' assertions differ from the +traditional circumflex and dollar (described in the next section) in +that they only ever match at the very start and end of the subject +string, whatever options are set. Thus, they are independent of +multiline mode. The difference between `\Z' and `\z' is that +`\Z' matches before a newline at the end of the string as well as +at the very end, whereas `\z' matches only at the end. + +
The `\G' assertion is true only when the current matching +position is at the start point of the match. As used in Monotone, +`\G' is always equal to `\A'. + +
Outside a character class, in the default matching mode, the +circumflex character, `^', is an assertion that is true only if +the current matching point is at the start of the subject string. +Inside a character class, circumflex has an entirely different meaning +(see below). + +
Circumflex need not be the first character of the pattern if a number +of alternatives are involved, but it should be the first thing in each +alternative in which it appears if the pattern is ever to match that +branch. If all possible alternatives start with a circumflex, that is, +if the pattern is constrained to match only at the start of the +subject, it is said to be an “anchored” pattern. (There are also +other constructs that can cause a pattern to be anchored.) + +
A dollar character, `$', is an assertion that is true only if the +current matching point is at the end of the subject string, or +immediately before a newline at the end of the string (by +default). Dollar need not be the last character of the pattern if a +number of alternatives are involved, but it should be the last item in +any branch in which it appears. Dollar has no special meaning in a +character class. + +
The meanings of the circumflex and dollar characters are changed if +the `(?m)' option is set. When this is the case, a circumflex +matches immediately after internal newlines as well as at the start of +the subject string. It does not match after a newline that ends the +string. A dollar matches before any newlines in the string, as well as +at the very end, when `(?m)' is set. When newline is specified as +the two-character sequence <CR><LF>, isolated <CR> and +<LF> characters do not indicate newlines. + +
For example, the pattern `^abc$' matches the subject string +`def\nabc' (where `\n' represents a newline) in multiline +mode, but not otherwise. Consequently, patterns that are anchored in +single line mode because all branches start with ^ are not anchored in +multiline mode. + +
Note that the sequences `\A', `\Z', and `\z' can be +used to match the start and end of the subject in both modes, and if +all branches of a pattern start with `\A' it is always anchored. + +
Outside a character class, a dot in the pattern matches any one +character in the subject string except (by default) a character that +signifies the end of a line. In UTF-8 mode, the matched character may +be more than one byte long. + +
When a line ending is defined as a single character, dot never matches +that character; when the two-character sequence <CR><LF> is +used, dot does not match <CR> if it is immediately followed by +<LF>, but otherwise it matches all characters (including isolated +<CR>s and <LF>s). When any Unicode line endings are being +recognized, dot does not match <CR> or <LF> or any of the +other line ending characters. + +
The behaviour of dot with regard to newlines can be changed. If the +`(?s)' option is set, a dot matches any one character, without +exception. If the two-character sequence <CR><LF> is present +in the subject string, it takes two dots to match it. + +
The handling of dot is entirely independent of the handling of circumflex and +dollar, the only relationship being that they both involve newlines. Dot has no +special meaning in a character class. + +
Outside a character class, the escape sequence `\C' matches any +one byte, both in and out of UTF-8 mode. Unlike a dot, it always +matches any line-ending characters. The feature is provided in Perl in +order to match individual bytes in UTF-8 mode. Because it breaks up +UTF-8 characters into individual bytes, what remains in the string may +be a malformed UTF-8 string. For this reason, the `\C' escape +sequence is best avoided. + +
PCRE does not allow `\C' to appear in lookbehind assertions +(described below), because in UTF-8 mode this would make it impossible +to calculate the length of the lookbehind. + +
An opening square bracket introduces a character class, terminated by +a closing square bracket. A closing square bracket on its own is not +special. If a closing square bracket is required as a member of the +class, it should be the first data character in the class (after an +initial circumflex, if present) or escaped with a backslash. + +
A character class matches a single character in the subject. In UTF-8 +mode, the character may occupy more than one byte. A matched character +must be in the set of characters defined by the class, unless the +first character in the class definition is a circumflex, in which case +the subject character must not be in the set defined by the class. If +a circumflex is actually required as a member of the class, ensure it +is not the first character, or escape it with a backslash. + +
For example, the character class `[aeiou]' matches any lower case +vowel, while `[^aeiou]' matches any character that is not a lower +case vowel. Note that a circumflex is just a convenient notation for +specifying the characters that are in the class by enumerating those +that are not. A class that starts with a circumflex is not an +assertion: it still consumes a character from the subject string, and +therefore it fails if the current pointer is at the end of the string. + +
In UTF-8 mode, characters with values greater than 255 can be included +in a class as a literal string of bytes, or by using the `\x{' +escaping mechanism. + +
When caseless matching is set, any letters in a class represent both +their upper case and lower case versions, so for example, a caseless +`[aeiou]' matches `A' as well as `a', and a caseless [^aeiou] +does not match `A', whereas a caseful version would. In UTF-8 mode, +PCRE always understands the concept of case for characters whose +values are less than 128, so caseless matching is always possible. For +characters with higher values, the concept of case is supported if +PCRE is compiled with Unicode property support, but not otherwise. If +you want to use caseless matching for characters 128 and above, you +must ensure that PCRE is compiled with Unicode property support as +well as with UTF-8 support. + +
Characters that might indicate line breaks are never treated in any +special way when matching character classes, whatever line-ending +sequence is in use, and whatever setting of the `(?s)' and +`(?m)' options is used. A class such as `[^a]' always +matches one of these characters. + +
The minus (hyphen) character can be used to specify a range of +characters in a character class. For example, `[d-m]' matches any +letter between `d' and `m', inclusive. If a minus character +is required in a class, it must be escaped with a backslash or appear +in a position where it cannot be interpreted as indicating a range, +typically as the first or last character in the class. + +
It is not possible to have the literal character `]' as the end +character of a range. A pattern such as `[W-]46]' is interpreted +as a class of two characters (`W' and `-') followed by a +literal string `46]', so it would match `W46]' or +`-46]'. However, if the `]' is escaped with a backslash it +is interpreted as the end of range, so `[W-\]46]' is interpreted +as a class containing a range followed by two other characters. The +octal or hexadecimal representation of `]' can also be used to +end a range. + +
Ranges operate in the collating sequence of character values. They can +also be used for characters specified numerically, for example +`[\000-\037]'. In UTF-8 mode, ranges can include characters whose +values are greater than 255, for example `[\x{100}-\x{2ff}]'. + +
If a range that includes letters is used when caseless matching is +set, it matches the letters in either case. For example, `[W-c]' +is equivalent to `[][\\^_`wxyzabc]', matched caselessly. + +
The character types `\d', `\D', `\p', `\P', +`\s', `\S', `\w', and `\W' may also appear in a +character class, and add the characters that they match to the +class. For example, `[\dABCDEF]' matches any hexadecimal digit. A +circumflex can conveniently be used with the upper case character +types to specify a more restricted set of characters than the matching +lower case type. For example, the class `[^\W_]' matches any +letter or digit, but not underscore. + +
The only metacharacters that are recognized in character classes are +backslash, hyphen (only where it can be interpreted as specifying a +range), circumflex (only at the start), opening square bracket (only +when it can be interpreted as introducing a POSIX class name—see the +next section), and the terminating closing square bracket. However, +escaping other non-alphanumeric characters does no harm. + +
Perl supports the POSIX notation for character classes. This uses +names enclosed by `[:' and `:]' within the enclosing square +brackets. PCRE also supports this notation. For example, + +
+ [01[:alpha:]%] ++ +
matches `0', `1', any alphabetic character, or `%'. The +supported class names are + +
alnumalphaasciiblankcntrldigitgraphlowerprintpunctspaceupperwordxdigitThe “space” characters are <HT> (9), <LF> (10), <VT> +(11), <FF> (12), <CR> (13), and space (32). Notice that this +list includes the <VT> character (code 11). This makes "space" +different to `\s', which does not include <VT> (for Perl +compatibility). + +
The name “word” is a Perl extension, and “blank” is a GNU +extension from Perl 5.8. Another Perl extension is negation, which is +indicated by a `^' character after the colon. For example, + +
+ [12[:^digit:]] ++ +
matches `1', `2', or any non-digit. PCRE (and Perl) also +recognize the POSIX syntax `[.ch.]' and `[=ch=]' +where ch is a “collating element,” but these are not +supported, and an error is given if they are encountered. + +
In UTF-8 mode, characters with values greater than 128 do not match +any of the POSIX character classes. + +
Vertical bar characters are used to separate alternative patterns. For +example, the pattern + +
+ gilbert|sullivan ++ +
matches either `gilbert' or `sullivan'. Any number of +alternatives may appear, and an empty alternative is permitted +(matching the empty string). The matching process tries each +alternative in turn, from left to right, and the first one that +succeeds is used. If the alternatives are within a subpattern (defined +below), "succeeds" means matching the rest of the main pattern as well +as the alternative in the subpattern. + +
The behavior of the matching engine can be adjusted from within the +pattern by a sequence of option letters enclosed between `(?' and +`)'. The option letters are + +
imsxJUXFor example, `(?im)' sets caseless, multiline matching. It is +also possible to unset these options by preceding the letters with a +hyphen, and a combined setting and unsetting such as `(?im-sx)' +is also permitted. (This would set the caseless and multiline options +while unsetting the dotall and extended-syntax options.) If a letter +appears both before and after the hyphen, the option is unset. The +lowercase option letters are Perl-compatible; the uppercase ones are +PCRE only. + +
When an option change occurs at top level (that is, not inside +subpattern parentheses), the change applies to the remainder of the +pattern that follows. An option change within a subpattern (see below +for a description of subpatterns) affects only that part of the +current pattern that follows it, so + +
+ (a(?i)b)c ++ +
matches `abc' and `aBc' and no other strings. By this +means, options can be made to have different settings in different +parts of the pattern. Any changes made in one alternative do carry on +into subsequent branches within the same subpattern. For example, + +
+ (a(?i)b|c) ++ +
matches `ab', `aB', `c', and `C', even though when +matching `C' the first branch is abandoned before the option +setting. This is because the effects of option settings happen when +the pattern is parsed. There would be some very weird behaviour +otherwise. + +
Note: Unlike these options, the similar, PCRE-specific option +sequences that start with `(*' may appear only at the very +beginning of the pattern. Details of these sequences are given in the +section entitled “Newline sequences,” above. + +
Subpatterns are delimited by parentheses (round brackets), which can +be nested. Turning part of a pattern into a subpattern does two +things: + +
+ cat(aract|erpillar|) ++ +
matches one of the words `cat', `cataract', or +`caterpillar'. Without the parentheses, it would match +`cataract', `erpillar' or an empty string. + +
regex.search. Opening parentheses are counted from
+left to right (starting from 1) to obtain numbers for the capturing
+subpatterns.
+
+ For example, if the string `the red king' is matched against the pattern + +
+ the ((red|white) (king|queen)) ++ +
the captured substrings are `red king', `red', and +`king', and are numbered 1, 2, and 3, respectively. +
The fact that plain parentheses fulfil two functions is not always +helpful. There are often times when a grouping subpattern is required +without a capturing requirement. If an opening parenthesis is followed +by a question mark and a colon, the subpattern does not do any +capturing, and is not counted when computing the number of any +subsequent capturing subpatterns. For example, if the string `the +white queen' is matched against the pattern + +
+ the ((?:red|white) (king|queen)) ++ +
the captured substrings are `white queen' and `queen', and +are numbered 1 and 2. The maximum number of capturing subpatterns is +65535. + +
As a convenient shorthand, if any option settings are required at the +start of a non-capturing subpattern, the option letters may appear +between the `?' and the `:'. Thus the two patterns + +
+ (?i:saturday|sunday) + (?:(?i)saturday|sunday) ++ +
match exactly the same set of strings. Because alternative branches +are tried from left to right, and options are not reset until the end +of the subpattern is reached, an option setting in one branch does +affect subsequent branches, so the above patterns match `SUNDAY' +as well as `Saturday'. + +
Perl 5.10 introduced a feature whereby each alternative in a +subpattern uses the same numbers for its capturing parentheses. Such a +subpattern starts with `(?|' and is itself a non-capturing +subpattern. For example, consider this pattern: + +
+ (?|(Sat)ur|(Sun))day ++ +
Because the two alternatives are inside a `(?|' group, both sets +of capturing parentheses are numbered one. Thus, when the pattern +matches, you can look at captured substring number one, whichever +alternative matched. This construct is useful when you want to capture +part, but not all, of one of a number of alternatives. Inside a +`(?|' group, parentheses are numbered as usual, but the number is +reset at the start of each branch. The numbers of any capturing +buffers that follow the subpattern start after the highest number used +in any branch. The following example is taken from the Perl +documentation. The numbers underneath show in which buffer the +captured content will be stored. + +
+ # before ---------------branch-reset----------- after + / ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x + # 1 2 2 3 2 3 4 ++ +
A backreference or a recursive call to a numbered subpattern always +refers to the first one in the pattern with the given number. + +
An alternative approach to using this “branch reset” feature is to +use duplicate named subpatterns, as described in the next section. + +
Identifying capturing parentheses by number is simple, but it can be +very hard to keep track of the numbers in complicated regular +expressions. Furthermore, if an expression is modified, the numbers +may change. To help with this difficulty, PCRE supports the naming of +subpatterns. This feature was not added to Perl until release +5.10. Python had the feature earlier, and PCRE introduced it at +release 4.0, using the Python syntax. PCRE now supports both the Perl +and the Python syntax. + +
In PCRE, a subpattern can be named in one of three ways: +`(?<name>...)' or `(?'name'...)' as in Perl, or +`(?P<name>...)' as in Python. References to capturing +parentheses from other parts of the pattern, such as backreferences, +recursion, and conditions, can be made by name as well as by number. + +
Names consist of up to 32 alphanumeric characters and +underscores. Named capturing parentheses are still allocated numbers +as well as names, exactly as if the names were not present. + +
By default, a name must be unique within a pattern, but it is possible +to relax this constraint by setting the `(?J)' option. This can +be useful for patterns where only one instance of the named +parentheses can match. Suppose you want to match the name of a +weekday, either as a 3-letter abbreviation or as the full name, and in +both cases you want to extract the abbreviation. This pattern +(ignoring the line breaks) does the job: + +
+ (?Jx) + (?<DN>Mon|Fri|Sun)(?:day)?| + (?<DN>Tue)(?:sday)?| + (?<DN>Wed)(?:nesday)?| + (?<DN>Thu)(?:rsday)?| + (?<DN>Sat)(?:urday)? ++ +
There are five capturing substrings, but only one is ever set after a +match. (An alternative way of solving this problem is to use a +“branch reset” subpattern, as described in the previous section.) + +
Repetition is specified by quantifiers, which can follow any of +the following items: + +
The general repetition quantifier specifies a minimum and maximum +number of permitted matches, by giving the two numbers in curly +brackets (braces), separated by a comma. The numbers must be less than +65536, and the first must be less than or equal to the second. For +example: + +
+ z{2,4}
+
+
+matches `zz', `zzz', or `zzzz'. A closing brace on its +own is not a special character. If the second number is omitted, but +the comma is present, there is no upper limit; if the second number +and the comma are both omitted, the quantifier specifies an exact +number of required matches. Thus + +
+ [aeiou]{3,}
+
+
+matches at least 3 successive vowels, but may match many more, while + +
+ \d{8}
+
+
+matches exactly 8 digits. An opening curly bracket that appears in a +position where a quantifier is not allowed, or one that does not match +the syntax of a quantifier, is taken as a literal character. For +example, `{,6}' is not a quantifier, but a literal string of four +characters. + +
In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to +individual bytes. Thus, for example, `\x{100}{2}' matches two +UTF-8 characters, each of which is represented by a two-byte +sequence. Similarly, `\X{3}' matches three Unicode extended +sequences, each of which may be several bytes long (and they may be of +different lengths). + +
The quantifier `{0}' is permitted, causing the expression to +behave as if the previous item and the quantifier were not present. + +
For convenience, the three most common quantifiers have +single-character abbreviations: + +
*+?It is possible to construct infinite loops by following a subpattern that can +match no characters with a quantifier that has no upper limit, for example: + +
+ (a?)* ++ +
Earlier versions of Perl and PCRE used to give an error at compile +time for such patterns. However, because there are cases where this +can be useful, such patterns are now accepted, but if any repetition +of the subpattern does in fact match no characters, the loop is +forcibly broken. + +
By default, the quantifiers are greedy, that is, they match as +much as possible (up to the maximum number of permitted times), +without causing the rest of the pattern to fail. The classic example +of where this gives problems is in trying to match comments in C +programs. These appear between `/*' and `*/', and within the +comment, individual `*' and `/' characters may appear. An +attempt to match C comments by applying the pattern + +
+ /\*.*\*/ ++ +
to the string + +
+ /* first comment */ not comment /* second comment */ ++ +
fails, because it matches the entire string owing to the greediness of +the `.*' item. + +
However, if a quantifier is followed by a question mark, it ceases to +be greedy, and instead matches the minimum number of times possible, +so the pattern + +
+ /\*.*?\*/ ++ +
does the right thing with the C comments. The meaning of the various +quantifiers is not otherwise changed, just the preferred number of +matches. Do not confuse this use of question mark with its use as a +quantifier in its own right. Because it has two uses, it can sometimes +appear doubled, as in + +
+ \d??\d ++ +
which matches one digit by preference, but can match two if that is the only +way the rest of the pattern matches. + +
If the `(?U)' option is set (an option that is not available in +Perl), the quantifiers are not greedy by default, but individual ones +can be made greedy by following them with a question mark. In other +words, it inverts the default behaviour. + +
When a parenthesized subpattern is quantified with a minimum repeat count that +is greater than 1 or with a limited maximum, more memory is required for the +compiled pattern, in proportion to the size of the minimum or maximum. + +
If a pattern starts with `.*' or `.{0,}' and the +`(?s)' option is set, thus allowing the dot to match newlines, +the pattern is implicitly anchored, because whatever follows will be +tried against every character position in the subject string, so there +is no point in retrying the overall match at any position after the +first. PCRE normally treats such a pattern as though it were preceded +by `\A'. + +
In cases where it is known that the subject string contains no +newlines, it is worth setting `(?s)' in order to obtain this +optimization, or alternatively using `^' or `\A' to indicate +anchoring explicitly. + +
However, there is one situation where the optimization cannot be +used. When .* is inside capturing parentheses that are the subject of +a backreference elsewhere in the pattern, a match at the start may +fail where a later one succeeds. Consider, for example: + +
+ (.*)abc\1 ++ +
If the subject is `xyz123abc123' the match point is the fourth +character. For this reason, such a pattern is not implicitly anchored. + +
When a capturing subpattern is repeated, the value captured is the +substring that matched the final iteration. For example, after + +
+ (tweedle[dume]{3}\s*)+
+
+
+has matched `tweedledum tweedledee' the value of the captured +substring is `tweedledee'. However, if there are nested capturing +subpatterns, the corresponding captured values may have been set in +previous iterations. For example, after + +
+ (a|(b))+ ++ +
matches `aba' the value of the second captured substring is `b'. + +
With both maximizing (greedy) and minimizing (ungreedy or +lazy) repetition, failure of what follows normally causes the +repeated item to be re-evaluated to see if a different number of +repeats allows the rest of the pattern to match. Sometimes it is +useful to prevent this, either to change the nature of the match, or +to cause it fail earlier than it otherwise might, when the author of +the pattern knows there is no point in carrying on. + +
Consider, for example, the pattern `\d+foo' when applied to the +subject line + +
+ 123456bar ++ +
After matching all 6 digits and then failing to match `foo', the +normal action of the matcher is to try again with only 5 digits +matching the `\d+' item, and then with 4, and so on, before +ultimately failing. Atomic grouping (a term taken from Jeffrey +Friedl's book) provides the means for specifying that once a +subpattern has matched, it is not to be re-evaluated in this way. + +
If we use atomic grouping for the previous example, the matcher gives +up immediately on failing to match `foo' the first time. The +notation is a kind of special parenthesis, starting with `(?>' as in +this example: + +
+ (?>\d+)foo ++ +
This kind of parenthesis “locks up” the part of the pattern it +contains once it has matched, and a failure further into the pattern +is prevented from backtracking into it. Backtracking past it to +previous items, however, works as normal. Atomic grouping subpatterns +are not capturing subpatterns. + +
An alternative description is that a subpattern of this type matches +the string of characters that an identical standalone pattern would +match, if anchored at the current point in the subject string. + +
Simple cases such as the above example can be thought of as a +maximizing repeat that must swallow everything it can. So, while both +`\d+' and `\d+?' are prepared to adjust the number of digits +they match in order to make the rest of the pattern match, +`(?>\d+)' can only match an entire sequence of digits. + +
Atomic groups in general can of course contain arbitrarily complicated +subpatterns, and can be nested. However, when the subpattern for an +atomic group is just a single repeated item, as in the example above, +a simpler notation, called a possessive quantifier, can be +used. This consists of an additional `+' character following a +quantifier. Using this notation, the previous example can be rewritten +as + +
+ \d++foo ++ +
Note that a possessive quantifier can be used with an entire group, for +example: + +
+ (abc|xyz){2,3}+
+
+
+Possessive quantifiers are always greedy; the setting of the +`(?U)' option is ignored. They are a convenient notation for the +simpler forms of atomic group. However, there is no difference in the +meaning of a possessive quantifier and the equivalent atomic group, +though there may be a performance difference; possessive quantifiers +should be slightly faster. + +
The possessive quantifier syntax is an extension to the Perl 5.8 +syntax. Jeffrey Friedl originated the idea (and the name) in the +first edition of his book. Mike McCloskey liked it, so implemented it +when he built Sun's Java package, and PCRE copied it from there. It +ultimately found its way into Perl at release 5.10. + +
PCRE has an optimization that automatically “possessifies” certain +simple pattern constructs. For example, the sequence `A+B' is +treated as `A++B' because there is no point in backtracking into +a sequence of `A's when `B' must follow. + +
When a pattern contains an unlimited repeat inside a subpattern that +can itself be repeated an unlimited number of times, the use of an +atomic group is the only way to avoid some failing matches taking a +very long time indeed. The pattern + +
+ (\D+|<\d+>)*[!?] ++ +
matches an unlimited number of substrings that either consist of +non-digits, or digits enclosed in `<>', followed by either +`!' or `?'. When it matches, it runs quickly. However, if it +is applied to + +
+ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ++ +
it takes a long time before reporting failure. This is because the +string can be divided between the internal `\D+' repeat and the +external `*' repeat in a large number of ways, and all have to be +tried. (The example uses `[!?]' rather than a single character at +the end, because both PCRE and Perl have an optimization that allows +for fast failure when a single character is used. They remember the +last single character that is required for a match, and fail early if +it is not present in the string.) If the pattern is changed so that it +uses an atomic group, like this: + +
+ ((?>\D+)|<\d+>)*[!?] ++ +
sequences of non-digits cannot be broken, and failure happens quickly. + +
Outside a character class, a backslash followed by a digit greater +than 0 (and possibly further digits) is a back reference to a +capturing subpattern earlier (that is, to its left) in the pattern, +provided there have been that many previous capturing left +parentheses. + +
However, if the decimal number following the backslash is less than +10, it is always taken as a back reference, and causes an error only +if there are not that many capturing left parentheses in the entire +pattern. In other words, the parentheses that are referenced need not +be to the left of the reference for numbers less than 10. A “forward +back reference” of this type can make sense when a repetition is +involved and the subpattern to the right has participated in an +earlier iteration. + +
It is not possible to have a numerical “forward back reference” to a +subpattern whose number is 10 or more using this syntax because a +sequence such as `\50' is interpreted as a character defined in +octal. See the subsection entitled “Non-printing characters” above +for further details of the handling of digits following a +backslash. There is no such problem when named parentheses are used. A +back reference to any subpattern is possible using named parentheses +(see below). + +
Another way of avoiding the ambiguity inherent in the use of digits +following a backslash is to use the `\g' escape sequence, which +is a feature introduced in Perl 5.10. This escape must be followed by +an unsigned number or a negative number, optionally enclosed in +braces. These examples are all identical: + +
+ (ring), \1
+ (ring), \g1
+ (ring), \g{1}
+
+
+ An unsigned number specifies an absolute reference without the +ambiguity that is present in the older syntax. It is also useful when +literal digits follow the reference. A negative number is a relative +reference. Consider this example: + +
+ (abc(def)ghi)\g{-1}
+
+
+The sequence `\g{-1}' is a reference to the most recently +started capturing subpattern before `\g', that is, is it +equivalent to `\2'. Similarly, `\g{-2}' would be +equivalent to `\1'. The use of relative references can be helpful +in long patterns, and also in patterns that are created by joining +together fragments that contain references within themselves. + +
A back reference matches whatever actually matched the capturing +subpattern in the current subject string, rather than anything +matching the subpattern itself (see “Subpatterns as subroutines” below +for a way of doing that). So the pattern + +
+ (sens|respons)e and \1ibility ++ +
matches `sense and sensibility' and `response and +responsibility', but not `sense and responsibility'. If caseful +matching is in force at the time of the back reference, the case of +letters is relevant. For example, + +
+ ((?i)rah)\s+\1 ++ +
matches `rah rah' and `RAH RAH', but not `RAH rah', +even though the original capturing subpattern is matched caselessly. + +
There are several different ways of writing back references to named +subpatterns. The .NET syntax `\k{name}' and the Perl syntax +`\k<name>' or `\k'name'' are supported, as is the Python +syntax (?P=name). Perl 5.10's unified back reference syntax, in which +`\g' can be used for both numeric and named references, is also +supported. We could rewrite the above example in any of the following +ways: + +
+ (?<p1>(?i)rah)\s+\k<p1>
+ (?'p1'(?i)rah)\s+\k{p1}
+ (?P<p1>(?i)rah)\s+(?P=p1)
+ (?<p1>(?i)rah)\s+\g{p1}
+
+
+A subpattern that is referenced by name may appear in the pattern +before or after the reference. + +
There may be more than one back reference to the same subpattern. If a +subpattern has not actually been used in a particular match, any back +references to it always fail. For example, the pattern + +
+ (a|(bc))\2 ++ +
always fails if it starts to match `a' rather than +`bc'. Because there may be many capturing parentheses in a +pattern, all digits following the backslash are taken as part of a +potential back reference number. If the pattern continues with a digit +character, some delimiter must be used to terminate the back +reference. If the `(?x)' option is set, this can be whitespace. +Otherwise an empty comment (see “Comments” below) can be used. + +
A back reference that occurs inside the parentheses to which it refers +fails when the subpattern is first used, so, for example, `(a\1)' +never matches. However, such references can be useful inside repeated +subpatterns. For example, the pattern + +
+ (a|b\1)+ ++ +
matches any number of `a's and also `aba', `ababbaa' +etc. At each iteration of the subpattern, the back reference matches +the character string corresponding to the previous iteration. In order +for this to work, the pattern must be such that the first iteration +does not need to match the back reference. This can be done using +alternation, as in the example above, or by a quantifier with a +minimum of zero. + +
An assertion is a test on the characters following or preceding the +current matching point that does not actually consume any +characters. The simple assertions coded as `\b', `\B', +`\A', `\G', `\Z', `\z', `^' and `$' are +described above. + +
More complicated assertions are coded as subpatterns. There are two +kinds: those that look ahead of the current position in the subject +string, and those that look behind it. An assertion subpattern is +matched in the normal way, except that it does not cause the current +matching position to be changed. + +
Assertion subpatterns are not capturing subpatterns, and may not be +repeated, because it makes no sense to assert the same thing several +times. If any kind of assertion contains capturing subpatterns within +it, these are counted for the purposes of numbering the capturing +subpatterns in the whole pattern. However, substring capturing is +carried out only for positive assertions, because it does not make +sense for negative assertions. + +
Lookahead assertions start with `(?=' for positive assertions and +`(?!' for negative assertions. For example, + +
+ \w+(?=;) ++ +
matches a word followed by a semicolon, but does not include the semicolon in +the match, and + +
+ foo(?!bar) ++ +
matches any occurrence of `foo' that is not followed by +`bar'. Note that the apparently similar pattern + +
+ (?!foo)bar ++ +
does not find an occurrence of `bar' that is preceded by +something other than `foo'; it finds any occurrence of `bar' +whatsoever, because the assertion `(?!foo)' is always true when +the next three characters are `bar'. A lookbehind assertion is +needed to achieve the other effect. + +
If you want to force a matching failure at some point in a pattern, +the most convenient way to do it is with `(?!)' because an empty +string always matches, so an assertion that requires there not to be +an empty string must always fail. + +
Lookbehind assertions start with `(?<=' for positive assertions +and `(?<!' for negative assertions. For example, + +
+ (?<!foo)bar ++ +
matches an occurrence of `bar' that is not preceded by +`foo'. The contents of a lookbehind assertion are restricted such +that all the strings it matches must have a fixed length. However, if +there are several top-level alternatives, they do not all have to have +the same fixed length. Thus + +
+ (?<=bullock|donkey) ++ +
is permitted, but + +
+ (?<!dogs?|cats?) ++ +
causes an error at compile time. Branches that match different length +strings are permitted only at the top level of a lookbehind +assertion. This is an extension compared with Perl (at least for 5.8), +which requires all branches to match the same length of string. An +assertion such as + +
+ (?<=ab(c|de)) ++ +
is not permitted, because its single top-level branch can match two different +lengths, but it is acceptable if rewritten to use two top-level branches: + +
+ (?<=abc|abde) ++ +
In some cases, the Perl 5.10 escape sequence `\K' (see above) can +be used instead of a lookbehind assertion; this is not restricted to a +fixed-length. + +
The implementation of lookbehind assertions is, for each alternative, +to temporarily move the current position back by the fixed length and +then try to match. If there are insufficient characters before the +current position, the assertion fails. + +
PCRE does not allow the `\C' escape (which matches a single byte +in UTF-8 mode) to appear in lookbehind assertions, because it makes it +impossible to calculate the length of the lookbehind. The `\X' +and `\R' escapes, which can match different numbers of bytes, are +also not permitted. + +
Possessive quantifiers can be used in conjunction with lookbehind +assertions to specify efficient matching at the end of the subject +string. Consider a simple pattern such as + +
+ abcd$ ++ +
when applied to a long string that does not match. Because matching +proceeds from left to right, PCRE will look for each `a' in the +subject and then see if what follows matches the rest of the +pattern. If the pattern is specified as + +
+ ^.*abcd$ ++ +
the initial `.*' matches the entire string at first, but when this fails +(because there is no following `a'), it backtracks to match all +but the last character, then all but the last two characters, and so +on. Once again the search for `a' covers the entire string, from +right to left, so we are no better off. However, if the pattern is +written as + +
+ ^.*+(?<=abcd) ++ +
there can be no backtracking for the `.*+' item; it can match +only the entire string. The subsequent lookbehind assertion does a +single test on the last four characters. If it fails, the match fails +immediately. For long strings, this approach makes a significant +difference to the processing time. + +
Several assertions (of any sort) may occur in succession. For example, + +
+ (?<=\d{3})(?<!999)foo
+
+
+matches `foo' preceded by three digits that are not +`999'. Notice that each of the assertions is applied +independently at the same point in the subject string. First there is +a check that the previous three characters are all digits, and then +there is a check that the same three characters are not `999'. +This pattern does not match `foo' preceded by six +characters, the first of which are digits and the last three of which +are not `999'. For example, it doesn't match `123abcfoo'. A +pattern to do that is + +
+ (?<=\d{3}...)(?<!999)foo
+
+
+This time the first assertion looks at the preceding six characters, +checking that the first three are digits, and then the second +assertion checks that the preceding three characters are not +`999'. + +
Assertions can be nested in any combination. For example, + +
+ (?<=(?<!foo)bar)baz ++ +
matches an occurrence of `baz' that is preceded by `bar' +which in turn is not preceded by `foo', while + +
+ (?<=\d{3}(?!999)...)foo
+
+
+is another pattern that matches `foo' preceded by three digits +and any three characters that are not `999'. + +
It is possible to cause the matching process to obey a subpattern +conditionally or to choose between two alternative subpatterns, +depending on the result of an assertion, or whether a previous +capturing subpattern matched or not. The two possible forms of +conditional subpattern are + +
If the condition is satisfied, the yes-pattern is used; +otherwise the no-pattern (if present) is used. If there are more +than two alternatives in the subpattern, a compile-time error occurs. + +
There are four kinds of condition: references to subpatterns, +references to recursion, a pseudo-condition called `DEFINE', and +assertions. + +
If the text between the parentheses consists of a sequence of digits, +the condition is true if the capturing subpattern of that number has +previously matched. An alternative notation is to precede the digits +with a plus or minus sign. In this case, the subpattern number is +relative rather than absolute. The most recently opened parentheses +can be referenced by `(?(-1)', the next most recent by +`(?(-2)', and so on. In looping constructs it can also make sense +to refer to subsequent groups with constructs such as `(?(+2)'. + +
Consider the following pattern, which contains non-significant white +space to make it more readable and to divide it into three parts for +ease of discussion (assume a preceding `(?x)'): + +
+ ( \( )? [^()]+ (?(1) \) ) ++ +
The first part matches an optional opening parenthesis, and if that +character is present, sets it as the first captured substring. The +second part matches one or more characters that are not +parentheses. The third part is a conditional subpattern that tests +whether the first set of parentheses matched or not. If they did, that +is, if subject started with an opening parenthesis, the condition is +true, and so the yes-pattern is executed and a closing parenthesis is +required. Otherwise, since no-pattern is not present, the subpattern +matches nothing. In other words, this pattern matches a sequence of +non-parentheses, optionally enclosed in parentheses. + +
If you were embedding this pattern in a larger one, you could use a +relative reference: + +
+ ...other stuff... ( \( )? [^()]+ (?(-1) \) ) ... ++ +
This makes the fragment independent of the parentheses in the larger pattern. + +
Perl uses the syntax `(?(<name>)...)' or `(?('name')...)' to +test for a used subpattern by name. For compatibility with earlier +versions of PCRE, which had this facility before Perl, the syntax +`(?(name)...)' is also recognized. However, there is a possible +ambiguity with this syntax, because subpattern names may consist +entirely of digits. PCRE looks first for a named subpattern; if it +cannot find one and the name consists entirely of digits, PCRE looks +for a subpattern of that number, which must be greater than +zero. Using subpattern names that consist entirely of digits is not +recommended. + +
Rewriting the above example to use a named subpattern gives this: + +
+ (?<OPEN> \( )? [^()]+ (?(<OPEN>) \) ) ++ +
If the condition is the string `(R)', and there is no subpattern +with the name `R', the condition is true if a recursive call to +the whole pattern or any subpattern has been made. If digits or a name +preceded by ampersand follow the letter `R', for example: + +
+ (?(R3)...) or (?(R&name)...) ++ +
the condition is true if the most recent recursion is into the +subpattern whose number or name is given. This condition does not +check the entire recursion stack. + +
At “top level,” all these recursion test conditions are false. Recursive +patterns are described below. + +
If the condition is the string `(DEFINE)', and there is no +subpattern with the name `DEFINE', the condition is always +false. In this case, there may be only one alternative in the +subpattern. It is always skipped if control reaches this point in the +pattern; the idea of DEFINE is that it can be used to define +subroutines that can be referenced from elsewhere. (The use of +subroutines is described below.) For example, a pattern to match an +IPv4 address could be written like this (ignore whitespace and line +breaks): + +
+ (?(DEFINE) (?<byte> 2[0-4]\d | 25[0-5] | 1\d\d | [1-9]?\d) )
+ \b (?&byte) (\.(?&byte)){3} \b
+
+
+ The first part of the pattern is a DEFINE group inside which a another +group named "byte" is defined. This matches an individual component of +an IPv4 address (a number less than 256). When matching takes place, +this part of the pattern is skipped because DEFINE acts like a false +condition. + +
The rest of the pattern uses references to the named group to match +the four dot-separated components of an IPv4 address, insisting on a +word boundary at each end. + +
If the condition is not in any of the above formats, it must be an +assertion. This may be a positive or negative lookahead or lookbehind +assertion. Consider this pattern, again containing non-significant +white space, and with the two alternatives on the second line: + +
+ (?(?=[^a-z]*[a-z])
+ \d{2}-[a-z]{3}-\d{2} | \d{2}-\d{2}-\d{2} )
+
+
+ The condition is a positive lookahead assertion that matches an +optional sequence of non-letters followed by a letter. In other words, +it tests for the presence of at least one letter in the subject. If a +letter is found, the subject is matched against the first alternative; +otherwise it is matched against the second. This pattern matches +strings in one of the two forms `dd-aaa-dd' or +`dd-dd-dd', where aaa are letters and +dd are digits. + +
The sequence `(?#' marks the start of a comment that continues up +to the next closing parenthesis. Nested parentheses are not +permitted. The characters that make up a comment play no part in the +pattern matching at all. + +
If the `(?x)' option is set, an unescaped `#' character +outside a character class introduces a comment that continues to +immediately after the next newline in the pattern. + +
Consider the problem of matching a string in parentheses, allowing for +unlimited nested parentheses. Without the use of recursion, the best +that can be done is to use a pattern that matches up to some fixed +depth of nesting. It is not possible to handle an arbitrary nesting +depth. + +
PCRE supports special syntax for recursion of the entire pattern, and +also for individual subpattern recursion. After its introduction in +PCRE and Python, this kind of recursion was introduced into Perl at +release 5.10. + +
A special item that consists of `(?' followed by a number greater +than zero and a closing parenthesis is a recursive call of the +subpattern of the given number, provided that it occurs inside that +subpattern. (If not, it is a subroutine call, which is described in +the next section.) The special item `(?R)' or `(?0)' is a +recursive call of the entire regular expression. + +
In PCRE (like Python, but unlike Perl), a recursive subpattern call is +always treated as an atomic group. That is, once it has matched some +of the subject string, it is never re-entered, even if it contains +untried alternatives and there is a subsequent matching failure. + +
This PCRE pattern solves the nested parentheses problem (whitespace is +insignificant): + +
+ \( ( (?>[^()]+) | (?R) )* \) ++ +
First it matches an opening parenthesis. Then it matches any number of +substrings which can either be a sequence of non-parentheses, or a +recursive match of the pattern itself (that is, a correctly +parenthesized substring). Finally there is a closing parenthesis. + +
If this were part of a larger pattern, you would not want to recurse +the entire pattern, so instead you could use this: + +
+ ( \( ( (?>[^()]+) | (?1) )* \) ) ++ +
We have put the pattern into parentheses, and caused the recursion to +refer to them instead of the whole pattern. + +
In a larger pattern, keeping track of parenthesis numbers can be +tricky. This is made easier by the use of relative references. (A Perl +5.10 feature.) Instead of `(?1)' in the pattern above you can +write `(?-2)' to refer to the second most recently opened +parentheses preceding the recursion. In other words, a negative number +counts capturing parentheses leftwards from the point at which it is +encountered. + +
It is also possible to refer to subsequently opened parentheses, by +writing references such as `(?+2)'. However, these cannot be +recursive because the reference is not inside the parentheses that are +referenced. They are always subroutine calls, as described in the next +section. + +
An alternative approach is to use named parentheses instead. The Perl +syntax for this is `(?&name)'; PCRE's earlier syntax +`(?P>name)' is also supported. We could rewrite the above example +as follows: + +
+ (?<pn> \( ( (?>[^()]+) | (?&pn) )* \) ) ++ +
If there is more than one subpattern with the same name, the earliest +one is used. + +
This particular example pattern that we have been looking at contains +nested unlimited repeats, and so the use of atomic grouping for +matching strings of non-parentheses is important when applying the +pattern to strings that do not match. For example, when this pattern +is applied to + +
+ (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() ++ +
it fails quickly. However, if atomic grouping is not used, the match +runs for a very long time indeed because there are so many different +ways the `+' and `*' repeats can carve up the subject, and +all have to be tested before failure can be reported. + +
At the end of a match, the values set for any capturing subpatterns +are those from the outermost level of the recursion at which the +subpattern value is set. If the pattern above is matched against + +
+ (ab(cd)ef) ++ +
the value for the capturing parentheses is `ef', which is the +last value taken on at the top level. If additional parentheses are +added, giving + +
+ \( ( ( (?>[^()]+) | (?R) )* ) \)
+ ^ ^
address@hidden example
+
address@hidden
+the string they capture is @samp{ab(cd)ef}, the contents of the top
+level parentheses.
+
+Do not confuse the @samp{(?R)} item with the condition @samp{(?(R)},
+which tests for recursion. Consider this pattern, which matches text
+in angle brackets, allowing for arbitrary nesting. Only digits are
+allowed in nested brackets (that is, when recursing), whereas any
+characters are permitted at the outer level.
+
address@hidden
+ < (?: (?(R) \d++ | [^<>]*+) | (?R)) * >
+
+
+In this pattern, `(?(R)' is the start of a conditional +subpattern, with two different alternatives for the recursive and +non-recursive cases. The `(?R)' item is the actual recursive +call. + +
If the syntax for a recursive subpattern reference (either by number +or by name) is used outside the parentheses to which it refers, it +operates like a subroutine in a programming language. The called +subpattern may be defined before or after the reference. A numbered +reference can be absolute or relative, as in these examples: + +
+ (...(absolute)...)...(?2)... + (...(relative)...)...(?-1)... + (...(?+1)...(relative)... ++ +
An earlier example pointed out that the pattern + +
+ (sens|respons)e and \1ibility ++ +
matches `sense and sensibility' and `response and +responsibility', but not `sense and responsibility'. If instead +the pattern + +
+ (sens|respons)e and (?1)ibility ++ +
is used, it does match `sense and responsibility' as well as the +other two strings. Another example is given in the discussion of +DEFINE above. + +
Like recursive subpatterns, a subroutine call is always treated as an +atomic group. That is, once it has matched some of the subject string, +it is never re-entered, even if it contains untried alternatives and +there is a subsequent matching failure. + +
When a subpattern is used as a subroutine, processing options such as +case-independence are fixed when the subpattern is defined. They +cannot be changed for different calls. For example, consider this +pattern: + +
+ (abc)(?i:(?-1)) ++ +
It matches `abcabc'. It does not match `abcABC' because the +change of processing option does not affect the called subpattern. + +
Perl 5.10 introduced a number of special backtracking control +verbs, which are described in the Perl documentation as +“experimental and subject to change or removal in a future version of +Perl.” It goes on to say: “Their usage in production code should be +noted to avoid problems during upgrades.” The same remarks apply to +the PCRE features described in this section. + +
The new verbs make use of what was previously invalid syntax: an +opening parenthesis followed by an asterisk. In Perl, they are +generally of the form `(*VERB:ARG)' but PCRE does not support the +use of arguments, so its general form is just `(*VERB)'. Any +number of these verbs may occur in a pattern. There are two kinds: + +
The following verbs act as soon as they are encountered: + +
(*ACCEPT)+ A(A|B(*ACCEPT)|C)D ++ +
This matches `AB', `AAD', or `ACD', but when it matches +`AB', no data is captured. + +
(*FAIL) or (*F)The following verbs do nothing when they are encountered. Matching +continues with what follows, but if there is no subsequent match, a +failure is forced. The verbs differ in exactly what kind of failure +occurs. + +
(*COMMIT)(*COMMIT) has been passed, the regular expression engine is
+committed to finding a match at the current starting point, or not at
+all. For example:
+
+ + a+(*COMMIT)b ++ +
This matches `xxaab' but not `aacaab'. It can be thought of +as a kind of dynamic anchor, or “I've started, so I must finish.” + +
(*PRUNE)(*PRUNE), or when matching to the right of (*PRUNE), but
+if there is no match to the right, backtracking cannot cross
+(*PRUNE). In simple cases, the use of (*PRUNE) is just
+an alternative to an atomic group or possessive quantifier, but there
+are some uses of (*PRUNE) that cannot be expressed in any other
+way.
+
+ (*SKIP)(*PRUNE), except that if the pattern is
+unanchored, the "bumpalong" advance is not to the next character, but
+to the position in the subject where (*SKIP) was
+encountered. (*SKIP) signifies that whatever text was matched
+leading up to it cannot be part of a successful match. Consider:
+
+ + a+(*SKIP)b ++ +
If the subject is `aaaac...', after the first match attempt fails +(starting at the first character in the string), the starting point +skips on to start the next attempt at `c'. Note that a possessive +quantifer does not have the same effect in this example; although it +would suppress backtracking during the first match attempt, the second +attempt would start at the second character instead of skipping on to +`c'. + +
(*THEN)+ ( COND1 (*THEN) FOO + | COND2 (*THEN) BAR + | COND3 (*THEN) BAZ ) ... ++ +
If the `COND1' pattern matches, `FOO' is tried (and possibly
+further items after the end of the group if `FOO' succeeds); on
+failure the matcher skips to the second alternative and tries
+`COND2', without backtracking into COND1. If (*THEN) is used
+outside of any alternation, it acts exactly like (*PRUNE).
+
+ +Next: Regexp Details, +Up: Regexps +
This is a quick-reference summary of the regular expression syntax +used in Monotone. + +
\x\Q...\E\a\cx\e\f\n\r\t\ddd\xhh\x{hhh...}.\C\d\D\h\H\p{xx}\P{xx}\R\s\S\v\V\w\W\X`\d', `\D', `\s', `\S', `\w', and `\W' +recognize only ASCII characters. + +
CCcCfCnCoCsLLlLmLoLtLuL&MMcMeMnNNdNlNoPPcPdPePfPiPoPsSScSkSmSoZZlZpZsArabic, +Armenian, +Balinese, +Bengali, +Bopomofo, +Braille, +Buginese, +Buhid, +Canadian_Aboriginal, +Cherokee, +Common, +Coptic, +Cuneiform, +Cypriot, +Cyrillic, +Deseret, +Devanagari, +Ethiopic, +Georgian, +Glagolitic, +Gothic, +Greek, +Gujarati, +Gurmukhi, +Han, +Hangul, +Hanunoo, +Hebrew, +Hiragana, +Inherited, +Kannada, +Katakana, +Kharoshthi, +Khmer, +Lao, +Latin, +Limbu, +Linear_B, +Malayalam, +Mongolian, +Myanmar, +New_Tai_Lue, +Nko, +Ogham, +Old_Italic, +Old_Persian, +Oriya, +Osmanya, +Phags_Pa, +Phoenician, +Runic, +Shavian, +Sinhala, +Syloti_Nagri, +Syriac, +Tagalog, +Tagbanwa, +Tai_Le, +Tamil, +Telugu, +Thaana, +Thai, +Tibetan, +Tifinagh, +Ugaritic, +Yi. + +
[...][^...][x-y][[:xxx:]][[^:xxx:]]alnumalphaasciiblankcntrldigitgraphlowerprintpunctspaceupperwordxdigitIn PCRE, POSIX character set names recognize only ASCII +characters. You can use `\Q...\E' inside a character class. + +
??+??**+*?++++?{n}{n,m}{n,m}+{n,m}?{n,}{n,}+{n,}?\b\B^\A$\Z\z\G\K|expr|expr...(...)(?<name>...)(?'name'...)(?P<name>...)(?:...)(?|...)(?>...)(?#....)(?i)(?J)(?m)(?s)(?U)(?x)(?-...)(?=...)(?!...)(?<=...)(?<!...)Each top-level branch of a look behind must be of a fixed length. + +
\n\gn\g{n}\g{-n}\k<name>\k'name'\g{name}\k{name}(?P=name)(?R)(?n)(?+n)(?-n)(?&name)(?P>name)(?(condition)yes-pattern)(?(condition)yes-pattern|no-pattern)
+(?(n)...(?(+n)...(?(-n)...(?(<name>)...(?('name')...(?(name)...(?(R)...(?(Rn)...(?(R&name)...(?(DEFINE)...(?(assert)...The following act immediately they are reached: + +
(*ACCEPT)(*FAIL)The following act only when a subsequent match failure causes a backtrack to +reach them. They all force a match failure, but they differ in what happens +afterwards. Those that advance the start-of-match point do so only if the +pattern is not anchored. + +
(*COMMIT)(*PRUNE)(*SKIP)(*THEN)These are recognized only at the very start of the pattern or after a +`(*BSR_...)' option. + +
(*CR)(*LF)(*CRLF)(*ANYCRLF)(*ANY)These are recognized only at the very start of the pattern or after a +`(*...)' option that sets the newline convention. + +
(*BSR_ANYCRLF)(*BSR_UNICODE)+ +Previous: Mark-Merge, +Up: Special Topics +
Monotone expects user-provided regular expressions in
+.mtn-ignore files and as the result of the
+get_encloser_pattern Lua hook (for the --show-encloser
+option to diff). User-written Lua hooks may also use the
+function regex.search as they see fit. All these regular
+expressions should be written with the same syntax, which is that
+expected by the Perl-Compatible Regular Expression library (PCRE).
+
+
existonpath(possible_command)alias_command(original, alias)existonpath(possible_command)existonpath would search
for “xxdiff.exe”.
- get_confdir()get_confdir()get_ostype()get_ostype()guess_binary_file_contents(filespec)guess_binary_file_contents(filespec) 0x00 thru 0x06
0x0E thru 0x1a
0x1c thru 0x1f
- include(scriptfile)include(scriptfile)includedir(scriptpath)includedir(scriptpath)includedirpattern(scriptpath, pattern)includedirpattern(scriptpath, pattern)is_executable(filespec)is_executable(filespec)kill(pid [, signal])kill(pid [, signal])make_executable(filespec)make_executable(filespec)match(glob, string)match(glob, string)mkstemp(template)mkstemp(template)temp_file()<
file in the standard TMP/TEMP directories.
For the definition of temp_file(), see Default hooks.
-
parse_basic_io(data)-
+
mtn_automate( ... )-
+The
mtn_automate Lua function calls the Monotone
+automate command passed in its arguments. The result of the
+call is a pair consisting of a boolean return code, indicating whether
+the call was successful or not, and a string being the stdout
+output from the automate command. This function is not for use
+in ordinary Lua hooks, but rather for Lua based commands as defined by
+the Lua function register_command.
+
+
parse_basic_io(data)-
Parse the string data, which should be in basic_io format. It returns nil
if it can't parse the string; otherwise it returns a table. This will be a list
of all statements, with each entry being a table having a "name" element that is
@@ -133,17 +146,23 @@ the arguments.
The output table will be:
{
- 1 = { name = "thingy", args = { 1 = "foo", 2 = "bar" } },
- 2 = { name = "thingy", args = { 1 = "baz" } },
- 3 = { name = "spork", args = { } },
- 4 = { name = "frob", args = { 1 = "oops" } }
+ 1 = { name = "thingy", values = { 1 = "foo", 2 = "bar" } },
+ 2 = { name = "thingy", values = { 1 = "baz" } },
+ 3 = { name = "spork", values = { } },
+ 4 = { name = "frob", values = { 1 = "oops" } }
}
-
regex.search(regexp, string)-
+
regex.search(regexp, string)-
Returns true if a match for regexp is found in str, return
-false otherwise.
+false otherwise. See Regexps, for the syntax of regexp.
-
server_request_sync(what, address, include, exclude)-
+
register_command(name, params, abstract, description, function)-
+Add a command named name to the user command group in monotone. This function is
+normally called directly from a monotonerc file rather than a hook. When the user issues the
+registered command, monotone will call the lua function name supplied. That function would
+then normally use mtn_automate() calls to service the request.
+
+
server_request_sync(what, address, include, exclude)-
Initiate a netsync connection to the server at address, with the
given include and exclude patterns, of type sync, push,
or pull, as given by the what argument.
@@ -151,10 +170,10 @@ command, this function has no effect.
When called by a monotone instance which is not running the serve
command, this function has no effect.
-
sleep(seconds)-
+
sleep(seconds)-
Makes the calling process sleep for the specified number of seconds.
-
spawn(executable [, args ...])-
+
spawn(executable [, args ...])-
Starts the named executable with the given arguments. Returns the
process PID on POSIX systems, the process handle on Win32 or -1 if
there was an error.
@@ -166,17 +185,17 @@ in a standardized way.
option.
execute() builds on spawn() and wait()
in a standardized way.
-
spawn_pipe(executable [, args ...])-
+
spawn_pipe(executable [, args ...])-
Like spawn(), but returns three values, where the first two are the
subprocess' standard input and standard output, and the last is the
process PID on POSIX systems, the process handle on Win32 or -1 if
there was an error.
-
spawn_redirected(infile, outfile, errfile, executable [, args ...])-
+
spawn_redirected(infile, outfile, errfile, executable [, args ...])-
Like spawn(), but with standard input, standard output and standard
error redirected to the given files.
-
wait(pid)-
+
wait(pid)-
Wait until the process with given PID (process handle on Win32) exits.
Returns two values: a result value and the exit code of the waited-for
process.
============================================================
--- docs/Automation.html aeef8408071b1a6a89c6030e08903f40fddfe1c9
+++ docs/Automation.html 356f171884468556f00bb7a1cf101d1a52e8d86c
@@ -40,7 +40,7 @@ messages.
messages.
-- mtn automate interface_version
-
+
- mtn automate interface_version
-
- Arguments:
-
None.
@@ -69,7 +69,7 @@ None.
-
- mtn automate heads [branch]
-
+
- mtn automate heads [branch]
-
- Arguments:
-
One branch name, branch. If none is given, the current default branch is used.
@@ -96,7 +96,7 @@ If the given branch contains no members
-
- mtn automate ancestors rev1 [rev2 [...]]
-
+
- mtn automate ancestors rev1 [rev2 [...]]
-
- Arguments:
-
One or more revision IDs, rev1, rev2, etc.
@@ -129,7 +129,7 @@ an error message to stderr, and exits wi
-
- mtn automate common_ancestors rev1 [rev2 [...]]
-
+
- mtn automate common_ancestors rev1 [rev2 [...]]
-
- Arguments:
-
One or more revision IDs, rev1, rev2, etc.
@@ -162,7 +162,7 @@ an error message to stderr, and exits wi
-
- mtn automate parents rev
-
+
- mtn automate parents rev
-
- Arguments:
-
One revision ID, rev.
@@ -192,7 +192,7 @@ stdout, prints an error message to stder
-
- mtn automate descendents rev1 [rev2 [...]]
-
+
- mtn automate descendents rev1 [rev2 [...]]
-
- Arguments:
-
One or more revision IDs, rev1, rev2, etc.
@@ -225,7 +225,7 @@ an error message to stderr, and exits wi
-
- mtn automate children rev
-
+
- mtn automate children rev
-
- Arguments:
-
One revision ID, rev.
@@ -255,7 +255,7 @@ stdout, prints an error message to stder
-
- mtn automate graph
-
+
- mtn automate graph
-
- Arguments:
-
None.
@@ -292,7 +292,7 @@ None.
-
- mtn automate erase_ancestors [rev1 [rev2 [...]]]
-
+
- mtn automate erase_ancestors [rev1 [rev2 [...]]]
-
- Arguments:
-
One or more revision IDs, rev1, rev2, etc.
@@ -326,7 +326,7 @@ an error message to stderr, and exits wi
-
- mtn automate toposort [rev1 [rev2 [...]]]
-
+
- mtn automate toposort [rev1 [rev2 [...]]]
-
- Arguments:
-
One or more revision IDs, rev1, rev2, etc.
@@ -358,7 +358,7 @@ an error message to stderr, and exits wi
-
- mtn automate ancestry_difference new [old1 [old2 [...]]]
-
+
- mtn automate ancestry_difference new [old1 [old2 [...]]]
-
- Arguments:
-
A “new” revision ID new, followed by zero or more “old”
@@ -395,7 +395,7 @@ an error message to stderr, and exits wi
-
- mtn automate leaves
-
+
- mtn automate leaves
-
- Arguments:
-
None.
@@ -427,7 +427,7 @@ None.
-
- mtn automate roots
-
+
- mtn automate roots
-
- Arguments:
-
None.
@@ -454,7 +454,7 @@ None.
-
- mtn automate branches
-
+
- mtn automate branches
-
- Arguments:
-
None.
@@ -481,7 +481,7 @@ None.
-
- mtn automate tags [branch_pattern]
-
+
- mtn automate tags [branch_pattern]
-
- Arguments:
-
A branch pattern (optional).
@@ -549,7 +549,7 @@ specified.
-
- mtn automate select selector
-
+
- mtn automate select selector
-
- Arguments:
-
One selector (or combined selector).
@@ -577,7 +577,7 @@ None.
-
- mtn automate identify path
-
+
- mtn automate identify path
-
- Arguments:
-
A file path.
@@ -604,227 +604,326 @@ marker.
-
- mtn automate inventory
-
+
- mtn automate inventory [files...]
-
- Arguments:
-
-None.
+One or more file paths (optional). If present, only show inventory for the given
+files or directories (and their sub-directories). You can use --depth
+and --exclude to control what is selected through this restriction.
-
- Added in:
-
-1.0
+
- Changes:
-
+
+- 6.0 – converted to basic_io format (restriction support, various fixes)
+
- 1.0 – initial, line-based format
+
+
- Purpose:
-
Prints the inventory of every file found in the workspace or its
-associated base manifest. Each unique path is listed on a line prefixed by
-three status characters and two numeric values used for identifying
-renames.
+associated base and revision manifests. Each unique path is
+listed in a basic_io stanza. Stanzas are separated by blank lines.
- Sample output:
-
-All basic status codes:
+All basic status cases:
+
+ path "added"
+ new_type "file"
+ fs_type "file"
+ status "added" "known"
+ changes "content"
+
+ path "dropped"
+ old_type "file"
+ fs_type "none"
+ status "dropped"
+
+ path "ignored~"
+ fs_type "file"
+ status "ignored"
+
+ path "missing"
+ old_type "file"
+ new_type "file"
+ fs_type "none"
+ status "missing"
+
+ path "original"
+ old_type "file"
+ new_path "renamed"
+ fs_type "none"
+ status "rename_source"
+
+ path "patched"
+ old_type "file"
+ new_type "file"
+ fs_type "file"
+ status "known"
+ changes "content"
+
+ path "renamed"
+ new_type "file"
+ old_path "original"
+ fs_type "file"
+ status "rename_target" "known"
+
+ path "unchanged"
+ old_type "file"
+ new_type "file"
+ fs_type "file"
+ status "known"
+
+ path "unknown"
+ fs_type "file"
+ status "unknown"
+
-
- M 0 0 missing
- AP 0 0 added
- D 0 0 dropped
- R 1 0 renamed-from-this
- R 0 1 renamed-to-this
- P 0 0 patched
- 0 0 unchanged
- U 0 0 unknown
- I 0 0 ignored
+ Two files swapped in both the revision manifest and the workspace:
+
+ path "original"
+ old_type "file"
+ new_path "unchanged"
+ new_type "file"
+ old_path "unchanged"
+ fs_type "file"
+ status "rename_source" "rename_target" "known"
+
+ path "unchanged"
+ old_type "file"
+ new_path "original"
+ new_type "file"
+ old_path "original"
+ fs_type "file"
+ status "rename_source" "rename_target" "known"
- Two files swapped:
-
-
- RR 1 2 unchanged
- RR 2 1 original
+ Recorded in the revision manifest that two files were swapped, but
+they were not actually swapped in the workspace. Thus they both appear
+as patched:
+
+ path "original"
+ old_type "file"
+ new_path "unchanged"
+ new_type "file"
+ old_path "unchanged"
+ fs_type "file"
+ status "rename_source" "rename_target" "known"
+ changes "content"
+
+ path "unchanged"
+ old_type "file"
+ new_path "original"
+ new_type "file"
+ old_path "original"
+ fs_type "file"
+ status "rename_source" "rename_target" "known"
+ changes "content"
+
- Recorded with monotone that two files were swapped, but they were not
-actually swapped in the filesystem. Thus they both appear as patched:
+
Rename (in the manifest and the workspace) foo to bar;
+add (in the manifest and the workspace) new file foo:
+
+ path "foo"
+ old_type "file"
+ new_path "bar"
+ new_type "file"
+ fs_type "file"
+ status "rename_source" "added" "known"
+
+ path "bar"
+ new_type "file"
+ old_path "foo"
+ fs_type "file"
+ status "rename_target" "known"
+
-
- RRP 1 2 unchanged
- RRP 2 1 original
+ Rotated files foo -> bar -> baz -> foo (in
+the manifest and the workspace):
+
+ path "foo"
+ old_type "file"
+ new_path "bar"
+ new_type "file"
+ old_path "baz"
+ fs_type "file"
+ status "rename_source" "rename_target" "known"
+
+ path "bar"
+ old_type "file"
+ new_path "baz"
+ new_type "file"
+ old_path "foo"
+ fs_type "file"
+ status "rename_source" "rename_target" "known"
+
+ path "baz"
+ old_type "file"
+ new_path "foo"
+ new_type "file"
+ old_path "bar"
+ fs_type "file"
+ status "rename_source" "rename_target" "known"
- Rename foo to bar; add new file foo:
+
Recorded in the revison manifest the rotation of files foo ->
+bar -> baz -> foo, but the actual files in the
+workspace were not moved, so monotone interprets all files as having
+been patched:
+
+ path "foo"
+ old_type "file"
+ new_path "bar"
+ new_type "file"
+ old_path "baz"
+ fs_type "file"
+ status "rename_source" "rename_target" "known"
+ changes "content"
+
+ path "bar"
+ old_type "file"
+ new_path "baz"
+ new_type "file"
+ old_path "foo"
+ fs_type "file"
+ status "rename_source" "rename_target" "known"
+ changes "content"
+
+ path "baz"
+ old_type "file"
+ new_path "foo"
+ new_type "file"
+ old_path "bar"
+ fs_type "file"
+ status "rename_source" "rename_target" "known"
+ changes "content"
+
-
- RAP 1 0 foo
- R 0 1 bar
+ Dropped from the manifest but not removed in the workspace and thus
+unknown:
+
+ path "dropped"
+ old_type "file"
+ fs_type "file"
+ status "dropped" "unknown"
- Rotated files foo -> bar -> baz -> foo:
+
Added in the manifest but not in the workspace, and thus missing:
+
+ path "added"
+ new_type "file"
+ fs_type "none"
+ status "added" "missing"
+
-
- RR 1 3 foo
- RR 2 1 bar
- RR 3 2 baz
+ Recorded a rename in the manifest, but not moved in the workspace,
+and thus unknown source and missing target:
+
+ path "original"
+ old_type "file"
+ new_path "renamed"
+ fs_type "file"
+ status "rename_source" "unknown"
+
+ path "renamed"
+ new_type "file"
+ old_path "original"
+ fs_type "none"
+ status "rename_target" "missing"
- Recorded the rotation of files foo -> bar -> baz ->
-foo, but the actual files in the workspace were not
-moved, so monotone interprets all files as having been patched:
+
Moved in the workspace but no rename recorded in the manifest, and
+thus missing source and unknown target:
+
+ path "original"
+ old_type "file"
+ new_type "file"
+ fs_type "none"
+ status "missing"
+
+ path "renamed"
+ fs_type "file"
+ status "unknown"
+
-
- RRP 1 3 foo
- RRP 2 1 bar
- RRP 3 2 baz
+ Renamed in the manifest and the workspace and patched:
+
+ path "original"
+ old_type "file"
+ new_path "renamed"
+ fs_type "none"
+ status "rename_source"
+
+ path "renamed"
+ new_type "file"
+ old_path "original"
+ fs_type "file"
+ status "rename_target" "known"
+ changes "content"
- Dropped but not removed and thus unknown:
+
- Output format:
-
+Each path is printed in one basic_io stanza. Stanzas are separated by
+a blank line. Each stanza starts with a
path line, and contains
+up to seven lines. The order of the lines is not important, and may
+change in future revisions, except that the first line will always be
+path.
-
- D U 0 0 dropped
-
+
+- 'path'
- the file or directory path, relative to the workspace root. The file
+either exists in the workspace, or is listed in the base or revision
+manifest.
-
Added a non-existent file which is thus missing:
+
- old_type
- gives the type of the file in the base manifest. “type” is
+either file, directory, or none.
+old_type is output only if it is different from the type in
+the revision manifest or workspace.
-
- AM 0 0 added
-
+
- new_type
- the type of the file in the revision manifest. new_type is
+output only if it is different from the type in the base manifest or
+workspace.
-
Recorded a rename, but not moved in the filesystem, and thus unknown
-source and missing target:
+
- fs_type
- the type of the file in the workspace (also called the filesystem).
+fs_type is always output.
-
- R U 1 0 original
- RM 0 1 renamed
-
+
- old_path
- the old path for the file, if it has been renamed in the revision
+manifest.
-
Moved in the filesystem but no rename recorded, and thus missing source
-and unknown target:
+
- new_path
- the new path for the file, if it has been renamed in the revision
+manifest.
-
- M 0 0 original
- U 0 0 renamed
-
+
- status
- status is always output. Its value is one or more of:
-
Renamed and patched:
+
+- rename_source
- the old name of a file that has been renamed.
-
- R 1 0 original
- RP 0 1 renamed
-
+
- rename_target
- the new name of a file that has been renamed.
-
- Output format:
-
-Each path is printed on its own line, prefixed by three status
-characters described below. The status is followed by a single space and
-two numbers, each separated by a single space, used for identifying renames.
-The numbers are followed by a single space and then the pathname, which
-includes the rest of the line. Directory paths are identified as ending with
-the "/" character, file paths do not end in this character.
+
- added
- the file is new in the revision manifest (not in the base
+manifest).
-
The three status characters are as follows.
+
- dropped
- the file is deleted in the revision manifest and the workspace.
-
- column 1 pre-state
- ' ' the path was unchanged in the pre-state
- 'D' the path was deleted from the pre-state
- 'R' the path was renamed from the pre-state name
- column 2 post-state
- ' ' the path was unchanged in the post-state
- 'R' the path was renamed to the post-state name
- 'A' the path was added to the post-state
- column 3 file-state
- ' ' the file is known and unchanged from the current manifest version
- 'P' the file is patched to a new version
- 'U' the file is unknown and not included in the current manifest
- 'I' the file is ignored and not included in the current manifest
- 'M' the file is missing but is included in the current manifest
-
+
- missing
- the file is deleted in the workspace but not the revision manifest.
-
Note that out of the 45 possible status code combinations, only 26 are valid,
-detailed below.
+
- ignored
- the file is ignored by monotone.
-
- ' ' unchanged
- ' P' patched (contents changed)
- ' U' unknown (exists on the filesystem but not tracked)
- ' I' ignored (exists on the filesystem but excluded by Lua hook)
- ' M' missing (exists in the manifest but not on the filesystem)
-
- ' A ' added (invalid, add should have associated patch)
- ' AP' added and patched
- ' AU' added but unknown (invalid)
- ' AI' added but ignored (invalid, added files are no longer ignored)
- ' AM' added but missing from the filesystem
-
- ' R ' rename target
- ' RP' rename target and patched
- ' RU' rename target but unknown (invalid)
- ' RI' rename target but ignored (invalid)
- ' RM' rename target but missing from the filesystem
-
- 'D ' dropped
- 'D P' dropped and patched (invalid)
- 'D U' dropped and unknown (still exists on the filesystem)
- 'D I' dropped and ignored
- 'D M' dropped and missing (invalid)
-
- 'DA ' dropped and added (invalid, add should have associated patch)
- 'DAP' dropped and added and patched
- 'DAU' dropped and added but unknown (invalid)
- 'DAI' dropped and added but ignored (invalid, added files are no longer ignored)
- 'DAM' dropped and added but missing from the filesystem
-
- 'DR ' dropped and rename target
- 'DRP' dropped and rename target and patched
- 'DRU' dropped and rename target but unknown (invalid)
- 'DRI' dropped and rename target but ignored (invalid)
- 'DRM' dropped and rename target but missing from the filesystem
-
- 'R ' rename source
- 'R P' rename source and patched (invalid)
- 'R U' rename source and unknown (still exists on the filesystem)
- 'R I' rename source and ignored
- 'R M' rename source and missing (invalid)
-
- 'RA ' rename source and added (invalid, add should have associated patch)
- 'RAP' rename source and added and patched
- 'RAU' rename source and added but unknown (invalid)
- 'RAI' rename source and added but ignored (invalid, added files are no longer ignored)
- 'RAM' rename source and added but missing from the filesystem
-
- 'RR ' rename source and target
- 'RRP' rename source and target and target patched
- 'RRU' rename source and target and target unknown (invalid)
- 'RRI' rename source and target and target ignored (invalid)
- 'RRM' rename source and target and target missing
-
+
- known
- the file exists in the workspace, and in the revision manifest.
-
The two numbers are used to match up the pre-state and post-state of a
-rename. Imagine a situation where there are two renames.
-automate inventory will print something like:
+
- unknown
- the file exists in the workspace, but not in the revision manifest.
-
- R 1 0 a
- R 2 0 b
- R 0 2 c
- R 0 1 d
-
+
- invalid
- the file exists in the workspace and revision manifest, but with
+different types (one is a directory, the other a file).
+
+
- Here the status characters tell us that a and b were
-renamed, and we can tell that one was renamed to c and one was
-renamed to d, but we can't tell which was renamed to which. To
-do that, we have to refer to the numbers. The numbers do not themselves
-mean anything; their only purpose is to let you match up the two
-“ends” of a rename. The 1 in the left column by a means that
-a was the source of a rename, and the 1 in the right column by
-d means that d was the target of that same rename.
-Similarly, the two 2's tell us that b was renamed to c.
+
- changes
-
+
+ - content
- the file contents have changed.
-
There are two columns of numbers because the same file can
-simultaneously be the target and source of a rename. The number '0' is
-used as a meaningless placeholder in all cases where a file is not a
-source or target of a rename. Any non-zero number that occurs at all
-will occur exactly once in the first column and exactly once in the
-second column.
+
- attrs
- the file attributes have changed.
-
Full support for versioned directories is not yet complete and the
-inventory will only list entries for renamed or dropped
-directories.
+
+
- Error conditions:
-
When executed from outside of a workspace directory, prints an error
@@ -832,7 +931,7 @@ message to stderr, and exits with status
-
- mtn automate certs id
-
+
- mtn automate certs id
-
- Arguments:
-
A revision ID id, for which any certificates will be printed.
@@ -844,12 +943,11 @@ the following values are provided:
Prints all certificates associated with the given revision ID.
Each certificate is contained in a basic IO stanza. For each certificate,
the following values are provided:
-
-
+
'key'
a string indicating the key used to sign this certificate.
'signature'
- a string indicating the status of the signature. Possible
+ a string indicating the status of the signature. Possible
values of this string are:
'ok' : the signature is correct
'bad' : the signature is invalid
@@ -906,7 +1004,7 @@ or invalid prints an error message to st
-
- mtn automate stdio
-
+
- mtn automate stdio
-
- Arguments:
-
none
@@ -989,7 +1087,7 @@ whatever error message would have been g
-
- mtn automate get_revision
- mtn automate get_revision id
-
+
- mtn automate get_revision
- mtn automate get_revision id
-
- Arguments:
-
Specifying the option id argument outputs the changeset
@@ -1085,7 +1183,7 @@ to stderr and exits with status 1.
-
- mtn automate get_base_revision_id
-
+
- mtn automate get_base_revision_id
-
- Arguments:
-
None.
@@ -1111,7 +1209,7 @@ message to stderr, and exits with status
-
- mtn automate get_current_revision_id
-
+
- mtn automate get_current_revision_id
-
- Arguments:
-
None.
@@ -1138,7 +1236,7 @@ message to stderr, and exits with status
-
- mtn automate get_manifest_of
- mtn automate get_manifest_of revid
-
+
- mtn automate get_manifest_of
- mtn automate get_manifest_of revid
-
- Arguments:
-
Specifying the optional revid argument outputs the manifest for the
@@ -1275,7 +1373,7 @@ message to stderr and exits with status
-
- mtn automate get_attributes path
-
+
- mtn automate get_attributes path
-
- Arguments:
-
The argument path determines which path's attributes should be printed.
@@ -1350,7 +1448,7 @@ message to stderr and exits with status
-
- mtn automate set_attribute path key value
-
+
- mtn automate set_attribute path key value
-
- Arguments:
-
A path, an attribute key and an attribute value.
@@ -1372,7 +1470,7 @@ message to stderr and exits with status
-
- mtn automate drop_attribute path [key]
-
+
- mtn automate drop_attribute path [key]
-
- Arguments:
-
A path and an attribute key (optional).
@@ -1396,7 +1494,7 @@ status 1.
-
- mtn automate content_diff [--revision=id1 [--revision=id2]] [files ...]
-
+
- mtn automate content_diff [--revision=id1 [--revision=id2]] [files ...]
-
- Arguments:
-
One or more file arguments restrict the diff output to these files,
@@ -1449,7 +1547,7 @@ restrictions can't be applied, the comma
-
- mtn automate get_file id
-
+
- mtn automate get_file id
-
- Arguments:
-
The id argument specifies the file hash of the file to be output.
@@ -1478,7 +1576,7 @@ to stderr and exits with status 1.
-
- mtn automate get_file_of filename [--revision=id]
-
+
- mtn automate get_file_of filename [--revision=id]
-
- Arguments:
-
The filename argument specifies the filename of the file to be output.
@@ -1511,7 +1609,7 @@ with status 1.
-
- mtn automate get_option option
-
+
- mtn automate get_option option
-
- Arguments:
-
The option argument specifies the option name of the option to be output.
@@ -1536,7 +1634,7 @@ with status 1.
-
- mtn automate keys
-
+
- mtn automate keys
-
- Arguments:
-
None.
@@ -1578,7 +1676,7 @@ None.
-
- mtn automate packet_for_rdata id
-
+
- mtn automate packet_for_rdata id
-
- Arguments:
-
The id specifies the revision to output an rdata packet for.
@@ -1610,7 +1708,7 @@ and exits with status 1.
-
- mtn automate packet_for_certs id
-
+
- mtn automate packet_for_certs id
-
- Arguments:
-
The id specifies the revision to output cert packets for.
@@ -1668,7 +1766,7 @@ and exits with status 1.
-
- mtn automate packet_for_fdata id
-
+
- mtn automate packet_for_fdata id
-
- Arguments:
-
The id specifies the file to output an fdata packet for.
@@ -1698,7 +1796,7 @@ and exits with status 1.
-
- mtn automate packet_for_fdelta from-id to-id
-
+
- mtn automate packet_for_fdelta from-id to-id
-
- Arguments:
-
The from-id specifies the file to use as the base of the delta,
@@ -1730,7 +1828,7 @@ message to stderr and exits with status
-
- mtn automate get_content_changed id file
-
+
- mtn automate get_content_changed id file
-
- Arguments:
-
The id specifies a revision ID, from which content change calculations will be based.
@@ -1762,7 +1860,7 @@ message to stderr and exits with status
-
- mtn automate get_corresponding_path source_id file target_id
-
+
- mtn automate get_corresponding_path source_id file target_id
-
- Arguments:
-
The source_id specifies a revision ID in which file is current extant.
@@ -1800,7 +1898,7 @@ 1. Note that file not existin
-
- mtn automate db_get domain name
-
+
- mtn automate db_get domain name
-
- Arguments:
-
The domain and name specify the database variable
@@ -1813,8 +1911,8 @@ Read a database variable, see also Vars.
- Sample output:
-
-
- off.net
+
+ off.net
- Output format:
-
@@ -1827,7 +1925,7 @@ status 1.
-
- mtn automate db_put domain name value
-
+
- mtn automate db_put domain name value
-
- Arguments:
-
The domain and name specify the database variable
@@ -1840,8 +1938,8 @@ Change a database variable, see also Vars.
- Sample usage:
-
-
- mtn automate db_set database default-server off.net
+
+ mtn automate db_set database default-server off.net
- Output format:
-
@@ -1852,7 +1950,7 @@ None.
-
- mtn automate put_file [base-id] contents
-
+
- mtn automate put_file [base-id] contents
-
- Arguments:
-
The optional base-id specifies a file-id on which the contents are
@@ -1868,8 +1966,8 @@ See also aut
automate stdio.
- Sample output:
-
-
- 70a0f283898a18815a83df37c902e5f1492e9aa2
+
+ 70a0f283898a18815a83df37c902e5f1492e9aa2
- Output format:
-
@@ -1881,7 +1979,7 @@ exits with status 1.
-
- mtn automate put_revision revision-data
-
+
- mtn automate put_revision revision-data
-
- Arguments:
-
revision-data is the new revision. See example below. Note that
@@ -1913,8 +2011,8 @@ Workspace-less commit. Normally used via
content [5bf1fd927dfb8679496a2e6cf00cbe50c1c87145]
- Sample output:
-
-
- 4c2c1d846fa561601254200918fba1fd71e6795d
+
+ 4c2c1d846fa561601254200918fba1fd71e6795d
- Output format:
-
@@ -1928,7 +2026,7 @@ this fact, but otherwise works as normal
-
- mtn automate cert revision name value
-
+
- mtn automate cert revision name value
-
- Arguments:
-
revision is an existing revision, name is the certificate name
@@ -1942,8 +2040,8 @@ cert with a specific private key, use
--key.
- Sample usage:
-
-
- mtn automate cert 4c2c1d846fa561601254200918fba1fd71e6795d author address@hidden
+
+ mtn automate cert 4c2c1d846fa561601254200918fba1fd71e6795d author address@hidden
- Output format:
-
============================================================
--- docs/Branching-and-Merging.html e3acb9bc720c229d0f49419237fb1938979aabc8
+++ docs/Branching-and-Merging.html 2f0bd679bd2111d317fc4bbea458a889d1e655c2
@@ -110,5 +110,26 @@ should be able to handle it.
Whatever arrangement of branches you come up with, though, monotone
should be able to handle it.
+
If you are unsure of the name of a branch, you can list all branches using
+the ls branches command. This is very useful, but if you create
+a lot of branches then the list can become very long and unwieldy. To help
+this monotone has the suspend command which partially hides
+revisions/branches you are no longer using. Further commits on hidden branches
+will automatically unhide the branches.
+
+
For example, if Beth is now finished with the muffins branch, she can stop
+it from cluttering the list of branches by suspending the last revision in
+that branch:
+
+
$ mtn ls branches
+ jp.co.juicebot.jb7
+ jp.co.juicebot.jb7.muffins
+ $ mtn heads
+ mtn: branch 'jp.co.juicebot.jb7.muffins' is currently merged:
+ 4e48e2c9a3d2ca8a708cb0cc545700544efb5021 address@hidden 2007-07-08T02:17:37
+ $ mtn suspend 4e48e2c9a3d2ca8a708cb0cc545700544efb5021
+ $ mtn ls branches
+ jp.co.juicebot.jb7
+