LIBYARE - LIBrary for YARE (Yet Another Regular Expression) pattern matching

CONTENTS

    • 1.     Forewords
    • 1.1.   Introduction
    • 1.2.   Installation
    • 2.     Functions
    • 2.1.   Match Functions
    • 2.2.   Other Functions
    • 3.     Patterns
    • 3.1.   Simple Patterns
    • 3.1.1. Shell Patterns
    • 3.1.2. Numeric Patterns
    • 3.1.3. Charset Patterns
    • 3.1.4. Filtered Patterns
    • 3.2.   Compound Patterns
    • 4.     Masks
    • 4.1.   The disguise() Function
    • 5.     Afterwords
    • 5.1.   Versions
    • 5.2.   Credits

1. FOREWORDS

1.1. INTRODUCTION

LIBYARE  implements  YARE  (Yet  Another  Regular Expression). YARE is a
regular expression format intended to be more readable than the standard
one. It can accept simple patterns and compound patterns.

Simple  patterns  can  be  shell  patterns,  charset  patterns,  numeric
patterns or filtered patterns.

Compound  patterns  are  obtained  by combining together simple patterns
with  logical operators ('^' = not, '&' = and, ',' = or) and parenthesis
('(' and ')').

1.2. INSTALLATION

If for instance your Linux belongs to the Debian family, type:

    $ sudo apt install pipx

Then type:

    $ pipx install libyare
    $ pipx ensurepath

Later you can upgrade LIBYARE to a new version by:

    $ pipx upgrade libyare

2. FUNCTIONS

2.1. MATCH FUNCTIONS

In  order  to  use  LIBYARE  in  your  PyPI  project,  link  it  in your
pyproject.toml file:

    ...
    [project]
    ...
    dependencies = ["libyare", ...]
    ...

Then in your program you can write:

    from libyare import *

and use these four match functions:

    smatch(string, pattern) # case-sensitive match

    imatch(string, pattern) # case-insensitive match

    dmatch(string, pattern) # case-depending match, see 3.1.1.

    fmatch(string, pattern) # filename match, see 3.1.1.

2.2. OTHER FUNCTIONS

    int2human(int, length=6) # converts integer into human-readable,
                             # result length must be between 5 and 9

    human2int(string) # converts human-readable string into integer,
                      # on error raises ValueError

For details about strings in human-readable integer format, see 3.1.3.

    disguise(string, mask) # disguise a string by a mask

3. PATTERNS

3.1. SIMPLE PATTERNS

3.1.1. SHELL PATTERNS

Shell  patterns  are  standard Unix shell patterns, see documentation of
fnmatch Python module.

General rules:

    • '*' matches everything
    • '?' matches any single character
    • '[seq]' matches any single character in seq
    • '[!seq]' matches any single character not in seq

Examples:

    • pattern 'abc*' matches any string starting with 'abc'
    • pattern '*abc' matches any string ending with 'abc'
    • pattern '*abc*' matches any string containing 'abc'
    • pattern '[az]' matches 'a' or 'z'
    • pattern '[!az]' matches any single character except 'a' or 'z'
    • pattern  '[a-z]'  matches any single character between 'a' and 'z'
      ('z' included)
    • pattern  '[!a-z]' matches any single character not between 'a' and
      'z' ('z' included)
    • pattern  '[a-z0-9_]'  matches any single character between 'a' and
      'z' or between '0' and '9' or equal to '_'
    • pattern  '[!a-z0-9_]' matches any single character not between 'a'
      and 'z' and not between '0' and '9' and not equal to '_'

If  a  metacharacter  must  belong  to  a  shell pattern with no special
meaning, it must be quoted between '[' and ']'. More exactly:

    • '*' '?' '[' '^' '&' ',' '(' and ')' must always be quoted
    • '!' and '-' if not between '[' and ']' have no special meaning and
      don't need to be quoted
    • '=' '<' and '>' need to be quoted only if in first position
    • ']'  only can not be quoted, but you should not need it because an
      unmatched  ']'  has  no special meaning and doesn't raise a syntax
      error, while unmatched '[' '(' and ')' do

Examples:

    • pattern  '[(]*[)]' matches any string starting with '(' and ending
      with ')'
    • pattern  '[[]*]'  matches  any string starting with '[' and ending
      with ']'
    • pattern  '[<]*>'  matches  any string starting with '<' and ending
      with '>'
    • pattern '[=][[]?*]' matches any charset pattern, see 3.1.2.

You can quote '!' too, but not immediately after '[':

    • pattern '[?!]' matches '?' and '!'
    • pattern '[!?]' matches any character except '?'

You  can  quote metacharacter '-' too, a '-' after '[' or before ']' has
no special meaning:

    • patterns '[-pr]' and '[pr-]' match '-' 'p' and 'r'
    • pattern '[p-r]' matches 'p' 'q' and 'r'

'-' stands for itself even after a character interval:

    • pattern '[p-rx]' matches 'p' 'q' 'r' and 'x'
    • pattern '[p-r-x]' matches 'p' 'q' 'r' '-' and 'x'
    • pattern '[p-rx-z]' matches 'p' 'q' 'r' 'x' 'y' and 'z'
    • pattern '[p-r-x-z]' matches 'p' 'q' 'r' '-' 'x' 'y' and 'z'

Descending character intervals do not work:

    • pattern '[z-z]' is accepted and is equivalent to '[z]'
    • pattern '[z-a]' is accepted but it does not match anything

They  are  only  two  differences  between  shell  patterns  defined  by
fnmatch()  and  fnmatchcase()  functions  in  Python3 fnmatch module and
shell patterns accepted by YARE:

    • unmatched  '[' (as in pattern 'abc[def') is allowed by fnmatch but
      is rejected by YARE as a syntax error
    • null pattern '' is allowed by fnmatch but is rejected by YARE as a
      syntax error (see later for a workaround to match a null string by
      a not null pattern)

Match of shell patterns can be:

    • case-sensitive, by yarecsmatch() function
    • case-insensitive, by yarecimatch() function
    • case-depending, by yarecdmatch() function
    • filename, by yareosmatch() function

case-depending  match is case-sensitive for shell patterns containing at
least one lowercase letter, case-insensitive for the others:

    • dmatch('ram', 'RAM,?*.db') --> True
    • dmatch('Ram', 'RAM,?*.db') --> True
    • dmatch('x.db', 'RAM,?*.db') --> True
    • dmatch('x.Db', 'RAM,?*.db') --> False

Filename  match  for  shell  patterns  is  case-sensitive if the current
platform requires it (namely on Linux), else is case-insensitive (namely
on Windows):

    • fmatch('x.JPG',  '*.jpg')  -->  True on MS-Windows, False on Linux
      but not under Linux

3.1.2. NUMERIC PATTERNS

A  numeric  pattern  is  made up of a comparison operator followed by an
integer   in  human-readable  format.  It  matches  all  strings  which,
converted  from  human-readable  format  into integer, satisfy the given
comparison. Allowed comparison operators are:

    • '<' = less than
    • '=' = equal
    • '>' = greater than
    • '<=' = less or equal
    • '<>' = not equal
    • '>=' = greater or equal

WARNING: no other comparison operator (as '==' or '!=') is accepted.

An integer in human-readable format is made up of:

    • an optional plus '+' or minus '-' sign
    • an integer or float literal
    • an optional final alphabetic multiplier:
        • 'K' = 1024
        • 'M' = 1024 ** 2
        • 'G' = 1024 ** 3
        • 'T' = 1024 ** 4
        • 'P' = 1024 ** 5
        • 'E' = 1024 ** 6
        • 'Z' = 1024 ** 7
        • 'Y' = 1024 ** 8

Examples:

    • patterns '<0.5K' and '<512' are equivalent, they match all strings
      which,  interpreted as a human-readable integer, give a value less
      than 512
    • patterns  '<0.5E3'  and  '<500'  are  equivalent,  they  match all
      strings  which,  interpreted  as  a human-readable integer, give a
      value less than 500

Numeric   match   is   always  case-insensitive,  the  final  alphabetic
multiplier and the 'E' in float literals can be uppercase or lowercase.

Both   pattern   and  string  can  give  an  error  in  conversion  from
human-readable format into integer. A pattern error raises a SyntaxError
exception, while a string error makes the match function return a result
of False:

    • pattern   '=0,<>0'   matches   only  the  strings  in  well-formed
      human-readable integer format

3.1.3. CHARSET PATTERNS

A  charset  pattern is matched by the '[=][[]*]' pattern, it starts with
'=['  and  ends with ']', it is made up of a '=' character followed by a
shell  pattern suitable to match a single character. It matches the null
string  and  all  strings  where  each character matches the given shell
pattern, examples:

    • pattern '=[0-9]' matches the null string and any string made up of
      only digits (it is equivalent to '^*[!0-9]*')
    • pattern  '=[!0-9]'  matches the null string and any string made up
      of only non-digit characters (it is equivalent to '^*[0-9]*')

Charset match is always case-sensitive:

    • pattern '=[a-zA-Z0-9_]&[!0-9]*' matches Python identifiers

3.1.4. FILTERED PATTERNS

A filtered pattern is matched by the '[=][[]*]?*' pattern, it is made by
two components:

    • a charset pattern...
    • ...followed  by a pattern of any type (also a filtered pattern, if
      you think you need it)

Match is performed as follows:

    • the  string  to be matched is filtered by the charset pattern, the
      matching  characters  are  kept,  while  unmatching characters are
      discarded
    • the  resulting  filtered  string  is matched against the following
      pattern

A  filtered  pattern  is  always  distinguished  from  a charset pattern
because the null pattern is not allowed. Examples:

    • pattern '=[0-9]2026*' matches all strings whose numeric characters
      make  a  string  starting  with  '2026', regardless any intermixed
      nonnumeric character

    • pattern '=[0-9]<1000' matches all strings whose numeric characters
      make a number less than 1000, regardless any intermixed nonnumeric
      character

    • pattern  '=[a-z]???'  matches all strings containing exactly three
      lowercase   alphabetic   characters,   regardless  any  intermixed
      character of other kinds

3.2. COMPOUND PATTERNS

A  compound  pattern  is  made by combining simple patterns with logical
operators:

    • '^' = not
    • '&' = and
    • ',' = or

and parenthesis '(' and ')'.

In the following examples, p and q are two simple patterns:

    • pattern '^p' matches any string not matched by p
    • pattern 'p&q' matches any string matched by both p and q
    • pattern 'p,q' matches any string matched by p or q or both
    • pattern  '*.jpg,*.mp4'  matches  any  string ending with '.jpg' or
      with '.mp4'
    • pattern '^*' does not match anything
    • pattern '?*' matches any string of one or more characters, so...
    • ...pattern '^?*' matches the null string and nothing else

Two '^' characters cancel each other out:

    • patterns '^^p' and 'p' are equivalent

Precedence  is  of  course  '^' > '&' > ','. Precedence can be forced by
parenthesis,  so YARE follows the usual rules of Boolean algebra, namely
by the De Morgan's laws we get for each pattern p and q:

    • patterns '^p&^q' and '^(p,q)' are equivalent
    • patterns '^p,^q' and '^(p&q)' are equivalent

and by the distribution laws we get for each pattern p, q and r:

    • patterns 'p&(q,r)' and '(p&q),(p&r)' are equivalent
    • patterns 'p,(q&r)' and '(p,q)&(p,r)' are equivalent

Nesting of parenthesis has no practical limit. Example:

    • pattern  '=[0-9.]&=[.]...&[0-9]*[0-9]&^*..*'  matches  any  string
      containing four dot-separated unsigned decimal numbers

4. MASKS

4.1. THE disguise() FUNCTION

The  disguise(string,  mask)  function  transforms a string by a mask. A
mask can contain:

    • any  character (except '[' and '*') which is copied as is from the
      mask into the result
    • index  expressions  between  '['  and  ']', which extract a single
      character from the string into the result
    • slice   expressions  between  '['  and  ']',  which  extract  many
      characters from the string into the result
    • asterisks '*', which are simply shortcuts for the slice expression
      '[:]' which copies the whole original string into the result

Indexing and slicing follow the syntax of Python's indexing and slicing.
Indexing works as follows:

    • '[j]' selects the j-th character in the string

Index  j  must  be an integer literal, can be negative and is mandatory,
'[]'  is  not  allowed.  The  first  character is selected by '[0]', the
second  one  by  '[1]' and so on. A negative j means index counting from
the   end,  '[-1]'  selects  the  last  character,  '[-2]'  selects  the
penultimate  one, and so on. If j falls out of the string boundaries, no
error is raisen and nothing is added to the result.

Slicing works as follows:

    • '[a:z]'  selects  all  characters  in the string with index j such
      that a <= j < z
    • '[a:z:s]' selects all items in the string with index j where j = i
      + k * s, with k >= 0 and a <= j < z

Indexes  a,  z and s (start, end and step) must be integer literals, can
be negative and are optional, s can not be zero. Defaults are:

    • default for s is 1
    • if  s  is  positive,  default for a and z are start and end of the
      string
    • if  s  is  negative, defaults for a and z are end and start of the
      string

Slice  indicators  may  be preceded by a case-insensitive modifier char:
'L' (lowercase) 'U' (uppercase) or 'T' (titlecase).

Examples:

    • disguise('abcd', 'xy') -> 'xy'
    • disguise('abcd', 'x[1]y') -> 'xby'
    • disguise('abcd', 'x[11]y') -> 'xy'
    • disguise('abcd', 'x[-1]y') -> 'xdy'
    • disguise('abcd', 'x[:3]y') -> 'xabcy'
    • disguise('abcd', 'x[1:3]y') -> 'xbcy'
    • disguise('abcd', 'x[3:1]y') -> 'xy'
    • disguise('abcd', 'x[3:1:-1]y') -> 'xdcy'
    • disguise('abcd', 'x[9:99]y') -> 'xy'
    • disguise('abcd', 'x[:]y') -> 'xabcdy'
    • disguise('abcd', 'x*y') -> 'xabcdy'
    • disguise('abcd', 'x**y') -> 'xabcdabcdy'
    • disguise('abcd', 'x[::-1]y') -> 'xdcbay'
    • disguise('aB.cD', 'x[:]y') -> 'aB.cD'
    • disguise('aB.cD', 'x[l:]y') -> 'ab.cd'
    • disguise('aB.cD', 'x[u:]y') -> 'AB.CD'
    • disguise('aB.cD', 'x[t:]y') -> 'Ab.Cd'

5. AFTERWORDS

5.1. VERSIONS

    • 1.4.1 (Production/Stable)
        • added:  'L'  'U'  and  'T' (lowercase uppercase and titlecase)
          modifiers in slice indicators

    • 1.3.1 (Production/Stable)
        • added: filtered patterns
        • added:  new  names  for  match  functions:  smatch()  imatch()
          dmatch() and fmatch()
        • maintained   for   back-compatability:  old  names  for  match
          functions   yarecsmatch()   yarecimatch()   yarecdmatch()  and
          yareosmatch()
        • added: disguise() is_pattern() and is_mask() functions
        • bug: error in human2int() function, fixed

    • 1.2.1 (Production/Stable)
        • changed: algorithm to convert human-readable to int
        • bug:  '|'  operator  not  allowed between dicts in Python 3.6,
          fixed

    • 1.2.0 (Production/Stable)
        • compatible with previous version
        • added: numeric patterns

    • 1.1.0 (Production/Stable)
        • compatible with previous version
        • added: charset patterns
        • added: case-depending match by yarecdmatch() function

    • 1.0.0 (Production/Stable)
        • incompatible with previous versions
        • simplified redefined and optimized

    • 0.4.3 (Experimental/Deprecated)
        • updated: documentation

    • 0.4.2 (Experimental/Deprecated)
        • updated: documentation

    • 0.4.1 (Experimental/Deprecated)
        • first version published on pypi.org '

5.2. CREDITS

LIBYARE  program  has  been developed by Python 3.11.2 and IDLE 3.11.2.,
see:

    https://www.python.org

under Debian GNU/Linux 12.11 (bookworm), see:

    https://www.debian.org

LIBYARE  package  has been built and published on pypi.org by FLIT 4.0.2
(a simple packaging tool for simple packages), see:

    https://pypi.org/project/flit

This help text has been written and formatted by YAWP 2.1.1 (Yet Another
Word Processor, a word processor for plain text files, with PDF export),
see:

    https://pypi.org/project/yawp