Narex

If you have ever used regular expressions, then you know how difficult they can be. Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems [1].

Challenges of using regular expressions:

• Expressions easily become unreadable, as they are extremely dense.
• No standardization or cross-engine compatibility. Depending on the engine, it can vary significantly. Supported features and syntax often differ.
• Unnatural pattern memorization. Humans quickly forget the syntax, as it is not intuitive.
• The learning curve is steep, especially for non-tech users. Even though many non-tech users need data processing, regular expressions remain out of reach for them.

Roadmap

The ultimate goal is to produce a DSL that uses natural language and enables cross-engine compatibility.

The main use case is for the user to define the desired engine (Perl, Python, etc.) and write a regular expression using natural language. The output will be raw regular expression, which can be directly used within the specified engine.

This DSL can be widely used by people from different backgrounds, as it uses natural language. Tricky regular expressions are abstracted, and a universal tool for cross-engine support is provided. Learning this DSL frees you from ever having to remember regular expression syntax again.

The biggest issues are the vast number of engines, subtle differences, and partially supported advanced features. Due to the complexity of implementing a DSL that handles advanced features and multiple engine engines, support will be added gradually. Currently, only Python engine is supported.

Quick intro

Before we start, consider that more examples can be found in examples directory, while the full rules specification with many small examples can be found in docs/rules_specification.

Match phone number

""" 
    MATCH: +381 62 123 4567
    MATCH: 062/123-4567
    MATCH: 062-123-4567
    MATCH: 0621234567
    MATCH: 062 123 4567
    SKIP:  062.123.4567
"""

carrier {
      digit repeat 2 times
}

state {
      '+'
      digit between 1 and 9
      digit repeat 2 to 3 times
}

no_state {
      '0'
}

separator {
      maybe either '/' or '-' or whitespace
}

local {
      digit repeat 3 times
      separator
      digit repeat 4 times
}

phone_number {
      starts
      either state or no_state
      separator
      carrier
      separator
      local
      ends
}

flags:
      global match,
      multiline

engine:
      python

tests:
      "+381 62 123 4567",
      "062/123-4567",
      "062-123-4567",
      "0621234567",
      "062 123 4567",
      "062.123.4567"

target:
      phone_number 

User defines sub-regexes within clauses. The better the clause naming is, the easier it is for anyone to understand the regex. Therefore, the clause is the main concept. Each clause can contain multiple sub-clauses, which must be defined first, and each clause must contain at least one rule. Understanding the rules is quite simple, as they are already known from regex. Besides clauses, the user can also test the regular expression by using tests, define desired flags with flags, and specify the desired engine using engine. The last required keyword is target which holds one of the defined clause references.

Generated code:

##############################################################
######################### Raw regex ########################## 
##############################################################
#
#  ^(\+[1-9]\d{2,3}|0)((\/|-|\s))?\d{2}((\/|-|\s))?\d{3}((\/|-|\s))?\d{4}$
#
##############################################################
########################### Engine ########################### 
##############################################################
#
#  Python
#
##############################################################
########################### Tests ############################ 
##############################################################
#
#  test 1:
#      pattern: 
#              +381 62 123 4567
#      match 1: 
#              +381 62 123 4567
#          group 1: 
#               ...
#
#  test 2:
#      pattern: 
#              062/123-4567
#      match 1: 
#              062/123-4567
#          group 1: 
#               ...
#
#  test 3:
#      pattern: 
#              062-123-4567
#      match 1: 
#              062-123-4567
#          group 1: 
#               ...
#
#  test 4:
#      pattern: 
#              0621234567
#      match 1: 
#              0621234567
#          group 1: 
#               ...
#
#  test 5:
#      pattern: 
#              062 123 4567
#      match 1: 
#              062 123 4567
#          group 1: 
#               ...
#
#  test 6:
#      pattern: 
#              062.123.4567
#      No matches
#
##############################################################
####################### Generated code ####################### 
##############################################################
import re

text = ""   # empty
regex = '^(\\+[1-9]\\d{2,3}|0)((\\/|-|\\s))?\\d{2}((\\/|-|\\s))?\\d{3}((\\/|-|\\s))?\\d{4}$'

match_strings = re.findall(
    regex, 
    text, 
    flags=re.MULTILINE
)
match_objects = re.finditer(
    regex, 
    text, 
    flags=re.MULTILINE
)

As we can see, the chosen engine is Python, hence we got generated code for Python. The generated code file has comments separated into sections. We have the raw regex output, which can be useful for easier debugging by comparing it with the written model. Then we have information about the chosen engine. Finally, we have concrete code which is specific to the engine and supported libraries. This output depends on the chosen flags.

Structure

Narex/
|
├── src/narex/
|         ├── validators/
|         ├── generators/
|         ├── grammar/
|         ├── utils/
|         ├── cli/
|
├── extension/
├── examples/
├── tests/
├── docs/
|
├── .github/workflows/
├── pyproject.toml
├── LICENSE
├── README.md

Getting started:

Prerequsities:

• Python 3

Check pyproject.toml for more info.

NOTE: Don't activate the extension yet. If you already did, check VSCode extension section.

Regular user workflow

1. Create a virtual environment:

python -m venv .venv

2. Activate the virtual environment (Windows):

.\.venv\Scripts\activate 

3. Install dependencies:

pip install git+https://github.com/Vasilijez/Narex.git

4. Run VSCode from activated terminal:

code .

Pulling of the source code is optional.

If you are a contributor, check the developer workflow.

Using

You can use either of the two CLIs, Narex or textX. This is possible as Narex belongs to the textX ecosystem. They share logic, although the commands are slightly different.

Run the project:

i. You can optionally validate the model before running:

narex validate --path=<path>  # i.  Narex
textx check <path>            # ii. textX

ii. You can just run (includes validation):

narex run --path=<path> --cli-only        # i.  Narex    
textx generate <path> --target <engine>   # ii. textX

Flags

1. Please use --help flag at the beginning to understand all possible flags for certain command within concrete CLI.
2. --path and --output-path flags support both absolute and relative paths. For instance:

--path=C:\Users\...\model.nx
--path=./model.nx

3. --cli-only flag provides only the raw regex within CLI. In contrast, when the flag is omitted, the full code is generated in a standalone file.
4. --overwrite flag provides overwriting the file if already exists.
5. --output-path flag is used for specifing the output directory path of the generated file.
6. --engine (Narex) or --target (textX) flag provides an engine selection. Engine can be defined within model clause engine: as well. Engine defined by using parameter has higher priority than the engine defined by using the model.

One example with as many flags as possible:

narex run --path=input.nx --output-path=./dir --engine=python --overwrite     # i.  Narex
textx generate input.nx --target python --output-path=./dir --overwrite       # ii. textX

VSCode extension

Prerequisites:

• Python VSCode extension (don't care now, it will be prompted if missing).

Before activating the extension, be sure to follow getting started either for regular user or for developer, as the extension requires all dependencies to be installed. If something goes wrong, check Troubleshooting.

Installation

1. Navigate to the extension directory in order to find the narex-x.y.z.vsix extension file.
2. Install the extension by choosing the Install from VSIX option.

Troubleshooting

Skip this section if the extension works fine for you. Continue if you still have headaches.

You must choose Python from one of the following:

• i. Virtual environment from an already running VSCode instance.
• ii. Global Python from an already running VSCode instance.
• iii. Terminal with an activated virtual environment used to open VSCode (explained earlier).

Therefore, your setup must have all Narex dependencies installed in order for the extension to work properly. The main hurdle is buggy VSCode behavior. For instance, you may create a virtual environment and install all dependencies, but the extension still may not work. Activating the virtual environment can be done by command, but sometimes VSCode refuses to choose Python from the virtual environment, even if the virtual environment is activated in the terminal.

Try restarting the extension by sequentially clicking the Disable, Restart Extensions, and Enable button, then check the Python path selected by VSCode. The Python path will be explicitly shown each time you rerun the extension. If you don't see something like: Selected Python: c:\Users\John\Documents\GitHub\test\.venv\Scripts\python.exe then you definitely didn't activate the virtual environment inside VSCode (green (.venv) is not enough). You will very likely see the path to the global Python .exe, where you don't have the required dependencies installed.

The solution is to press CTRL + SHIFT + P, choose Select Interpreter: ..., and select python.exe from the .venv/Scripts directory.

Contributing

As this project is open source, everyone is welcome to contribute. If you have any suggestions, feel free to propose them by opening an issue. The most interesting ones can be analyzed and placed within the /docs directory. Like, Reverse engineering analysis.

Developer workflow (getting started)

1. Clone the project:

git clone https://github.com/Vasilijez/Narex.git

2. Change directory to Narex:

cd Narex

3. Create and activate the virtual environment (Windows):

python -m venv .venv
.\.venv\Scripts\activate

5. Install mandatory dependencies:

pip install -e .

6. Optionally, if developer needs all dependencies (e.g. tests):

pip install -e ".[dev]"

7. Run VSCode from activated terminal:

code .

VSCode extension

1. If you want to play with the extension, open the extension subproject in VSCode and run the following command:

npm install

2. Press the F5 key in Windows to start extension debugging.

3. If you made some changes, don't forget to re-run the command from 1.

4. Packaging is possible by running the following:

vsce package

5. After packaging, the extension's .vsix file will be available.

Releasing

You can automatically trigger the release process by pushing a tag that starts with the letter v. For instance, v1.2.3.

1. Navigate to the main branch:

git checkout main

2. Make sure to pull the changes before tagging:

git pull

3. Create a new tag:

git tag <tag-name>

4. Push the tag to the remote repository:

git push origin <tag-name>

Static analysis

You can do it on your own.

Run the static analysis locally:

mypy --strict <file-name>

Caveat: Static analysis is triggered automatically by GitHub Actions. Therefore, it is smart to run a type checker from time to time before creating a pull request.

References:

[1] Source of the famous “Now you have two problems” quote (Author: Jeffrey Friedl, Accessed: July 19, 2025)