hyperstyle 1.4.4

A tool for running a set of pre-configured linters and evaluating code quality.

Hyperstyle

A tool for running a set of pre-configured linters and evaluating code quality. It is used on the Hyperskill platform to check the quality of learners' code.

Read more detail about the project at Hyperskill Help Center

The dockerized version

What it does:

• Runs linters for several programming languages and parses their output;
• Prints the result using a unified JSON-based format;
• Evaluates the code quality value (EXCELLENT, GOOD, MODERATE, or BAD) based on the linters' output and some heuristics.

License and 3rd party software

The source code of hyperstyle is distributed under the Apache 2.0 License.

The 3rd party software we use in this project has its own licenses.

Python language (all versions can be found in the requirements.txt file):

• flake8 [MIT]

  • Site and docs
  • Repository

• Pylint [GNU LGPL v2]

  • Site and docs
  • Repository

• Radon [MIT]

  • Site and docs
  • Repository

Java language:

• PMD [BSD] (Version: 6.37.0)

  • Site and docs
  • Repository

• Checkstyle [GNU LGPL v2.1] (Version: 8.44)

  • Site and docs
  • Repository

Kotlin language:

• Detekt [Apache 2.0] (Version: 1.14.2)
  • Site and docs
  • Repository

JavaScript language:

• ESlint [MIT] (Version: 7.5.0)
  • Site and docs
  • Repository

Go language:

• golangci-lint [GNU GPL v3.0] (Version: 1.49.0)
  • Site and docs
  • Repository

---

Installation

You have to create set of environment variables:

• CHECKSTYLE_VERSION (the value of the variable must be the same with its value in Dockerfile)
• CHECKSTYLE_DIRECTORY (the directory with CHECKSTYLE linter sources)
• DETEKT_VERSION (the value of the variable must be the same with its value in Dockerfile)
• DETEKT_DIRECTORY (the directory with DETEKT linter sources)
• PMD_VERSION (the value of the variable must be the same with its value in Dockerfile)
• PMD_DIRECTORY (the directory with PMD linter sources)
• GOLANG_LINT_VERSION (the value of the variable must be the same with its value in Dockerfile)
• GOLANG_LINT_DIRECTORY (the directory with GOLANG_LINT linter sources)

Using script

Just run the following command:

./setup_environment.sh

and install everything the script suggests.

Note: You can also use this script to update linters. To do this, just update the corresponding linter version variables, run the script, and reinstall only the necessary linters.

Manually

If you don't want to use the script, you can install the environment manually.

Simply clone the repository and run the following commands:

1. pip install -r requirements.txt
2. pip install -r requirements-test.txt for tests
3. npm install eslint@7.5.0 -g && eslint --init

You can download all linters sources manually or by the following commands:

• CHECKSTYLE:

curl -L https://github.com/checkstyle/checkstyle/releases/download/checkstyle-${CHECKSTYLE_VERSION}/checkstyle-${CHECKSTYLE_VERSION}-all.jar > ${CHECKSTYLE_DIRECTORY}/checkstyle-${CHECKSTYLE_VERSION}-all.jar

• DETEKT:

curl -sSLO https://github.com/detekt/detekt/releases/download/v${DETEKT_VERSION}/detekt-cli-${DETEKT_VERSION}.zip \
&& unzip detekt-cli-${DETEKT_VERSION}.zip -d ${DETEKT_DIRECTORY} \
&&  curl -H "Accept: application/zip" https://repo.maven.apache.org/maven2/io/gitlab/arturbosch/detekt/detekt-formatting/${DETEKT_VERSION}/detekt-formatting-${DETEKT_VERSION}.jar -o ${DETEKT_DIRECTORY}/detekt-formatting-${DETEKT_VERSION}.jar

• PMD:

curl -sSLO https://github.com/pmd/pmd/releases/download/pmd_releases/${PMD_VERSION}/pmd-bin-${PMD_VERSION}.zip \
&& unzip pmd-bin-${PMD_VERSION}.zip -d ${PMD_DIRECTORY}

• GOLANG_LINT:

curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b ${GOLANG_LINT_DIRECTORY} v${GOLANG_LINT_VERSION}

Using docker

Alternatively, you can build a docker image by Dockerfile and run the tool inside this image. Or use the public docker image, that we use in the build.yml file.

Usage

Run the run_tool.py with the arguments.

A simple configuration: python run_tool.py <path>.

Required arguments:

1. path — path to file or directory to inspect.

Optional arguments:

Argument	Description
‑h, ‑‑help	show the help message and exit.
‑v, ‑‑verbosity	choose logging level according this list: 1 - ERROR; 2 - INFO; 3 - DEBUG; 0 - disable logging (CRITICAL value); default value is 0 (CRITICAL).
‑d, ‑‑disable	disable inspectors. Available values: for Python language: pylint for Pylint, flake8 for flake8, radon for Radon, python_ast to check different measures providing by AST; for Java language: checkstyle for the Checkstyle, pmd for PMD; for Kotlin language: detekt for Detekt; for JavaScript language: eslint for ESlint; for Go language: golang_lint for golangci-lint. Example: -d pylint,flake8.
‑‑allow-duplicates	allow duplicate issues found by different linters. By default, duplicates are skipped.
‑‑language-version, ‑‑language_version	specify the language version for JAVA inspectors. Available values: java7, java8, java9, java11, java15, java17. Note: ‑‑language_version is deprecated and will be deleted in the future.
‑‑n-cpu, ‑‑n_cpu	specify number of cpu that can be used to run inspectors. Note: ‑‑n_cpu is deprecated. Will be deleted in the future.
‑f, ‑‑format	the output format. Available values: json, text. Default value is json.
‑s, ‑‑start-line	the first line to be analyzed. By default it starts from 1.
‑e, ‑‑end-line	the end line to be analyzed. The default value is None, which meant to handle file by the end.
‑‑new-format	the argument determines whether the tool should use the new format. New format means separating the result by the files to allow getting quality and observed issues for each file separately. The default value is False.
‑‑history	JSON string with a list of issues for each language. For each issue its class and quantity are specified. Example: --history "{\"python\": [{\"origin_class\": \"SC200\", \"number\": 20}, {\"origin_class\": \"WPS314\", \"number\": 3}]}"
‑‑with‑all‑categories	Without this flag, all issues will be categorized into 5 main categories: CODE_STYLE, BEST_PRACTICES, ERROR_PRONE, COMPLEXITY, INFO.
‑‑group‑by‑difficulty	With this flag, the final grade and influence on penalty will be grouped by the issue difficulty.
‑‑language	Specify the language to inspect. The tool will check all languages by default. The default value is None.

The output examples:

(New format means separating the result by the files to allow getting quality and observed issues for each file separately)

1. Json old format (without ‑‑new-format argument):

{
  "quality": {
    "code": "BAD",
    "text": "Code quality (beta): BAD"
  },
  "issues": [
    {
      "code": "C002",
      "text": "Too long function. Try to split it into smaller functions / methods.It will make your code easy to understand and less error prone.",
      "line": "<the code line>",
      "line_number": 54,
      "column_number": 0,
      "category": "FUNC_LEN",
      "difficulty": "EASY",
      "influence_on_penalty": 0 
    },
    ...
  ]
}

2. Json new format (with ‑‑new-format argument):

{
  "quality": {
    "code": "BAD",
    "text": "Code quality (beta): BAD"
  },
  "file_review_results": [
    {
      "file_name": "<your file>",
      "quality": {
        "code": "BAD",
        "text": "Code quality (beta): BAD"
      },
      "issues": [
        {
          "code": "W0703",
          "text": "Catching too general exception Exception",
          "line": "<the code line>",
          "line_number": 174,
          "column_number": 12,
          "category": "BEST_PRACTICES", 
          "difficulty": "MEDIUM",
          "influence_on_penalty": 0 
        },
        ...
      ]
    }
  ]
}

3. Json old format (with ‑‑group‑by‑difficulty argument):

{
  "quality": {
    "EASY": {
      "code": "BAD",
      "text": "Code quality (beta): BAD"
    },
    "MEDIUM": {
      "code": "BAD",
      "text": "Code quality (beta): BAD"
    },
    "HARD": {
      "code": "BAD",
      "text": "Code quality (beta): BAD"
    }
  },
  "issues": [
    {
      "code": "C002",
      "text": "Too long function. Try to split it into smaller functions / methods.It will make your code easy to understand and less error prone.",
      "line": "<the code line>",
      "line_number": 54,
      "column_number": 0,
      "category": "FUNC_LEN",
      "difficulty": "EASY",
      "influence_on_penalty": {
        "EASY": 0,
        "MEDIUM": 0,
        "HARD": 0
      }
    },
    ...
  ]
}

4. Json new format (with ‑‑group‑by‑difficulty argument)

{
  "quality": {
    "EASY": {
      "code": "BAD",
      "text": "Code quality (beta): BAD"
    },
    "MEDIUM": {
      "code": "BAD",
      "text": "Code quality (beta): BAD"
    },
    "HARD": {
      "code": "BAD",
      "text": "Code quality (beta): BAD"
    }
  },
  "file_review_results": [
    {
      "file_name": "<your file>",
      "quality": {
        "EASY": {
          "code": "BAD",
          "text": "Code quality (beta): BAD"
        },
        "MEDIUM": {
          "code": "BAD",
          "text": "Code quality (beta): BAD"
        },
        "HARD": {
          "code": "BAD",
          "text": "Code quality (beta): BAD"
        }
      },
      "issues": [
        {
          "code": "W0703",
          "text": "Catching too general exception Exception",
          "line": "<the code line>",
          "line_number": 174,
          "column_number": 12,
          "category": "BEST_PRACTICES",
          "difficulty": "MEDIUM",
          "influence_on_penalty": {
            "EASY": 0,
            "MEDIUM": 0,
            "HARD": 0
          }
        },
        ...
      ]
    }
  ]
}

5. Text format:

Review of <path to your file or project> (N violations)
***********************************************************************************************************
File <file_name>
-----------------------------------------------------------------------------------------------------------
Line № : Column № : Type     : Inspector  : Origin : Description   : Line         : Path
54     : 0        : FUNC_LEN : PYTHON_AST : C002   : <Description> : <code line > : <path to the file>
...
-----------------------------------------------------------------------------------------------------------
Code quality (beta): BAD
Next level: EXCELLENT
Next level requirements:
FUNC_LEN: 12

***********************************************************************************************************
General quality:
Code quality (beta): BAD
Next level: EXCELLENT
Next level requirements:
FUNC_LEN: 12

---

Tests running

We use pytest library for tests.

Note: If you have ModuleNotFoundError while you try to run tests, please call pip install -e . before using the test system.

Note: We use eslint and open-jdk 11 in the tests. Please, set up the environment before running the tests. You can see en example of the environment configuration in the Dockerfile file.

Use pytest from the root directory to run ALL tests.