connor-nlp 1.0.0

Fast and fully local NLP file organizer that organizes files based on their content.

Connor is a file organizer written in python. It makes use of the sentence-transformers framework for the main organization process and the PyQt6 GUI toolkit for the graphical user interface. It is by no means supposed to substitute for organzing files by hand. It is just a concept. Connor features a fast and fully local file organizer that uses natural language processing to organize computer files based on their textual content.

Features

Connor works locally on your computer using a pre-trained NLP model sentence-transformers/paraphrase-MiniLM-L6-v2 to understand the meaning of the data and calculate the cosine similarity between files. The folders are appropriately named using topic modeling through the Latent Dirichlet Allocation (LDA) technique.

The file names and contents are read, then cosine similarity is used to calculate the similarity between the content of every file with respect to every other file. Based on the condition that the similarity scores between the files are above the provided threshold, the files are grouped in key-value pairs into a dictionary where each category corresponds to a folder.

Latent Dirichlet Allocation is then used to generate topic names for the contents in each folder, i.e., the categories in the dictionary. Folders are created using the most relevant topic names, and the corresponding files are then moved into their appropriate folders.

For files such as images (image support will be added later), executables, binaries, etc. that cannot be read are organized into a _misc folder based on their file extensions.

File Organization Summary

1. Organize files within a selected folder or manually uploaded files (uploading files is only supported for GUI).
2. Organize text-based files (.docx, .txt, .pdf, etc.) using NLP.
3. Creates a separate folder named "Miscellaneous" for dissimilar or unprocessable files based on extension.
4. Provide a summary (tree structure) of the organization process upon completion.

Customization Options

1. Similarity Threshold: Allows you to choose a similarity percentage threshold for grouping similar files.
2. Reading Word Limit: You can set a limit on the number of words to read from the file content.
3. Folder Name Word Limit: You can specify the maximum number of words allowed in the created folder names.
4. Default Parameters: You can modify these three parameters and save them for future sessions.

User Preferences

Command Line Interface: Simple and concise command line interface to quickly organize folders. Graphical User Interface: Provides a simplistic and straightforward GUI for ease of use with upload files feature.

Installation

There are installation instructions for both GUI and CLI. You can choose the one you want to install. If you're opting for building the application from source then adding the run file to path is recommended.

Install Connor via pip:

1. Make sure you have python and pip installed and added to path.
2. Run pip install connor-nlp

Install the GUI version of Connor (executable)

1. Go to the latest release.
2. Follow the steps there.
3. Run the executable (.exe).

Usage

Command Structure

connor [command] [options]

Commands

run: Run the folder organization process.

Usage:

connor run <folder_path>

Options:

• folder_path: Required. Absolute path to the folder that you want to organize.

Example:

connor run /path/to/your/folder

settings: Update the default settings for the tool.

Usage:

connor settings [options]

Options:

• -f, --folder-word-limit: Set the maximum length for folder names. (default: 3)
• -r, --reading-limit: Specify the word limit for reading files. (default: 200)
• -t, --similarity-threshold: Define the similarity threshold percentage. (default: 50)
• --show: Show current settings

Example:

connor settings -f 2 -r 150 -t 60

$ connor settings --show
To see how to update: Connor settings [-h]

Current settings:
  folder words limit     3
  reading limit          200
  similarity threshold   50%

--gui: Run Connor as a full fledged GUI from the terminal.

Usage:

connor --gui

Help

To view help information for commands and options use the -h or --help flag.

Example:

$ connor -h
usage: Connor [-h] [--gui] {settings,run} ...

Connor: Fast and local NLP file organizer

positional arguments:
  {settings,run}
    settings      Update the settings for the organizer
    run           Run the folder organization process

options:
  -h, --help      show this help message and exit
  --gui           Run the application in GUI mode.

Source

1. Clone repository:

git clone https://github.com/ycatsh/connor.git
cd connor

2. Create and activate virtual environment:

python3 -m venv venv
source venv/bin/activate

3. Install dependencies:

pip3 install -r requirements.txt

4. Run program:

For GUI:

python3 run.py --gui

For CLI:

python3 run.py -h

5. Install locally (optional):

pip3 install .

Example:

connor --gui

connor -h

License

This project is distributed under MIT License, which can be found in LICENSE in the root dir of the project. I reserve the right to place future versions of this project under a different license.