thegmu-imagemanage 1.0.0

The GMU Image Manage

The GMU Image Manage

https://www.thegmu.com/

Authors:

Mybrid Wonderful, Gregg Yearwood

Date:

11/14/2020

Support:

[e-mail]

Version:

1.0.0

• Documentation: http://thegmu-imagemanage.readthedocs.org/

• Source Code: https://bitbucket.org/thegmu/thegmu-imagemanage

---

---

Introduction

The GMU Image Manage is an Almost There Project where Almost There projects have the following distinctions:

1. Open source

2. Text interface

3. Text source

4. Do-it-yourself, or Maker design (DIY, Maker)

Almost There software has no web based or graphical interface because it is not considered a finished product. Also the goal is to get the public engaged into making slight changes to software such as naming things. Almost There projects represent a new market of enabling a much larger audience to modify software ever so slightly to realize big gains. See ALMOSTTHERESOFTWARE for further discussion.

I started this project because file manager software whether in Linux or in Windows only shows either a thumbnail or the file details but never both on a single line. This program does that.

Install this software and it will create a spreadsheet listing each image first as a thumbnail and then second a listing of image details, including width and height.

There are columns such as “delete” in the spreadsheet that one can use to manage files by first running the software to create the spreadsheet and afterwards again to process a spreadsheet that one has edited to delete, rename, and move images.

Bulk actions on all files in the directory are also possible such as removing all duplicate images with different names, removing files with zero size and removing all punctuation from a name so as to make the names more friendly to type within the spreadsheet.

---

Running

Quick reference:

cd ~/Pictures
source thegmu-imagemanage/almostthere/bin/activate
thegmu_im.py Pictures.commands.txt

---

Program

Program: thegmu_im.py <command file>

Each time you will need to do the following:

1. Open a command line shell.

2. Change directory to the image directory, example ‘cd ~/Pictures’.

3. Activate your Almost There Python environment, ‘source thegmu-imagemanage/almostthere/bin/activate’.

4. You will also need to create your first command file if you haven’t already.

In addition to the text of the program name then one other piece of text may follow the program name and that is the command file name. If the command file is not found in the image directory the program stops.

---

Command File

Default: thegmu_imagemanage.commands.txt

You must create a command file before running the program. If you run this Almost There application without a command file then the default command file is assumed. Whether you use the default command file or a specific command file processing stops if the file doesn’t exist.

Unlike traditional command line applications the Almost There software projects all use a file that lists one or more commands instead of passing commands on the command line. This do-it-yourself design requires combining many commands in a sequence as lines in a file where that sequence represents a new software application. Almost There developers can share command files like food recipes represent new cooking dishes.

Command files may contain only three kinds of lines:

1. Notes: lines that start with the pound ‘#’ sign are notes and are ignored.

2. Blanks: blank lines with nothing on them that are used for clarity when reading.

3. Commands: lines that are not a note or blank are treated as a command and if the command is not found by the Almost software then an error stops processing command file at that line.

Example:

# BEGIN
catalog

remove_empty_files
remove_duplicate_files
convert WEBP JPEG
convert BMP3 JPEG
convert_format_simple PNG JPEG
remove_multiple_format_files
catalog

jpeg_optimize_size 6000

# END

You can see in the example that further instructions for a command are provided to the command by instructions listed on the same line separated by spaces. For example, the convert command requires two additional instructions listing what format to convert from and convert to. I tend to keep files in JPEG format because JPEG has a feature most formats do not support and that is compressing files to file size and not just shrinking a file by width and height. The ‘jpeg_optimize_size 6000” command shrinks any JPEG image to approximately 6000KB, or 6MB.

---

Commands

Commands start each line in the command file. Commands may or may not take further instructions separated by spaces.

---

Catalog

Command: catalog <catalog file>

Default: thegmu_imagemanage.catalog.txt

The catalog file is used internally for storing information about files. You may refresh this catalog file at any time but it is up to you in your command file to do so. Building the catalog file can take time for directories with thousands of images. Once you get to know commands then you will learn when to use the catalog command to build a fresh catalog to ensure subsequent commands have an up-to-date catalog.

The catalog contains the following fields:

1. file_name: i.e. 011mlxq4s6e51.jpg.

2. format: Imagemagick format of JPEG, PNG, GIF, etc.

3. WxH: width by height, i.e. 1920x1080.

4. size: file size in bytes, i.e. 2985169.

5. date: created orlast modified to a second, i.e. 2020-07-31-14:45:44.

6. epoch: seconds since 1970, i.e. 1596221144.

7. md5sum: unique binary identifier used for checking duplicates, i.e. 84aa9f5563d106e0627d3d0a2f4049fe.

8. ext: file extension, i.e.jpg.

---

Catalog Excel

Command: catalog_excel <catalog file>

Output: thegmu_imagemanage.catalog.2020-11-14.01.xlsx

The spreadsheet catalog files created are intended for human editing using Excel, Libre Office or other spreadsheet editor. The output file name reflects the requested text catalog file name. The ‘txt’ file extension is replaced with ‘DATE.01.xlsx’. The ‘01’ designation is a count. Only one-thousand images are contained in each spreadsheet file. This means if there exists twenty-thousand images in a directory then there will be twenty different spreadsheet files created and ‘01’, ‘02, ‘03’, … files will be created. This approach was chosen because testing of various spreadsheet programs revealed that the performance of Excel and LibreOffice Calc varied greatly after more than a thousand thumbnails were embedded in the file, albeit on one sheet or multiple sheets.

Spreadsheet programs have many ways to sort rows based upon column filters. Sorting images by the various columns is one of the primary uses of the application. This does require one to have a certain degree of familiarity with spreadsheets. This is in keeping with the Almost There philosophy of being a do-it-yourself person. The more you do things on your own the more power you have to express yourself as you are not limited by the choices software designers make. So take the time to learn Excel or LibreOffice Calc and how to filter and sort rows based upon the data in the file. The “auto-filter” feature is a good place to start.

Once you run this command then subsequent commands will stop processing if these files already exist. This is because the assumption is that these files are being updated by you. What I do is create a new document directory in my ‘Documents’ folder and move these files into that folder if I need to rerun this command.

These spreadsheet files contain columns at the beginning of the sheet not found in the text catalog, most notably a thumbnail of the image. However there are also columns to manage individual files.

Excel catalog columns:

1. thumbnail: a 100pixel high thumbnail image.

2. original: hyperlink to the original image.

3. note: Add your notes here.

4. delete: Enter ‘delete’ in this columnto delete the file.

5. move: Enter a “copy” or “move” along with a directory name to copy or move the image to another directory.

6. rename: Enter a new file name to rename the file.

7. tags: Enter a comma separated list of tags. Tags are used in bulk commands to help organize files.

---

Convert

Command: convert <from> <to>

Deletes the original file. Convert image files from one format to another. See “Convert List Formats” for a listing of all the possible formats available for conversion. The format is not determined by the file name. The format is determined by the ones and zeroes in the file. If the format to be converted from like say with PNG is detected then the original PNG file is deleted after a new file is created with the PNG format using the PNG file extension, ‘png’. I standardize on JPEG format for all my images. I run this conversion process quite regularly and it is one of the more used features for me. Get a bunch of new images, convert them all to JPEG.

. .

Convert Format Simple

Command: convert_format_simple <from> <to>

See “Convert”. This does the same thing as the convert command except the original file is not deleted. I use this in tandem with the remove_multiple_format_files command. The two commands will leave the smallest file size of the two. This is particularly noteworthy for cartoons and graphic files that only have a few colors and compress well with PNG. In that case the PNG file will be favored by the remove_multiple_format_files command over JPEG and the JPEG file will be deleted.

---

Convert List Formats

Command: convert_list_formats

Prints a line of text showing all the formats allowed by the convert command:: 3FR 3G2 3GP AAI AI ART ARW AVI AVS BGR BGRA BGRO BIE BMP BMP2 BMP3 BRF CAL CALS CANVAS CAPTION CIN CIP CLIP CMYK CMYKA CR2 CRW CUR CUT DATA DCM DCR DCX DDS DFONT DJVU DNG DOT DPX DXT1 DXT5 EPDF EPI EPS EPS2 EPS3 EPSF EPSI EPT EPT2 EPT3 ERF EXR FAX FILE FITS FRACTAL FTP FTS G3 G4 GIF GIF87 GRADIENT GRAY GRAYA GROUP4 GV H HALD HDR HRZ HTM HTML HTTP HTTPS ICB ICO ICON IIQ INFO INLINE IPL ISOBRL ISOBRL6 JBG JBIG JNG JNX JPE JPEG JPG JPS JSON K25 KDC LABEL M2V M4V MAC MAGICK MAP MASK MAT MATTE MEF MIFF MKV MNG MONO MOV MP4 MPC MPEG MPG MRW MSL MSVG MTV MVG NEF NRW NULL ORF OTB OTF PAL PALM PAM PANGO PATTERN PBM PCD PCDS PCL PCT PCX PDB PDF PDFA PEF PES PFA PFB PFM PGM PGX PICON PICT PIX PJPEG PLASMA PNG PNG00 PNG24 PNG32 PNG48 PNG64 PNG8 PNM PPM PREVIEW PS PS2 PS3 PSB PSD PTIF PWP RAF RAS RAW RGB RGBA RGBO RGF RLA RLE RMF RW2 SCR SCT SFW SGI SHTML SIX SIXEL SR2 SRF STEGANO SUN SVG SVGZ TEXT TGA TIFF TIFF64 TILE TIM TTC TTF TXT UBRL UBRL6 UIL UYVY VDA VICAR VID VIFF VIPS VST WBMP WEBP WMF WMV WMZ WPG X X3F XBM XC XCF XPM XPS XV XWD YUV

---

Excel File Commands

Command: excel_file_commands <catalog excel file>

Only one Excel file can be processed per command. In order to process multiple Excel files then each file will need to be listed using a separate command.

This command will execute all the individual file commands entered into the spreadsheet such as to move, delete, or rename an image file.

See “Catalog Excel” for all the possible individual file commands.

---

Flatten Comma Names

Command : flatten_comma_names

The catalog command ignores all files with commas in the name. This is required because the catalog file separates fields using commas. The program will display all files skipped when the catalog command is run. If you see files ignored because they contain commas and you are comfortable replacing the comma with the underscore “_” for ALL files then run this commmand. You can always individually rename files and run the catalog command.

---

Flatten File Names

Command : flatten_file_names

Punctuation:

{}[](),:;<>!'`"@#$%^&*|

Flatten file names removes the listed punctutation from file names. In addition all spaces are replaced with underscores. I love this feature because image files that come from the wild do so with strange names, where the wild can be web site downloads and emails. Typing file names with punctuation is a pain so I use this command to strip the puncuation. The only caveat is if the resulting, stripped name already exists as a file. In that case this command will insert underscores at the first occurrence of puncuation until a new name can be found that doesn’t already exist.

---

JPEG Optimize Size

Command: jpeg_optimize_size <bytes>

There is a common Linux program called, jpegoptim. If you have this program installed on a Linux system then run this corresponding command to shrink all JPEG image files in a directory to approximately the size requested. The size given is in Kilobytes. This means a number like 6000 means 6MB, or 6000KB. Files of size smaller than the requested size are ignored.

---

List Empty Files

Command: list_empty_files

If one has thousands of files in a directory then something as simple as listing all empty image files can be quite handy. Empty files are often indication of failed downloads of image files. That is a common occurrence for me with my ISP and the size of some image files.

---

Remove Duplicate Files

Command: remove_duplicate_files

Remove all duplicate files leaving the original intact. The original is the file with earliest date. A duplicate file is determined by comparing file contents and not file names. If two files have exactly the same ones and zeroes that make up the file then the file is considered a duplicate and the file name is never taken into consideration.

---

Remove Empty Files

Command: remove_empty_files

Remove all empty image files in a directory.

---

Remove Multiple Format Files

Command: remove_multiple_format_files

Given two file names only differ by the file extension, like say ‘jpg’ versus ‘png’, then remove the larger one. I don’t know about you but sometimes I save a file as a different format, typically from say PNG to JPEG, and then I leave both files in the directory. Which format is removed? The answer is the file with the largest size. Whichever file is larger in size in bytes is the one removed. This is because I generally save PNG to JPEG to save space. However, sometimes PNG is smaller, especially for cartoons and other simple images. Whereas removing duplicate files only looks at the content of a file then removing multiple format files only looks at the file name. This means if two original files exist with the same name except for the file extension then one will be deleted. Make sure your original files have original names before running this command.

---

Installation

Things listed in the prerequisites require instructions found on the prerequisites web site for your operating system.

There exists a version of this application on the PyPi web site. However installing the package defeats the design of any Almost There software. Instead insall from source as instructed below. Build the package and then install from this build. Then you can start modifying the software files.

---

Installation Prerequisites

1. Linux: any current version should due but this has only been tested on Ubuntu 20.

2. GIT: a command line version runnable as “git”.

3. Imagemagick: Programs and libraries where libraries are used by Python for image processing. Some Imagemagick commands are used instead of libraries.

4. Python 3.6+: Dependencies require 3.6 or later.

5. Python virtualenv: Almost There software should not be installed to the OS but the image directory where all the files are owned by the user account and can be completely removed.

6. Make: Builds the application.

7. jpegpotim: This is optional and is used to compress JPEG files to a byte size.

---

Make

See MAKE.

MAKE.rst contains a comprehensive list of Makefile commands. Commands for runnging tests, creating web documents, and running code analysis using pylint are included.

The GMU Image Manage application is built using the The GMU PyPi Template project and the Makefile is from this project.

---

Installation Instructions

1. Open a command line shell.

2. Change directory into iamge directory that you wish to manage, example ‘cd ~/Pictures’.

3. Download the application with git, ‘git clone https://bitbucket.org/thegmu/thegmu-imagemanage’.

4. Change directory to the source directory, ‘cd thegmu-imagemanage’.

5. Create the Python environment for the Almost There application, ‘python3 -m venv almostthere’.

6. Activate your new Python envrionment, ‘source almostthere/bin/activate’.

7. Activate the build environment, ‘source bin/activate-almostthere’.

8. Build and install the application using make, ‘make install’.

9. Test installation with an empty command file that does nothing, ‘thegmu_im.py test/data/commands/no.commands.txt’.

Output:

thegmu_im.py test/data/commands/no.commands.txt
[11/14/2020 15:57:29] gim.prog.thegmu_im.py.119 % 'test/data/commands/no.commands.txt' command file name requested.

Shell commands only:

cd ~/Pictures
git clone https://bitbucket.org/thegmu/thegmu-imagemanage
cd thegmu-imagemanage
python3 -m venv almostthere
source almostthere/bin/activate
source bin/activate-almostthere
make install
thegmu_im.py test/data/commands/no.commands.txt

---

The End