Skip to main content

A package to analyze handwritten beam diagrams

Project description

Computer Vision for Beam Analysis

This repository contains the code for an AI-based system capable of recognizing and analyzing handwritten sketches of engineering beam diagrams.

Setup

All files implementing the core functionality of the system (namely analyze.py, beam.py, main.py, number.py, relationships.py, and yolo.py) are in the base directory.

  • The yolo.py file implements the object detection stage of the workflow (stage 1) via a wrapper to the YOLOv5 programs contained in the yolov5/ directory.
  • The number.py file implements the number reading stage of the workflow (stage 2) via a wrapper to the SimpleHTR programs contained in the numberhtr/ directory.
  • The beam.py, relationships.py, analyze.py files implement the feature association stage of the workflow (stage 3).
  • The analyze.py file also implements the structural analysis stage of the workflow (stage 4).
  • The main.py file utilizes functions in these programs to complete the overall workflow.

Training and testing datasets are contained in the data/ directory and the machine-learned models to be used are contained in the models/ directory. Each of these contains subfolders named features/, number/, and relationships/ which contain the relevant files for that model stage. Running pip install -r requirements.txt downloads and installs all required packages for the system to work.

Results

Using the baseline models included, 45% of the images in the testing dataset are analyzed entirely correctly, an impressive figure considering how many model inferences are required for an image to be entirely correct.

Usage

Proper usage of the end-to-end model is the following:

python3 main.py [-h] --image-name IMAGE_NAME [--features-path FEATURES_PATH] [--number-path NUMBER_PATH] [--relationships-path RELATIONSHIPS_PATH]

A path to an image to analyze must be provided following the --image-name (or --i) prefix. This produces structural analysis diagrams in a folder named after the image in the runs/ directory. The path to the models to be used in the object detection, number reading, and feature association stages can be specified if they differ from the baseline models. For example, python3 main.py --i data/test/IMG-8287.jpg analyzes the beam system in the first testing image.

To train a new MLP, ensure the number of parameters is set in the relationships.py file, and run:

python3 relationships.py --mode create --source data/relationships/preprocessed/<FILE> --preprocess no --epochs 30 --name models/relationships/<NAME>

Citation

@article{joffe2024cv,
    AUTHOR = {Joffe, Isaac and Qian, Yuchen and Talebi-Kalaleh, Mohammad and Mei, Qipei},
    TITLE = {A Computer Vision Framework for Structural Analysis of Hand-Drawn Engineering Sketches},
    JOURNAL = {Sensors},
    VOLUME = {24},
    YEAR = {2024},
    NUMBER = {9},
    ARTICLE-NUMBER = {2923},
    URL = {https://www.mdpi.com/1424-8220/24/9/2923},
    PubMedID = {38733029},
    ISSN = {1424-8220},
    ABSTRACT = {Structural engineers are often required to draw two-dimensional engineering sketches for quick structural analysis, either by hand calculation or using analysis software. However, calculation by hand is slow and error-prone, and the manual conversion of a hand-drawn sketch into a virtual model is tedious and time-consuming. This paper presents a complete and autonomous framework for converting a hand-drawn engineering sketch into an analyzed structural model using a camera and computer vision. In this framework, a computer vision object detection stage initially extracts information about the raw features in the image of the beam diagram. Next, a computer vision number-reading model transcribes any handwritten numerals appearing in the image. Then, feature association models are applied to characterize the relationships among the detected features in order to build a comprehensive structural model. Finally, the structural model generated is analyzed using OpenSees. In the system presented, the object detection model achieves a mean average precision of 99.1%, the number-reading model achieves an accuracy of 99.0%, and the models in the feature association stage achieve accuracies ranging from 95.1% to 99.5%. Overall, the tool analyzes 45.0% of images entirely correctly and the remaining 55.0% of images partially correctly. The proposed framework holds promise for other types of structural sketches, such as trusses and frames. Moreover, it can be a valuable tool for structural engineers that is capable of improving the efficiency, safety, and sustainability of future construction projects.},
    DOI = {10.3390/s24092923}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cv4beamanalysis-1.0.3.tar.gz (67.2 MB view details)

Uploaded Source

Built Distribution

cv4beamanalysis-1.0.3-py3-none-any.whl (67.3 MB view details)

Uploaded Python 3

File details

Details for the file cv4beamanalysis-1.0.3.tar.gz.

File metadata

  • Download URL: cv4beamanalysis-1.0.3.tar.gz
  • Upload date:
  • Size: 67.2 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.0 CPython/3.10.10

File hashes

Hashes for cv4beamanalysis-1.0.3.tar.gz
Algorithm Hash digest
SHA256 6cdedf0de01c32830676d1b55fbadecda28e86f03feaeccab0316aa338431718
MD5 e06ab6b85bd2f7a0fe4618997bdd3227
BLAKE2b-256 90c1d64010b5489a14c69c66cd71418649b4bf21c8633df671c3a82e91188bec

See more details on using hashes here.

File details

Details for the file cv4beamanalysis-1.0.3-py3-none-any.whl.

File metadata

File hashes

Hashes for cv4beamanalysis-1.0.3-py3-none-any.whl
Algorithm Hash digest
SHA256 ff0b70afdaab286b6887994f8be10111c2ad5df85de836bf988f18ab91c904ba
MD5 9e399f78c9c93b13d00a6aad96463026
BLAKE2b-256 68a00b8eb8136a13b84b6c68509441713e8e7f509002e3f21297e0b2fdd0c68e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page