Skip to main content

libcom: everything about image composition


PyPI Downloads visitors Static Badge Static Badge Static Badge Static Badge


Welcome to scan the WeChat QR code to join the discussion group on image composition/compositing, object insertion/addition.

Image

Introduction

libcom (the library of image composition) is an image composition toolbox. The goal of image composition (object insertion) is inserting one foreground into a background image to get a realistic composite image, by addressing the inconsistencies (appearance, geometry, and semantic inconsistency) between foreground and background. Generally speaking, image composition could be used to combine the visual elements from different images.


libcom covers a diversity of related tasks in the field of image composition, including image blending, standard/painterly image harmonization, shadow generation, reflection generation, object placement, generative composition, quality evaluation, etc. For each task, we integrate one or two selected methods considering both efficiency and effectiveness. The selected methods will be continuously updated upon the emergence of better methods.

The ultimate goal of this library is solving all the problems related to image composition with simple import libcom. Welcome to follow WeChat public account "Newly AIGCer" or Zhihu Column "Newly CVer" to get the latest information about image composition!

Welcome to recommend your self-developed models or models you have tested by opening an issue. If a model delivers outstanding and stable performance, we will consider integrating it into libcom.

If you specialize in model inference acceleration, we welcome both voluntary and paid collaborations. Please contact us at ustcnewly@sjtu.edu.cn.

Online Workbench

We have built the image composition workbench based on our Libcom toolbox. The interface is as follows:


![]

Main Functions

  • get_composite_image generates composite images using naive copy-and-paste followed by image blending (alpha blending or Poisson blending).
  • OPAScoreModel [OPA] evaluates the rationality of foreground object placement (location, scale) in a composite image.
  • FOPAHeatMapModel [FOPA] can predict the rationality scores for all locations/scales given a background-foreground pair, when placing the foreground with certain scale at certain location in the background. Based on the predicted rationality score map, the model can output the composite image with optimal location/scale.
  • color_transfer adjusts the foreground color to match the background using traditional color transfer method.
  • ImageHarmonizationModel [PCTNet] [LBM] adjusts the foreground illumination to be compatible the background given photorealistic background and photorealistic foreground.
  • PainterlyHarmonizationModel [PHDNet] [PHDiffusion] adjusts the foreground style to be compatible with the background given artistic background and photorealistic foreground.
  • HarmonyScoreModel [BargainNet] evaluates the harmony level between foreground and background in a composite image, that is, whether the illumination of foreground is harmonious with that of background.
  • InharmoniousLocalizationModel [MadisNet] localizes the inharmonious region in a synthetic image.
  • FOSScoreModel [DiscoFOS] evaluates the compatibility between foreground and background in a composite image in terms of geometry and semantics, that is, whether the geometric perspective of foreground matches that of background, and whether the semantic context of foreground in the background is reasonable.
  • ShadowGenerationModel [GPSDiffusion] generates plausible shadow for the inserted object in a composite image.
  • ReflectionGenerationModel [RGDiffusion] generates plausible reflection for the inserted object in a composite image.
  • KontextBlendingHarmonizationModel [FluxKontext] is a generative image composition model which inserts foreground into the specified bounding box in the background. The pose and view of foreground stay unchanged. The "blending" mode does not adjust the foreground illumination, while the "harmonization" mode adjusts the foreground illumination to make the composite image harmonious.
  • OSInsertModel [OSInsert] is a generative image composition model which inserts foreground into the specified bounding box in the background. The "conservative" mode has weak ability to adjust the pose and view of foreground according to the background. The "aggressive" mode has stronger ability to adjust the pose and view of foreground according to the background. You can also use foreground shape mask for controllable composition.

For the detailed method descriptions, code examples, visualization results, and performance comments, please refer to our [documents]. If the model performance is not satisfactory, you can finetune the pretrained model on your own dataset using the source repository and replace the original model.

Requirements

The main branch is built on the Linux system with Python 3.10 and PyTorch>=2.6. For other dependencies, please refer to [requirements].

Get Started

Please refer to [Installation] for installation instructions and [documents] for user guidance.

Contributors

License

This project is released under the Apache 2.0 license.

Bibtex

If you use our toolbox, please cite our survey paper using the following BibTeX [arxiv]:

@article{niu2021making,
  title={Making images real again: A comprehensive survey on deep image composition},
  author={Niu, Li and Cong, Wenyan and Liu, Liu and Hong, Yan and Zhang, Bo and Liang, Jing and Zhang, Liqing},
  journal={arXiv preprint arXiv:2106.14490},
  year={2021}
}

Metadata

Release files for libcom 0.1.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for libcom 0.1.8
File Size Uploaded Provenance Actions
libcom-0.1.8.tar.gz 527.8 kB Without provenance Details for libcom-0.1.8.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for libcom 0.1.8
File Interpreter ABI Platform Actions
libcom-0.1.8-py3-none-any.whl Python 3 none any Details for libcom-0.1.8-py3-none-any.whl

Total release size: 1.2 MB

Release files / libcom-0.1.8.tar.gz

Download URL libcom-0.1.8.tar.gz
Size 527.8 kB
Tags Source
SHA-256 checksum
How to use checksums
66a1655ee99b5a422b21ea69d0602f47ccfeb7bf6cdd7749ebe303e3b3244a33
BLAKE2b-256 checksum
How to use checksums
366e9cd68fd28cbfa821ca9bdecb67699e245223482bbc83a6f4c175b1c42e74
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.19

Release files / libcom-0.1.8-py3-none-any.whl

Download URL libcom-0.1.8-py3-none-any.whl
Size 652.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
92f21d3a4cf148d31df6ea83f83ceac6de1fa1f91c2dca50fa695ecf981d4977
BLAKE2b-256 checksum
How to use checksums
fb81a140ef7cbc9ecdfd1054c2f19c2876ac53b401336155b6e95ddaa6a5d23e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.10.19

Security for libcom 0.1.8

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page