Github

Introduction Last updated: 2019-06-01

This repository introduces GIMP3-ML, a set of Python plugins for the widely popular GNU Image Manipulation Program (GIMP). It enables the use of recent advances in computer vision to the conventional image editing pipeline. Applications from deep learning such as monocular depth estimation, semantic segmentation, mask generative adversarial networks, image super-resolution, de-noising and coloring have been incorporated with GIMP through Python-based plugins. Additionally, operations on images such as edge detection and color clustering have also been added. GIMP-ML relies on standard Python packages such as numpy, scikit-image, pillow, pytorch, open-cv, scipy. In addition, GIMP-ML also aims to bring the benefits of using deep learning networks used for computer vision tasks to routine image processing workflows.

YouTube Fork Now Report Bugs View Plugin Docs Contribution Star

Note

GIMP 3 has not been released yet and GIMP-ML is based on GIMP 2.99.6. Having a NVIDIA GPU (for CUDA) is recommeded for usage. Please run on images with smaller resolution or force CPU if CUDA is running of out memory.

Installation

Please follow below steps for Winodws/Linux :


Tip

It is suggested to manually download the weights and place in the GIMP-ML folder in the user home directory incase you have a slow internet connection.

Download Weights Manually

Windows Demo


Commands:

[1] git clone https://github.com/kritiksoman/GIMP-ML.git

[2] Move inside GIMP-ML folder, then git checkout --track origin/GIMP3-ML

[3] GIMP-ML\install.bat gpu or GIMP-ML\install.bat if GPU is not present.

Linux Demo

Commands:

[1] git clone https://github.com/kritiksoman/GIMP-ML.git

[2] Move inside GIMP-ML folder, then git checkout --track origin/GIMP3-ML

[3] bash GIMP-ML/install.bat

Model Zoo

Currently the following models are present in GIMP-ML.

List of available models:
Name License Dataset Model
deblur BSD 3-clause GoPro DeblurGANv2
faceparse MIT CelebAMask-HQ Modified BiSeNet
coloring MIT ImageNet Local Hints Network
monodepth MIT Multiple DPT
super-resolution MIT ImageNet SRResNet
matting Non-commercial purposes Adobe Deep Image Matting Encoder Decoder Refinement
semantic-segmentation MIT ADE20K DPT
kmeans BSD - Scipy
dehazing MIT Custom AOD-Net
denoising GPL3 BSD68 DnCNN
enlighten BSD Custom EnlightenGAN
interpolate-frames MIT HD RIFE
inpainting CC BY-NC 4.0 CelebA, CelebHQ, Places2, Paris StreetView edge-connect
Detect Objects Apache-2.0 COCO YOLOv4
Filter Folder Apache-2.0 COCO YOLOv4
Canny Edge Apache-2.0 - -

Use with GIMP

List of various plugins and usage:

In-painting (Edge-connect model)

Requires 2 inputs:
[1] Image Layer.
[2] Mask Layer containing mask of object to be removed. Background should be black (255,255,255) and object should be white (0,0,0).
Both layers should be selected and then the plugin should be run from the GIMP-ML menu.

Note

The mask layer should be created using paintbrush tool having 100 hardness and size as 15px.


Interpolate-frames

Requires 3 inputs:
[1] Image Layer which will be the starting frame.
[2] Image Layer which will be the ending frame.
[3] Output Location: Folder where interpolated frames should be saved.
Both layers should be selected and then the plugin should be run from the GIMP-ML menu.

De-blur

Used for removing motion blur. Works on currently selected layer as input.

De-haze

Works on currently selected layer as input

De-noise

Works on currently selected layer as input

Enlightening

Works on currently selected layer as input

MonoDepth

Works on currently selected layer as input

Semantic Segmentation

Works on currently selected layer as input containing any of the following: person, bird, cat, cow, dog, horse, sheep, aeroplane, bicycle, boat, bus, car, motorbike, train, bottle, chair, dining table, potted plant, sofa, and tv/monitor.

Face Parsing

Works on currently selected layer as input containing only portrait image of a person.

Image Super-resolution

Requires the factor by which the image is to be enlarged as input. Set "Use as filter" to True if image size is medium/large in size (i.e., >~ 400pixels in height or width), otherwise you might run out of memory.

K-means Clustering

[1] Number of clusters/colors in output.
[2] Use position: if (x,y) coordinates should be used as features for clustering.

Deep Image Matting

Requires 2 layers as input: [1] Image Layer
[2] Trimap Layer: Use RGB as [128,128,128] for boundaries, [255,255,255] for object and [0,0,0] for background.
Both layers should be selected and then the plugin should be run from the GIMP-ML menu.

Deep Image Coloring

The image should be greyscale but the image mode should be RGB. This can be done from Image->Mode->RGB...
Requires 2 layers as input:
[1] Greyscale Image Layer
[2] Color Mask Layer: A transparent RGB layer (with alpha channel) that contains (local points) dots of size 6 pixels specifying which color should be present at which location.
Both layers should be selected and then the plugin should be run from the GIMP-ML menu.
Optionally, only the greycale image layer can be selected and colored.

Note

Known issue: Color spreads arbitrarily when used with mask layer. Current solution is to color multiple times and selectively erase unwanted layer portions. Example: Coloring Demo


Detect Objects

Detects objects present in the current layer. Objects have to be from one of the class in COCO dataset.

Note

COCO Class: person|bicycle|car|motorbike|aeroplane|bus|
train|truck|boat|traffic light|fire hydrant|stop sign|parking meter
|bench|bird|cat|dog|horse|sheep|cow|elephant|bear|zebra|
giraffe|backpack|umbrella|handbag|tie|suitcase|frisbee|skis|
snowboard|sports ball|kite|baseball bat|baseball glove|
skateboard|surfboard|tennis racket|bottle|wine glass|cup|fork|knife|
spoon|bowl|banana|apple|sandwich|orange|broccoli|carrot
|hot dog|pizza|donut|cake|chair|sofa|pottedplant|bed|diningtable|
toilet|tvmonitor|laptop|mouse|remote|keyboard|cell phone|
microwave|oven|toaster|sink|refrigerator|book|clock|vase|scissors|
teddy bear|hair drier|toothbrush


Filter Folder

Moves files in a folder to a separate folder based on objects present in them. Objects must be from one of the class in COCO dataset. Objects to search should be pipeline separated list of classes.

Note

COCO Class: person|bicycle|car|motorbike|aeroplane|bus|
train|truck|boat|traffic light|fire hydrant|stop sign|parking meter
|bench|bird|cat|dog|horse|sheep|cow|elephant|bear|zebra|
giraffe|backpack|umbrella|handbag|tie|suitcase|frisbee|skis|
snowboard|sports ball|kite|baseball bat|baseball glove|
skateboard|surfboard|tennis racket|bottle|wine glass|cup|fork|knife|
spoon|bowl|banana|apple|sandwich|orange|broccoli|carrot
|hot dog|pizza|donut|cake|chair|sofa|pottedplant|bed|diningtable|
toilet|tvmonitor|laptop|mouse|remote|keyboard|cell phone|
microwave|oven|toaster|sink|refrigerator|book|clock|vase|scissors|
teddy bear|hair drier|toothbrush


Canny Edge

Apply canny edge detector to the current selected layer.

Citation

@article{soman2020GIMPML,
title={GIMP-ML: Python Plugins for using Computer Vision Models in GIMP},
author={Soman, Kritik},
journal={arXiv preprint arXiv:2004.13060},
year={2020}
}

References

Following are the code and paper references:

Code References

Paper References