Computer Vision Utilities¶

DJL comes equipped with a number of helpful image processing and object detection utilities to make model creation and training as simple as possible.

Using BufferedImageFactory to Read Images¶

The BufferedImageFactory lets you create Images from a variety of sources like URLs, local files, and input streams.

// Load image from URL
URL url = new URL("https://s3.amazonaws.com/images.pdpics.com/preview/3033-bicycle-rider.jpg");
Image img = BufferedImageFactory.getInstance().fromUrl(url);

Using opencv extension for high performance image process¶

The DJL OpenCV extension provides better performance compare to java's built-in ImageIO. You only need to add it into your project and DJL will automatically pick it up:

<dependency>
    <groupId>ai.djl.opencv</groupId>
    <artifactId>opencv</artifactId>
</dependency>

Image Manipulation¶

Image provides a suite of image manipulation functions to let you pre- and post-process your images all within DJL.

int width = img.getWidth();
int height = img.getHeight();
Image leftHalfImg = img.getSubImage(0, 0, width / 2, height); // get left half of the image
NDManager manager = NDManager.newBaseManager();
NDArray imageArray = leftHalfImg.toNDArray(manager); // convert to NDArray

Saving and Loading Your Images¶

Now that you have done your pre or post processing, you'll probably want to save your images for future use. Simply call the save function from your Image and pass in an OutputStream and the image type (file extension). Note: Some JDKs (openJDK) may not support saving JPG files when the image contains an alpha channel.

// Save file
OutputStream out = new FileOutputStream("bicycle.png");
img.save(out, "png");

You can then use BufferedImageFactory to load it back in!

// Load image from local file
Image imgLoaded = BufferedImageFactory.getInstance().fromFile(Path.of("bicycle.png"));

Draw Bounding Boxes¶

Image includes a useful function to draw bounding boxes given a DetectedObjects instance generated from a ObjectDetection model. We'll use the pre-trained SingleShotDetection model from the model zoo to demonstrate below.

// Load Object Detection Model
Criteria<Image, DetectedObjects> criteria = Criteria.builder()
        .setTypes(Image.class, DetectedObjects.class)
        .optArtifactId("ssd")
        .build();
ZooModel<Image, DetectedObjects> model = criteria.loadModel();
Predictor<Image, DetectedObjects> predictor = model.newPredictor();

// Detect Objects
DetectedObjects detectedObjects = predictor.predict(img);

// Draw Bounding Boxes
img.drawBoundingBoxes(detectedObjects);

// Save Image with Bounding Boxes
OutputStream out1 = new FileOutputStream("bicycleBoundBox.png");
img.save(out1, "png");

Draw Joints¶

You can also draw joints if you have a Joints instance generated from a PoseEstimation model. Here, we'll use the SimplePose model from the model zoo!

// Load Pose Detection Model
Criteria<Image, Joints> criteria = Criteria.builder()
        .setTypes(Image.class, Joints.class)
        .optArtifactId("simple_pose")
        .build();
Predictor<Image, Joints> predictor = model.newPredictor();

// Detect Joints
Joints joints = predictor.predict(img);

// Draw Joints
img.drawJoints(joints);

// Save Image with Joints
OutputStream out2 = new FileOutputStream("bicycleJoints.png");
img.save(out2, "png");

Useful Information¶

If you want to learn more about loading models, click here.

If you want to learn more about the model zoo, click here.