How can I isolate some objects from a image?

(hot_air_balloons.jpg)

(canyon.jpg)

I try to isolate the 4 yellow hot air balloons from the image to paste them to canyon.jpg (second image). I aim to obtain a binary image that the pixesls of hot air ballons white and all other areas black to be able to isolate them from rest of the image. Could you help me to obtain that binary image?

[Umar]

Hi Ufuk,

You need to generate a binary image where the pixels of the hot air balloons are white and all other areas are black. This binary image will help you isolate the hot air balloons from the rest of the image. So, you already know how to read the original image containing the hot air balloons using the imread function. Next,convert the RGB image to grayscale using the rgb2gray function to simplify the processing. Now, to create a binary image, you need to apply a threshold to the grayscale image. So, choose set threshold value and adjust based on the specific image and requirements.The resulting binary image will have white pixels representing the hot air balloons and black pixels for the background. Finally, display the binary image using imshow and save it as your picture_binary.jpg for further use. Hopefully, this will help resolve your problem.

Please let me know if you have any further questions.

[DGM]

See also:

https://www.mathworks.com/matlabcentral/answers/1873142-replace-the-image-background

https://www.mathworks.com/matlabcentral/answers/1876692-filling-holes-in-the-masked-image-please-help-me-to-replace-blue-color-background-of-given-giraffe

https://www.mathworks.com/matlabcentral/answers/1615875-merging-two-images-with-background

Answer 1

[Kaustab Pal]

Hi @Ufuk Can,

From what I understand, you need a mask to segment out the balloons. While thresholding is one method to achieve this, it is most effective when the background color is uniform. In your case, the balloons are the foreground, and the background contains a wide range of colors.

A suitable approach would be to use the lazysnapping tool from the Image Processing Toolbox. With lazysnapping, you need to select both the foreground and background areas. To improve the accuracy of the results, you should adjust the number of foreground and background bounding boxes along with the number of the superpixels.

Below is a code snippet demonstrating how to use this tool for your example:

RGB = imread('Hot Air Balloons.jpeg');
imshow(RGB)
L = superpixels(RGB,5000); % vary the amount to make the results more accurate

% add more foreground bounding boxes to make the results more accurate
f1 = drawrectangle(gca,'Color','g');
f2 = drawrectangle(gca,'Color','g');
f3 = drawrectangle(gca,'Color','g');
foreground = createMask(f1,RGB) + createMask(f2,RGB) + createMask(f3,RGB);

% add more background bounding boxes to make the results more accurate
b1 = drawrectangle(gca,'Color','r');
b2 = drawrectangle(gca,'Color','r');
b3 = drawrectangle(gca,'Color','r');
b4 = drawrectangle(gca,'Color','r');
background = createMask(b1,RGB) + createMask(b2,RGB) + createMask(b3,RGB) + createMask(b4,RGB);

BW = lazysnapping(RGB,L,foreground,background);
imshow(BW);

You can read more about lazysnapping here: https://www.mathworks.com/help/images/ref/lazysnapping.html

Hope this helps.

Answer 2

[DGM]

As per usual with these assignments, the images are either microscopic, or they're low-quality JPGs or both. Besides the practical challenge of trying to get a good mask using automated magic on a pixel salad, we also have to contend with the usual issues of mismatched lighting and improper perspective making the composition conspicuously questionable.

How would I get a mask? I would use an image editing application to make a mask that's exactly as good or bad as I care to make it. I just threw it in GIMP, made a few lasso selections and tightened them up with brushwork. I'm only going to select two of the balloons because it's not my assignment and trying to cram all four into the output is just going to make everything look that much more ridiculous.

% inputs and parameters

% two low-quality mismatched JPGs
FG = imread('Hot Air Balloons.jpeg'); % 659x1200
BG = imread('Canyon.jpeg'); % 182x276

% an antialiased (numeric) mask
mk = imread('balloonmask.png'); % I, uint8

% amount to rescale objects
% this isn't necessarily the ratio of image sizes
scale = 0.25;

% object offsets in BG space (px, [x y])
% one row per object in the mask
% offsets must be non-negative integers
% offsets must keep the objects within BG
offset = [150 30; 40 10];

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% rescale the FG and mask
FG = imresize(FG,scale);
mk = imresize(mk,scale);

% get blob bounding boxes
S = regionprops(mk > 0,'boundingbox');

% compose the output one blob at a time
outpict = im2double(BG);
for k = 1:numel(S)
    % selections from FG, mask
    xr = ceil(S(k).BoundingBox(1)) + (0:S(k).BoundingBox(3) - 1);
    yr = ceil(S(k).BoundingBox(2)) + (0:S(k).BoundingBox(4) - 1);
    thisFG = im2double(FG(yr,xr,:));
    thismk = im2double(mk(yr,xr,:));
    
    % selection from BG
    xr = offset(k,1) + (1:S(k).BoundingBox(3));
    yr = offset(k,2) + (1:S(k).BoundingBox(4));
    thisBG = outpict(yr,xr,:);
    
    % compose
    outpict(yr,xr,:) = thisFG.*thismk + thisBG.*(1-thismk);
end

% rescast and scale as desired
outpict = im2uint8(outpict);

% show it
imshow(outpict)

I've made no effort to ensure that the selections stay within the extents of BG either by constraining subscripts or by cropping the selections. If you select a scaling factor and/or offset which places the output bounding box outside of the area of BG, it will produce an indexing error.

There really isn't any arrangement where the lighting and perspective make much sense with these objects and background, but I suppose it's not as bad as a basketball hovering in an arctic wasteland.

[Ufuk Can]

How did you obtain balloonmask.png? Could you share the codes to obtained balloonmask.png?

[Image Analyst]

@Ufuk Can, he said "How would I get a mask? I would use an image editing application to make a mask that's exactly as good or bad as I care to make it. I just threw it in GIMP, made a few lasso selections and tightened them up with brushwork."

For example you can use Photoshop and use the region selection ROI tool and just drag a box around your object and it will make its best guess as to what you think the region or interest is and it will select it. From there you can use the fill menu item to fill it with white, then inverse the mask and fill it with black. Then save the image (a copy of it so you don't overwrite the original). I think Photoshop would do a pretty good job of finding the balloons. But I think you want it all done it Photoshop. I haven't used lazysnapping like @Kaustab Pal suggested but it sounds like it might be pretty similar to the Photoshop implementation.

[DGM]

FWIW, I did try the lazysnapping demo. I'm not familiar with the algorithm, but this doesn't seem like a case where it would work well. Then again, I've never run into a scenario where such a variably crude mask is an appropriate compromise.

These are the selections I made:

... and this is the mask I automagically recieved.

Is that good enough for something that only needs crude object segmentation? Maybe.

Is that good enough for image composition? I'm going to say no. Could it be improved if I spent an hour fiddling with it and doing cleanup? Maybe, but it would take less time and effort to do the work manually instead.

If the goal is to do this in MATLAB, how would I even clean it up within MATLAB? Certainly, there are no brush tools, but maybe I could spend a bunch of time making freehand or polygon selections to refine the mask. That might be ridiculous, but why not just do that to begin with?

Like I said, doing it manually means that the outcome is exactly as accurate as you make it. There is no illusion of convenience, and there are no disappointing surprises.

So let's say that you wanted to generate the mask manually in MATLAB using basic tools, what tools would you use and how would the results differ? You can use the ROI tools like drawfreehand(), drawpolygon(), or drawassisted() to create editable selections for each region. You can then use createMask to convert the ROI objects into logical masks.

% an image
inpict = imread('Hot Air Balloons.jpeg');

% display the image
imshow(inpict);

% use the ROI tools to generate a selection
ROI = drawpolygon(gca)

% create a logical mask from the ROI object
mask = createMask(ROI);

I made only a very crude polygon selection of one balloon here, but considering the output resolution, we may not need to be much more precise than that. You can then simply combine multiple masks using basic logical operators such as ~, &, and |.

Other than completeness, how is this mask different than the other? This method produces a hard-edged logical mask, whereas balloonmask.png is an antialiased numeric mask. Using a logical mask has its limitations, but those are the tools that are directly available in MATLAB. The composition example given above will also work with a logical mask.