How to extract raw data from a rgb image visualized with known colormap?

Hello, everyone,

I have a rgb image, and it was visualized with a known colormap. Now I want to extract the raw 2D matrix data rather than RGB values of this image before it was mapped to the colormap . How should I do?

Here is an example:

raw_data = rand(256,256);
figure;
imagesc(raw_data);
axis image off;
colormap(jet);
colorbar;
title('saved image');

Then I save this visualized image, names as, for example 'saved image.jpg'.

How should I retrieve 'raw_data' from 'saved_img.jpg' according to the known 'jet' colormap?

Thank you!

Answer 1

[DGM]

Bear in mind that if what you have is a screenshot of plotted data, you're only going to get back some interpolated representation of quantized data. It will be lossy, polluted by any tick marks or grid lines, and the array you get back will not necessarily be the same size or aspect ratio as the original data.

An example

% let's say i ran some code an hour ago for some other thing
% that's totally unrelated to what we're doing now.
z = peaks(100);
surf(z)
colormap(jet(16))

% ... but let's just delete that because we don't need it anymore
clf

% now let's plot some 2D data with pcolor() and capture it!
% the original conditions
A = rand(100);
actualmap = parula();
actualrange = [0.02 0.98];

% display the matrix with the given map and range
pcolor(A)
caxis(actualrange)
colormap(actualmap)
colorbar

% take a screenshot
% this is a better scenario than is typical
% since the screenshot is already cropped for us
screenshot = frame2im(getframe(gca));

figure
imshow(screenshot,'border','tight') % show it

% let's recover the data!
% hmm what settings did i use?
assumedmap = parula(); % oh i just called parula()
assumedrange = [0 1]; % and it looks like the colorbar goes from 0 to 1
B = flipud(screenshot); % flip the image depending on the ydir used in the plot
B = rgb2ind(B,assumedmap); % convert to quantized grayscale
B = interp1([0 size(assumedmap,1)],assumedrange,double(B),'linear'); % rescale

% let's say you wanted it to be the original size
% and let's assume you knew what the original size was
B = imresize(B,size(A),'bicubic');
imshow(A-B+0.5,'border','tight') % error map

% the range of the error exceeds that of the data
[min(A(:)-B(:)) max(A(:)-B(:))]

ans = 1×2

-0.5297 0.9930

These are practical pitfalls. The aspect ratio of the displayed representation is purely an artifact of the figure/axes geometry, not the data Using pcolor() means that the data is represented on the vertices, not the faces. By default, pcolor() also adds edge lines, which will pollute the recovered data.

If you're obtaining the caxis limits from visual inspection of a colorbar, you're often left guessing.

Similarly, even if you know which colormap was used, common behavior will leave many users misunderstanding even their own recollection. You might be wondering why there's a surf() plot in this example. Note the way that jet() and parula() are called here. In the first part, jet() is explicitly given a length parameter. So at that point, the default map length for that figure is 16. In the next call to parula(), no length parameter is given, so it's inherited from the figure. The second call to parula() is identical, but because it's assigning the colormap property of a new figure, the default map length of 256 is used. Two identical calls to parula(); one gives a 16-color map, the other gives a 256-color map. Do you know what your actual map length was? I would argue that many people don't.

Does the length matter that much? When the other sources of error are this bad, not really. If the lengths of the two maps were swapped, it might be considered worse.

pcolor() in particular

What about using pcolor() instead of imagesc()? Will that change anything? Depends on the scale. You be the judge.

% the original conditions
A = magic(4); % a 4x4 image
actualmap = parula(16);
actualrange = [1 16];

% display the matrix with the given map and range
pcolor(A) % data is on vertices, not faces!
caxis(actualrange)
colormap(actualmap)
colorbar

% take a cropped screenshot
screenshot = frame2im(getframe(gca));

% hmm what settings did i use?
assumedmap = parula(16); % the exact same
assumedrange = [1 16]; % the exact same
B = flipud(screenshot);
B = rgb2ind(B,assumedmap);
B = interp1([0 size(assumedmap,1)],assumedrange,double(B),'linear');

% let's say you wanted it to be the original size
B = imresize(B,size(A),'bicubic');

% compare the two side by side
image([A B])
caxis(actualrange)
colormap(actualmap)
colorbar

That looks like complete junk. Could we try to back-calculate the vertex data from the face colors? Maybe? Could the gridlines be suppressed somehow? Possibly, yes.

I mentioned that scale matters. What if we did this same thing, but with a larger plot? Let's say we plotted some 64x64 matrix. Let's also say we turned off the edge coloring so that it didn't pollute the captured image.

So larger-scale features do get retained, but small details and exact locations are lost.

TL;DR

The main point here is that using screenshots as a source of data should seem like a really poor fix for a problem that should have been avoided in the first place.

Answer 2

[Sameer Pujari]

Hello Xin Liu

From the given info, I understand that you want to get RGB image back after you have visualised with a known colormap.

To do that you have to first get the colormap of orignal rgb image

RGB = imread('image.jpg');
[X,cmap] = rgb2ind(RGB,65536);

Then you can get the original image back using

X = ind2rgb(X,cmap);

The same can be done by using "colormap" function for the figure

RGB = imread('image.jpg');
[X,map] = rgb2ind(RGB,65536);
figure
image(X)
axis off
axis image
colormap(parula) %image will be visualized in parula colormap
colormap(cmap) %image will be visualized in orignal rgb colormap

I hope this information helps you!

[Xin Liu]

Dear Sameer,

Thanks very much for your answer. I've tried your solution, but I did not get the correct result as expected.

Maybe I did not explain my purpose clearly before, so I re-edit the question, and give an example.

I hope it will facilitate your understanding.

[Sameer Pujari]

Hi Xin Liu

I have attached a code according to your revised question.

% Saving current figure as a MATALB figure and saving orignal colormap 

raw_data = rand(5,5);
figure;
imagesc(raw_data);
axis image off;
cmap=colormap; % saves orignal data colormap
colorbar;
title('saved image');
savefig('saved_img.fig');

% Changing colormap and saving the jet colormap figure

colormap(jet); % changes colormap to jet
savefig('saved_img_jet.fig');
close(gcf); % closes current figure

% Opening saved figure and changing the colormap to orignal

openfig('saved_img_jet.fig'); % can open fig in another script file if required
colormap(cmap); % applying orignal colormap to jet

I hope this helps you !

[Xin Liu]

Hi Sameer,

Thank you for your code. You may misunderstand my purpose. My purpose is to get the original 2D MATRIX DATA rather than the original image. I saved this figure as a .jpg file and then I want to get the raw 2D MATRIX DATA from this .jpg file according to its colormap.

For example, a certain element in the raw MATRIX DATA is 0.5, and it was mapped to a rgb value of (0.3,0.5,0.8) in the saved .jpg file according to the 'jet' colormap. My purpose is to obtain 0.5 from the rgb value (0.3,0.5,0.8) according to the jet colormap.

Answer 3

[Saeed]

Hi Liu,

Have you found the solution?

[Image Analyst]

@Saeed the solution is to use rgb2ind. You need to have the colormap first of course. If you don't and have to extract it from the image, see my demo

https://www.mathworks.com/matlabcentral/fileexchange/113995-thermal-image-color-to-temperature?s_tid=srchtitle

where I transform a pseudocolored thermal image into a gray scale temperature image by extracting the colorbar and image from the screenshot.

[Saeed]

Hi @Image Analyst, thank you for your answer. I have an RGB colormap with the known colorbar range (min and max). I just need to get back to the 2d matrix that was first given to the pcolor() to create such colormap. You mean I can get it, yeah?

[Image Analyst]

You have a pseudocolor image, a colormap, and the underlying gray scale (indexed) image. If you have any two of those, you can get the third.

[Saeed]

Thank you @Image Analyst and also @DGM for your comment.