Noisy result.

[JustinhoCHN]

The model was trained for 134 epochs, unpaired learning -- fivek and hdr dataset, I check the test_img for every epoch, almost like these noisy image:

All the hyperparams remain the same as the origin, can anyone give any insight about this situation? @nothinglo

[JeremyAlain]

Hi justin,
Maybe have a look at this bug i found, it might help!
#36

[nothinglo]

Please check your image format. It should match to the image reading method.

[JustinhoCHN]

ahhhh, thank you for your remind, I printed the image array (tiff format image), and get 20000~30000 pixel value, I think that's why I got the noisy result! @JeremyAlain

[JustinhoCHN]

So if I use cv2.imread(filename, -1), I'll get the uint16 data, that's tiff original pixel value, if I use cv2.imread(filename, 1) instead, the pixel value range from [0, 255], the precision will be reduced, will it affect the final result? @nothinglo

[nothinglo]

I think the precision is ok but you should make sure that after pre-processing you use [0, 1] range in the training.

[JustinhoCHN]

You already did the normalization step for us right? In DATA.py line 193 and 194:

self.input1_src = tf.placeholder(tf.as_dtype(FLAGS['data_input_dtype']), shape=[b, FLAGS['data_image_size'], FLAGS['data_image_size'], FLAGS['data_image_channel']])
self.input1 = tf.cast(self.input1_src, FLAGS['data_compute_dtype']) / self.input1_src.dtype.max

Especially in line 194, first casting the datatype from uint16 (tiff image) to np.float32 ('data_compute_dtype'), and then divided by 65535 ('self.input1_src.dtype.max'), it should be range from [0, 1]. We don't need to do any additional normalization step right? @nothinglo

Even reading the tiff image with cv2.imread(filename, 1), I still got these noisy test image, I'm sure the training set tiff images are readable. @JeremyAlain

[JustinhoCHN]

May I ask your tif images are 8 bits or 16 bits? @nothinglo

[JustinhoCHN]

I found the problem, if we export the fivek dataset to tif format in 16 bits using lightroom, opencv will write the images in 8 bits during the testing, the testing image will be noisy, we should prepare the 8 bits dataset rather then 16 bits. @JeremyAlain

Therefore the correct dataset preparation pipeline is:

1. import all fivek dataset into adobe lightroom.
2. export all images in the following settings:
   
3. if you want to use flickr hdr data, download them first, and then do the step 1 and 2.
4. if you want to use expert C's images as label, double click the /fivek_dataset/raw_photos/fivek.lrcat, it will load the whole modify record from 5 experts, and you should see the expert C's images in the left side, and then do the step 1 and 2 again:
   

Feel free to add it to readme, maybe it can help other person. @nothinglo

Here're my hdr result, this model works so great!

[JeremyAlain]

@JustinhoCHN Thank you very much for your elaborate explanation. i think @nothinglo should add this to the readme, because this is something I have also experienced a lot with and still have not found out which solution they use.

One thing I would like to ask,

1. @JustinhoCHN why do you exactly use ProPhotoRGB and not sRGB? I am not totally sure which one to use but I thought the more standard one is sRGB. Also if you want to transform the RGB color space into other color spaces such as CIELAB, you need it in sRGB.

2. @JustinhoCHN Can you confirm that for your experiments you also used InputAsShotZeroed for your original images? (as I describe here What are the original images?  #34)

[JustinhoCHN]

@JeremyAlain My understanding:

1. According to the official fivek dataset website, they export the images in ProPhotoRGB mode, so I do the same, but I haven't think of other mode yet, you can try and see what you'll got.
2. I'm not 100% sure, but the InputAsShotZeroed collection looks like the original images, so I use them as input.

[JeremyAlain]

1. You are right, I didn't see that.
2. OK good to know that you also use that.

[nothinglo]

Thanks all. I will add this issue to the README.