Poor results with regression network

[poster]

I use NASNet (amongst other networks) as regression network. The output consists of 4 real numbers. The input is an 1536x1536 image resized to 331x331.

The output values describe polynomial coefficients. The first two values are much smaller than the last two values (1e-06 to 1e-02).

I've implemented the network in Matlab and used for the first try 125 training and 53 test images.

As a result, the network output values differ extremely from the desired values. All 4 values are in the region of 1e-01 and 1e00.

What can I do to get better results? More training data? Avoiding image resizing? Changing the network architecture? I've only replaced the classification and fully connected layer by a new fully connected layer (with the proper output number) and a regression layer.

More...