So then unity comes along, loads that texture, and re-compresses it into PVR/DXT or whatever compression format your platform uses. Your specular response is now going to look like total crap, and instead of appearing to be a nice smooth hi-poly surface, you'll have banding everywhere. Congratulations, you've just destroyed most of those gradients by quantizing them into 256 values in total instead of 256 values per axis (millions of values). These gradients will be quantized into 256 values per axis, effectively. This map will effectively contain lots of subtle gradients that describe the difference between the high polygon object and the low polygon one. Lets say you modeled a high polygon car and cast it against a low polygon version to produce this normal map. A normal map is not color information, it's data - specifically direction offsets which cannot be fully represented in an 8bpp format.

A normal map encodes a direction in the R/G component of the image (it's usually in tangent space, so the B channel is just the length of the vector). You should only use lossless compressed or uncompressed files as inputs to unity. Optimizing png files before adding them to Unity is a horrible idea - even more so on things like normal maps.

Tinypng crack