I went through this exercise a few years ago and did notice byte differences. But when I did binary compares there was no difference. That is why I think you are just seeing padding differences. At one point I looked at the files with a binary editor and that also convinced me that the files were the same, other than padding.
There were some "interesting" articles in Absolute Sound a while ago where 2 guys went back and forth between wav and flac multiple times and convinced themselves that the sound degraded the more times they did the conversions. It was a very controversial set of articles and, in the end, most people dismissed their "research", but there are still people who seem to believe that even though the conversions are bit perfect they can change the sound. I've given up worrying about it
There were some "interesting" articles in Absolute Sound a while ago where 2 guys went back and forth between wav and flac multiple times and convinced themselves that the sound degraded the more times they did the conversions. It was a very controversial set of articles and, in the end, most people dismissed their "research", but there are still people who seem to believe that even though the conversions are bit perfect they can change the sound. I've given up worrying about it