It probably reminds him of something he will never attain.
Do they really "love" each other? Is that really what the "suki" they use is trying to convey? Ugh, it bothers me. I feel like it should be something ala "I like you a lot". But heh I'm not a native English speaker so I might not even know the whole range on the word love.
I'll bet the dad could see her stroking him like a whore in heat the whole time in the rear mirror.