This question has been pondered for hundreds if not thousands of years. This problem is commonly referred to as the moon illusion.
Some have speculated that there is some effect that causes the atmosphere to act like a magnifying glass making the moon look bigger. It turns out that any distortion caused by the atmosphere would actually make the moon look a little smaller.
Most scientists agree that the reason the moon looks bigger is purely in our minds. Our mind interprets the things we see in interesting ways. For instance, if you look at any door frame you can see that it is rectangular. But if you were to sketch the outline of the door frame from the angle that you are looking at it, most likely you would sketch a trapezoid. Your mind adjusts the door so that you perceive it as a rectangle from whatever angle you look at it. That theory is called shape constancy.
Drag the bottom picture of the man around and see how big he looks.
Take a look at the picture above. Both images of the person are the same size, but you perceive the one in back to be bigger. This is because your mind sees the hallway and decides that the back person is farther away that the front person, so your mind adjusts the size of the person to make up for the increased distance. This phenomenon is called size constancy.
Size constancy is happening all the time. If you look down the street and see a sports car about 50 feet away, and behind it, about 100 feet away is a big SUV, you know that the SUV is bigger, even though it produces a smaller image on your eye.
One theory about the moon illusion says that when the moon is near the horizon we perceive it to be farther away from us than when it is high in the sky. But since the moon is actually the same size, our minds make it look bigger when it is near the horizon to compensate for the increased distance.
One way that you can trick your mind out of the moon illusion is to bend over at the waist and look at the moon upside down through your legs.
An alternative explanation holds that the moon illusion is caused by the way our eyes focus on distant and close objects. When we focus on the horizon moon, we focus on the moon at a great distance. The overhead moon lacks visual cues that tell us how far away the moon is, so we focus on the moon as if it was a short distance away. See here for the details of this theory.
So, for now, the right answer is that there is no right answer. But the one thing people agree on is that the moon does not physically change its size or distance from Earth as it moves across the sky. It's all in our heads.
Here are some interesting links: