The simple answer is yes and no. The compass is just a magnet that points to the strongest magnetic lines on it. Those lines can and do shift due to many things including large mountain masses, mountain tops, solar flares, not to mention that currently the Earths magnetic field is currently flipping.

During the flip we will not be protected by the Earths collapsing magnetospere and large solar outbursts could fry us of our electronics. Yes the GPS would die, but the compasses would still basically point into the magnetic field such as it is.

So NO it doesn't matter to you.
These are my own opinions based on wisdom earned through many wrong decisions. Your mileage may vary.