1) Try to understand their weird point of view
Non-audiophiles can't understand that listening to a good HiFi is the only thing one should do anywhere, anytime.

Their obscured minds believe (hear hear!) there are things like friends, society, work and even... family (!!!) that are of paramount importance when compared with what they simply call "music listening".

Hard to believe, I know, but that's the way they live their useless lives.

For this reason, if someone asks you which are the most relevant things in life try answering:
family, friends, a good job, good health or any other nonsense of this kind.
Don't even try to tell them the truth, cause it will take endless hours to go away and go back to your relieving HiFi system.

2) Use real life methaphors
Non-audiophiles aren't prepared to understand how your life is devoted to formal perfection (in frequency, pace and rhythm).
For this reason, you should learn to translate your feelings using phrases taken from "real world", so they can understand. Some examples:

-- Instead of trying to describe the feeling you get while listening to your system with that new mains cable (based on the Higgs' boson and discovered by an audiophile cable Company years before the CERN LHC experiments!), the extremely dark and silent background paired with a 3D scene that literally surrounds you...
Try saying: Have you ever tried to walk in the woods when the Sun is fading down, when it's light can be barely seen among the trees and an extraterrestrial silence seems to surround you?
Well, my mains cable does all that without having my shoes covered with mud and my neck bitten by mosquitoes.

-- Instead of trying to describe the joy of staying up late, trying to adjust the VTA of your turntable, the sweet sensation of feeling yourself as a part of a whole when the naked cartridge hits the first groove, after 7 hours of trials and errors...
Try this methaphor: Do you remember the feeling you get while waiting the dawn on the seaside?
It's exactly like that, without the need to get a cold and having to explain to your wife where did you spend the whole night.

-- Instead of trying to explain the second mortgage to buy a new mains filter with unobtanium-in-oil audiophile caps...
Try this way: Have you ever played the lotto? More or less it is the same thing.
Only difference is you don't know which is the final prize.