More than once I've visited (and so revisited) the notion of magnets inside a holster for pistol retention. Without ever actually trying it :-). Of course the solution to your own situation, is for the magnet to be against the slide (the only part we can count on to be at least metal) at the muzzle end; then there's the difficulty of someone with a stainless slide expecting his pistol to stay in! So makes a nice backup system but not a primary.
I've done the same (thought of it more than once, never built one) with another obvious carrier: the 'right' amount of velcro permanently glued to the slide, and a patch of opposite velcro on the shirt (or ideally a vest) under a coat. A bit noisy unless one only draws to fire; then I'm SURE no one will remember the ripping sound before the shots :-).
On that subject I'm reminded that -- in re-reading the Bond books by Fleming, I've yet to read in them of him even drawing his Beretta or Walther against an opponent. Odd.