Only you can answer that question, and it sounds to me like you won't be 100% happy with the gun until you "fix" this.
Not to be rude at all, but this is just another one of those cases where you'd like to have enough of us say to you, "Oh, yes, for sure, get that fixed" so you can justify it it in your mind. I think, (for those of us that have been around a while anyway), we've all done something similar at least once in our lives.
There's not a thing wrong with that either, but that's the reality of it, right?
Good luck, and when you get it back from the "gunsmith", show us a before and after photo would you?