The igm should've never been drawn on you either time. It was a total waste of your time and money both times. If you are getting these blood tests drawn thru your regular doctor I will gladly provide you with an article for your doctor that they should read to better understand why they aren't doing their clients any service by ordering the igm test.
If you tested negative for hsv1 and hsv2 3 months after the last possible exposure then it's very, very unlikely that you have herpes. The blood tests do still miss 1 out of every 10 hsv1 infections but I'd have confidence in your hsv2 result especially.
Congrats and get back to enjoying your life to the fullest