I had an argument with my friend's boyfriend today. Was I wrong?

I was in the art room with my friend (lets say her name is Jane and her boyfriend is Fred). Jane was doing a really large piece of work and Fred was helping her paint it. I was watching because I was fascinated by it. Fred then said to Jane "Why is THAT person watching for?" This guy has been pissing me off ever since they got together. I know he don't want me around but if she asks me to come somewhere with her I will go because she's my friend. Anyway I just snapped and asked him what the f.... his problem was and we got into one big argument. Was I wrong to retaliate? I really do not want to lose my friend over a boy that I don't even like.