Flirty Husband?

My husband made friends with a co worker, all 3 of us work together, although I'm not there that much becasue it isn't my main job. He had been helping her move, hugging her, posing for pics, playing golf with her ex husband, and even had a secret handshake only the two of them know (no, we are not in junior high) He did all this without telling me about any of it, and when I was around he would stay away from her, I almost never even saw them talking to each other, but then i find out when i'm not around he's a totally different person. Who's wrong here, him for behaving this way or me for being upset?