Should I just accept divorce?

This is kind of long but I don't know what to do. My husband and I have been married for a little over a year. He started a new job about 3 months ago and he has a supervisory position where his staff his 95% female. He works for a company that is stationed inside a medical facility. He is one of five male employees in the entire building. Just remember that. One day a month ago we were playing around with a little old mp3 player that records voices as well before I went to work. He put it up in our den and our day continued. He was off from work and I had to work the second shift. Fast forward to the next day. The mp3 player had an old audiobook on it that I wanted to transfer to my computer so when I connected it, I found that he accidently left it on when I left for work the day before. Approximately 15 minutes after I left to go to work, he called a female who was unbeknownst to me. Exactly 2 hours after I left, I could hear another woman in my house. She stayed for about 4 hours and they smoked marijuana together, drank beer together, talked about how badly they wanted to **** each other and trash-talked me like a Jerry Springer episode. I never met this woman and it bothered me that she was in MY home with MY husband. He instantly asked for a divorce when I brought the evidence to his face. I agreed but then he was saying that he was sorry and doesn't want to be without me. After a lot of soul-searching and a super brief honeymoon period, I decided to attempt to move on with my husband. Now I can't stand to be away from my home too long while he's not working. He claims he doesn't talk to the woman anymore but I'm not sure. To me he was too comfortable with her in our house for it to be over just like that plus they work in the same building. During one of our rough times he confessed to me that HR in the building had to speak to several of HIS female employees due to "inappropriate relationships" with him at work. Now he's talking divorce again and I'm almost tired of trying to save what I thought we once had. Now he says that he wasn't wrong because he's a "grown man" and can have "friends" over if he wants. Any married or divorced people have any advice you could spare?