Photo:TLC

Kody Brown

TLC

Kody Brownis reflecting on theseemingly sudden demiseof his polygamous family.

TheSister Wivesstar, 54, tells PEOPLE that his future is different from what he expected after taking “a major turn here, almost a U-turn” whenthree of his four marriages ended.

“I’m struggling to see what the future holds, maybe because I just haven’t let go enough of the past,” he admits.

Robyn Brown, Meri Brown, Kody Brown, Christine Brown and Janelle Brown from “Sister Wives” arrive at the grand opening of Mike Tyson’s one-man show “Mike Tyson: Undisputed Truth - Live on Stage”

Ethan Miller/Getty

The TLC patriarch notes the sudden crumbling of his marital relationships left him with “a lot of heartbreak” because he had once “loved” the life he was building with his exes.

“In the end," he says, it “felt like everybody else was unhappy because I felt like I was lodging all the complaints, that it just made me cynical. I loved [our situation]. It just seemed like not everybody else did.”

Christine Brown, Meri Brown, Robyn Brown, Janelle Brown, Kody Brown Sister Wives - 2010

Puddle Monkey Prods/Kobal/Shutterstock

Kody continues, “I didn’t want to be the guy that has this family that I struggle with until I’m dead. Then they all get along just great after I’m gone. I don’t understand why we can’t transcend this now. It broke my heart that we couldn’t transcend our differences. It’s part of life. I loved the life we have, and I hope I love the life that I’ll get.”

He called the collapse of his family “a Shakespearean love story that’s a tragedy,” adding, “It’s just we loved and we lost.”

Despite thechallenging times, Kody says that “the optimism must be coming back” to him.

“I remember this great life that we had and what an experience to be a team with these 22 people and just trying to survive the rapids of life and it was great,” he shares. “There’s my optimism. It’s back. I’m seeing my past again and saying, ‘It was lovely.’ It’s changed now.”

He continues, “Maybe I’ll find that optimism for the future. In the end, hopefully we just love each other and let each other move on.”

Never miss a story — sign up forPEOPLE’s free daily newsletterto stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Sister Wivesairs Sundays at 10 p.m. ET on TLC.

source: people.com