Any Gilmore Girls fan knows that people have *feelings* about Rory Gilmore’s boyfriends on the show, but Matt Czuchry, who played Logan Huntzberger, knows better than most just how real the Team Jess/Dean/Logan rivalries are. His character was so divisive that people started calling him out in real life.

Appearing on Stephen Colbert’s Late Show this week, Czuchry (now starring on The Resident) said that people started approaching him in the street within a few episodes of his season five debut. “In the first season especially, people would just come up to me immediately … and say, ‘You know, you’re a real a**hole,'” he recalled, laughing.

“I would ask them, ‘Okay, why am I an a**hole?’ and they’d say, ‘You know why you’re an a**hole,'” he said, admitting that he didn’t realize they were talking about his character, Logan.

“Then they’d say, ‘No, no, no, I’m Team Jess or Team Dean.’ I went, ‘Oh! You’re talking about the character on the show.’ I didn’t know what was going on. Literally people would just come up to me and say how much of an a**hole I was.”

Logan had his charms, but many fans felt that he and Rory weren’t well suited for each other — including Colbert, who admitted that he was very much Team Jess.

“You’re bad for Rory,” Colbert deadpanned as Czuchry cracked up. “Please stay out of her life.”

In fall 2016, Netflix’s Gilmore Girls revival ended with the bombshell that Rory was pregnant. Given that she had reignited her fling with Logan in the four feature-length episodes, many think Czuchry’s character could potentially be the father — but he has remained tight-lipped about it. We’ll just have to keep our fingers crossed for more episodes to learn the truth.

Were you team Logan? Let us know @BritandCo!

(Photos via Scott Kowalchyk/CBS + Frazer Harrison/Getty)