Maternity leave can be a contentious issue for new moms. It can be hard to decide how long to stay away from work with a new baby, and the choice will be different depending on the situation each family is in. And, while a woman’s decisions around mat leave are personal, and need to be based on what’s best for the mom and baby, in the US, that decision can leave a huge impression on her co-workers. A new study finds that no matter what a new mom does with maternity leave, her co-workers might judge her negatively for it, putting working moms in a lose-lose situation.

The new research comes from New York University and the University of Exeter in the UK, and will be published in full this September in the Journal of Experimental Psychology. For the study, NYU professor of psychology Madeline Heilman and Exeter psychology professor Thekla Morgenroth interviewed 296 men and women in the UK and the US about their views on maternity leave.

According to TIME, the study participants were split into three groups and presented with unique, hypothetical scenarios. In the first scenario, a fictional woman mentioned to her colleagues that she’d be taking maternity leave soon, and in the second, a pregnant fictional woman announces she won’t be taking maternity leave. The third was a control group where neither pregnancy nor maternity leave were mentioned. In both the first and second scenarios, the women were judged negatively by the study participants, but for different reasons.

In the first case, where the woman said she was going to take time off from work for a new baby, participants said she seemed incompetent at work. In the second, participants said the woman seemed like a worse parent and undesirable romantic partner. Yeesh.

Overall, the opinions of the fictional women did not vary much based on nationality, gender, or race; pretty much everyone judged the hypothetical new moms harshly regardless of their background. Surprisingly, being a parent didn’t seem to have an impact on how positively participants viewed the fictional women. Most of the study participants (71 percent) were not parents, but parents didn’t tend to rate the women less harshly than non-parents.

According to the researchers, this truly puts working pregnant women between a rock and a hard place with their colleagues. If they take their maternity leave, and spend time at home with their brand new baby, their co-workers will think they’re bad at their jobs. If they decide it’s best to stay at work and have a partner or hired help care for their infant, their co-workers are all walking around thinking they’re a terrible parent and partner. So what’s a woman to do?

Heilman tells TIME that the deeply ingrained stereotypes against women and moms in the workplace make maternity leave stigma a hard issue to deal with. While nothing can solve this issue anytime soon, it’s important to address maternity leave at work in order to make progress. Heilman suggests that women have frank discussions about their decision to use or not use maternity leave, and consider staying in touch with the office while on leave if they choose to take it. She also suggests that new moms just be cognizant of how they’re splitting their home life and work life.

But this places a lot of responsibility on the shoulders of women, who are ultimately being unfairly judged by their colleagues for a personal choice. We weren’t able to get in touch with Heilman for comment for this story, but maternity leave sounds like a topic that managers and other employees can discuss broadly with their teams. Workplaces can help remove stigma by affirming maternity leave choices, and by being involved in transition plans when an employee decides to take maternity leave.

In the end, women who are able to take maternity leave should be able to spend the time away from work focusing on other priorities without judgement. Women who have babies and decide they’d rather stay at work than at home should also be able to make that choice without suffering criticism from colleagues. In order to make work a better place for new parents, and women as a whole, everyone needs to learn to respect all maternity leave decisions.

Do you have thoughts about how to end the stigma associated with maternity leave? Tell us on Twitter @BritandCo.

(Photos via Getty)