I hate my partner’s family

My partner’s family has NEVER been friendly towards me, nor did I expect them to be. We’ve been dating for three years. Initially, we didn’t live together, I visited only during the weekends and didn’t have to be around his family too often. Then my partner rented an apartment, and we had more private space, but his apartment is 15 minutes away from his family’s house. And now I see them much more often than I’d like to. His mother is an insufferable whiny, and his father is an abusive alcoholic with no respect for any woman, including his wife. I’m on good terms with my partner’s younger sister, she’s still in college, and she’s not the real problem. His family is dysfunctional, their dynamics are just sick, and I’m grateful that despite it all my partner grew up a good and loving person. His father is always passive aggressive, and he’s taken to barging in without any warning every second evening. I began to feel uncomfortable living at my partner’s place, always waiting for his family to show up. We’ve talked about it many times, and my partner agrees his family is not an example of a healthy family. He acknowledged my feelings towards them, but they are his parents nonetheless, and he’ll never cut ties with them. I don’t want to ask him to choose between me and them. Yet, on the other hand, I have no respect for any of his parents, and I can’t wear a fake smile for the rest of my life. I mean, they didn’t even remember my name at first! And they showed total indifference when my parents invited them to dinner and never showed up or even excused their absence. I feel like they have never tried to get to know me, and at this point I don’t want any of it... All this makes me feel my wonderful relationship is doomed.