"The Favourite" star and Taylor have reportedly been dating for more than two years, but have only recently gone public with their romance, as she supported him at the Golden Globe Awards earlier this month (January 2019).
Joe, 27, has also rarely addressed his relationship with the 29-year-old popstar in interviews, and says he thinks most people would do the same as him.
He tells Mr Porter's digital magazine The Journal: "I don't think anyone you meet on the streets would just spill their guts out to you, and therefore why should I? And then that is defined as being 'strangely private'. Fine. But I don't think it is. I think it's normal."
Keeping things private means the rising star has avoided the fate of Taylor's previous beau Tom Hiddleston, who faced ridicule after he was pictured in an 'I Love T.S.' T-shirt and the volume of photos of the couple together in public led to speculation their relationship was staged.
Joe may struggle to maintain his privacy in the future, however, as he looks set to be a fixture on the red carpet during awards season after starring in a string of well-received films, including "The Favourite", "Boy Erased", and "Mary Queen of Scots".
Of the three, "The Favourite" looks set to bag a host of prizes, with Olivia Colman already winning a Best Actress trophy at the Golden Globes and the entire cast winning an ensemble award at the Critics' Choice Awards on Sunday, January 13.
Joe, who stars as Samuel Masham, husband to Abigail Masham, British monarch Queen Anne's courtier and lover, in the film, says its director Yorgos Lanthimos deserves huge credit for putting a new spin on historical filmmaking.
"The thing I like about it is that Yorgos has completely thrown out the textbook on what a period drama is," he explains. "Who knows what it was like back then? It probably wasn't as refined as we think it is, or as it's often performed. It can be quite tidy... the edges can be sanded off a bit."