Reports that the fashion designer, 44, and the former England soccer player had split last year (18) forced the couple to issue a statement, vehemently denying the story.
And in a new interview with the Guardian's Weekend magazine, Victoria said speculation on her relationship with David can be stressful.
"It can get quite frustrating. But I leave it to my PR team. I don't get involved," she explained. "You do have to be quite controlling because people do believe what they read, and when it's completely fabricated, that's really annoying."
The designer, who shares four children with her husband of 19 years - Brooklyn Beckham, 19, Romeo Beckham, 16, Cruz, 13, and seven-year-old Harper - confessed that she wasn't going to let the negative press affect her.
"People are interested in my personal life. And sometimes that works in my favour and sometimes it's things that I don't like. I'm not going to let it get me down," Victoria explained.
The former Spice Girls also revealed that when her kids were little, she used to tell them that the photographers that regularly followed them around were there because their grandparents missed them and wanted to see pictures of them.
"It was a way of explaining it to them when they were too young to understand. They are used to it now. They've all grown up with it. They understand that they have to act in a responsible way because of it. But they will make mistakes - we all do. And theirs will be in the public eye," she explained.
Victoria Beckham: ‘You have to be quite controlling’ https://t.co/CVfxW3Wk9V— Guardian Weekend (@guardianweekend) 19 January 2019
And Victoria was also keen to dismiss rumours that she was returning from the upcoming Spice Girls reunion tour in May.
When quizzed if she would be making a cameo appearance onstage - in person or via a hologram - she firmly replied, "No. Definitely not."
"And I'm sure when I'm there and they are on stage, there will be a part of me that feels a bit left out. Because even after all this, a part of me will always be a Spice Girl," she sweetly added.