During a recent interview with David Letterman, the 41-year-old also explained his reasoning for donning the infamous 'Make America Great Again' red cap - bearing the slogan which became an iconic part of Trump's presidential campaign.
"Have you ever been beat up in your high school for wearing the wrong hat?" the "Gold Digger" hitmaker said. "Liberals bully people who are Trump supporters!".
Legendary talk show host Letterman was jokingly sparring with Kanye about the rapper's anxiety when the outspoken musician brought up his relationship with the president.
"This (anxiety) is like my thing with Trump – we don't have to feel the same way, but we have the right to feel what we feel," Kanye said.
He then explained that wearing the MAGA hat is "not about politics," and startlingly claimed that, despite his political views, he doesn't even vote.
In the hour-long special for the second season of Letterman's Netflix show, the father-of-four also talked about his battle with bipolar disorder, his new music and his long-running feud with fellow rapper Drake.