They’ve already fallen out a few times…she loves to tease