Well, Sherlock knew. Obviously. But he was just trying to convince John to stay with Mary. Probably because he knew Mary would kill them all if she lost John.
Everyone remember "John can't ever know that I lied to him. It would break him and I would lose him, and Sherlock, I will never let that happen... Know that there is nothing in this world, that I will not do to stop that happening." ??
Well, imagine what would happen if she did lose him, seeing as how far she would go to not let it happen, and all.
(Yeah, also, am I the only one who realises that Mary's emphasis is placed not on JOHN and how he feels and how that actually matters, but instead on how she cannot lose him, because she's just a possession of hers and if she doesnt get to keep her little possession, she's throw a tantrum. Which in an assassin's case means a killing spree. Which in this case means starting with Sherlock. The one person John cares about MORE THAN ANYTHING. IN THIS WORLD. Honestly, this woman's morals are not meant to be questioned at all)
Thank you for attending my TED talk