Ruling two countries at once would be difficult, especially if the two countries don't like each other much.
I think I need to add the important clarificatory question - are we talking about James VI of Scotland (later James I of England) here? As your question reads James the Fourth.