I know James struggled to get this done with parliament, and I imagine the Civil War years certainly didn't help, but what happened in the interim years that made the process take over 100 years?
There have been many examples of countries being ruled separately by the same monarch. The British monarch was also the ruler of Hannover in Germany from 1714-1837, and the present Queen Elizabeth II is also queen of many other countries such as Australia. There is no expectation that countries ruled under a personal union should be joined together as England and Scotland were in the 1707 Act of Union.
u/deleted has a relevant answer here: https://www.reddit.com/r/AskHistorians/comments/fiqhe3/comment/fkjgkvk
u/wishbeaunash talks a bit about how Scotland was governed during the period of personal union here: https://www.reddit.com/r/AskHistorians/comments/fzobd3/comment/fn6djfl
As for why the Act of Union happened in 1707, a number of users have answers in this older thread: https://www.reddit.com/r/AskHistorians/comments/1xssak/what_was_the_popular_opinion_of_the_act_of_union/
There is a lot more that can be said, so hopefully someone else will be able to offer their own perspective and expertise.