We have no way of knowing. Counterfactuals have been explored, and plans revealed, but it's unclear whether they'd have been acted upon. There are, however, some myths that I think should be debunked because they frequently pop up, as well as some misconceptions.

I know that a large part of our army was fighting in the Yemeni civil war

For a long time, it was. In fact, Israel knew this, and in the lead-up to the war, Israeli leaders skeptical of the threat used this precise argument to argue against a preemptive strike. The crisis did not start right away and lead to war, as I'll explain in response to your second question. But it did take around 3 weeks to percolate before shots were fired, and during this time a variety of circumstances changed.

Among them were the situation in Yemen. As recently as 1965, Nasser had kept over 50,000 troops in Yemen, and they were a significant suck of resources. As such, in the years before 1967, Israel had projected that Egypt would not be "ready" for a war until sometime between 1967 and 1970, but believed it was likely to be far closer to 1970. This was on the assumption that Egypt remained pinned down for awhile longer in Yemen.

As tensions rose, however, Egypt began to return some troops from Yemen and stationed them in the Sinai, removing at least two brigades of battle-hardened troops. Thus, there is a common myth that Yitzhak Rabin (Israel's Chief of Staff, i.e. head of the military) did not believe that Nasser wanted war. But this statement, made after the war, was that they did not believe initially that Nasser wanted war because the two divisions he'd sent into the Sinai would be insufficient to attack Israel. Yet by the dawn of war, Egypt had at least six divisions, and two Yemen-transferred brigades, sitting in the Sinai and prepared for war. Ultimately, up to 100,000 troops were situated and ready for fighting Israel, and Syrian and Jordanian divisions were also ready. Israel, a much smaller country, had a similar amount of manpower at the ready as the other countries combined, but couldn't afford to keep them mobilized for nearly as long as Egypt given its much smaller overall population.

I’ve heard that the Arabs might have mobilized in the border because of faulty soviet intelligence.

The crisis initially did begin with a faulty Soviet intelligence report that claimed that Israel was massing troops on the border with Syria in preparation for an attack, around May 13 or 14. No one's quite sure why the Soviets had the report, if it was intentional or a real intelligence failure, and I discussed it in a prior thread. But the crisis began there, and it was false. And there are indications that Egypt and Syria knew it nearly right away, but mobilized their troops anyways, and Egypt expelled UN peacekeepers from the Sinai and blockaded the Straits of Tiran. Scholars pin this on some combination of being unsure if the report was truly false before a series of unfortunate events fed off each other, on fears that the report was about what would happen even if it was wrong about it having happened already, on an escalatory cycle in a time already tense, and on reports that one or more of the parties involved would be able to deescalate but never did (i.e. a game of chicken where neither side backed out).

So while we may never know what would've happened, it is true that around 1/3 of Egypt's troops were in Yemen in 1967, but some were transferred to prepare for war with Israel, and that the Soviets gave a false intelligence report that began the series of events leading to Israel's preemptive strike on Egypt.