This is a long one, so anyone reading should check out the responses to this comment; I needed 4 comments due to character limits, so it's all one long chain.

To get a handle around the intelligence situation, and the surprise attack, we have to look at the lead-up to the conflict itself, as well as what was revealed afterwards.

For one, Israel was still quite confident in its position. This confidence led to mistaken intelligence assessments, coupled with a problem in the intelligence community's structure, that made it think war would not occur. Another factor was incorrect information provided by a variety of sources, which also ruined the Israeli picture of the situation. And the final factor was that the Arab forces were quite clever in the set-up of the war, and avoided past mistakes that had made plots apparent.

The lead-up to the war can help explain why Israel missed the signs of a coming conflict first.

Israel's officials were not unaware that a war would likely occur, at some point in the future, after the 1967 war's end. They knew that the Arabs described that war as a "humiliation", one that would need to be "righted". Israel's Defense Minister, Moshe Dayan, also one of the most popular figures because of the successes in the 1967 war attributed to him, said as early as April 1973 (the war was in October) that the Arab states would prefer war because it would force international intervention and end a political stalemate that developed after 1967. At the same time, when the war would come was a matter of huge debate. The Israeli intelligence services had points of disagreements. In April, Eli Zeira (head of the Israeli military intelligence branch, AMAN) decided there was at best a "low probability" of war. He believed that given the existing situation and the "humiliation" of 1967, the Arab states would not declare war. Mossad, on the other hand, believed that war was possible, but didn't forecast it as likely.

Zeira formulated, at this point, something called "the concept". The phrase is a shorthand for a two-pronged belief held by much of the Israeli leadership at this time, convinced and pushed by Zeira. The idea was that the situation dictated that an Arab attack would not come until:

1. Egypt got Soviet fighter-bombers capable of destroying the Israeli Air Force.

2. Egypt got SCUD missiles capable of hitting Tel Aviv.

The view was that Egypt knew that until it had both of those, it would never win a war, and therefore it wouldn't launch one, since it couldn't regain the entire Sinai. And if Egypt wasn't "in", no other Arab state would be either, since Egypt was the most powerful of the Arab armies. Israel had taken the wrong lessons from the 1967 war, in that sense. They believed that Israeli superiority in the air and ground in 1967 demonstrated the futility of a war, and therefore that no war would result. However, what they missed, was that the Israeli victory was in part because of effective use of its assets, and relied on Arab incompetence in air defense too. Neither was the case by 1973, and arrogance played into it.

Additionally, the "concept" was obsolete when Zeira took over AMAN. Sadat had already told his senior commanders that the strategic assessment on launching a war, which he did indeed hold at one point, was no longer the case. In fact, he told his commanders that they'd have to manage without the SCUDs or the fighter-bombers. So the Israeli view, that Egypt would launch a war only when it had those materials because otherwise it couldn't win back the Sinai, was not only wrong...it was dangerously so, since Sadat had already decided that he would accept merely re-crossing the Suez Canal and establishing a foothold there. That was a far easier goal, because Egypt had improved its anti-air defenses, gaining Soviet-made SAMs which would, while not being able to destroy the Israeli Air Force in its bases, would be able to keep it away from having air superiority. Thus, all of Israel's investments into the air force, which got 52% of the defense budget, were effectively useless in trying to win the war. This failure to prepare for a more limited war was huge in explaining why the Israelis were caught by surprise; they simply didn't contemplate it as possible.

Israeli certainty was not the only problem. The other was that Israel missed numerous signs, and its intelligence services and intelligence structure compounded the problem. For one, AMAN was overconfident. Zeira was not only wrong about "the concept", he believed that if that changed, he'd easily be able to give warning to the IDF in time to prepare. Zeira even said that he could promise five to six days warning in case a war was coming, which would've been plenty of time to respond, as well as at least 48 hours in the worst case scenario of warnings. Basically, AMAN was overconfident in its ability to understand Egypt's thinking, and to get information before an attack. Not only was that incorrect, it was something that the Arab states may have inadvertently played on.

Part of the reason why AMAN was so widely believed, compared to Mossad's less sanguine view of the likelihood of war, was because Zeira had held the consistent assessment for a long time. And part of why that remained consistent and persuasive is because at least twice in the months leading up to the war, Israel saw unusual Egyptian troop movement and responded with calling up reserves and preparing for war. These costly events, setting the IDF on alert, happened despite Zeira repeatedly insisting no war was coming. So Zeira had been "proven right" twice. Some Israelis insisted that the mobilizations had deterred Egypt, but war would've come otherwise, while Egypt actually claimed after the 1973 War that they were intentionally deceiving Israel with these mobilizations, trying to make Israel think that mobilizations didn't mean war. That much remains somewhat unclear, but it certainly strengthened Zeira's appearance as someone who truly understood Egyptian aims and the likelihood of war.

Additionally, the intelligence structure was different too. Mossad was responsible for a lot of foreign intelligence collection, but AMAN handled all military intelligence, so on military matters AMAN was typically seen as more influential. Thus, when AMAN was so certain under Zeira and appeared to have been correct twice, Mossad was more deferential. Zeira became so convinced of his own correctness that he began to trust that he "knew" Sadat, and therefore that any other measures were nothing more than bluffs. And in the lead-up to the war, reliance on AMAN became even more common. Even on October 3, amid Egyptian military build-ups for the war that was imminent and to be launched in just a few days, AMAN presented a "low probability" assessment to the Prime Minister. Mossad's Director? Wasn't even invited to the meeting. After the war, the Agranat Commission tasked with studying the failure recommended not only Zeira's dismissal, but also to make Mossad and Israel's Foreign Ministry develop their own analysis structures, which would help put them on an even footing with AMAN and allow the analysts to disagree with equal ability to access information and also equal credibility, which was important to ensuring that no group was "captured" by wanting to please political entities or a particularly ineffective director. Pre-war, Mossad's analytical abilities were near-zero, so it was less believed than AMAN for that reason too.

Then there were all the signs that Israel missed, also largely because of AMAN's confidence and ability to explain away the signs. For example, on October 4 and 5, the Soviets began to evacuate their nationals from Cairo and sent their ships out of Egyptian ports, in preparation for the launch of war on October 6. The Soviets had been told by Egypt on October 3 that war was coming. The Israeli explanation, provided by AMAN, was that the Soviets and Egyptians had a falling-out, like the one in 1972 where Egypt expelled Soviet advisors. The head of AMAN's Signals Intelligence Section told Zeira that "This can't mean anything except war." Zeira finally started to doubt himself, calling it a "new dimension", but he refused to say outright that war was likely. He said the same type of thing about the military buildup that was happening at the same time.

Israel got warnings from elsewhere, too. Israel had developed "the concept" based on information from someone referred to as "the Source". The Source, code name Babel, was a spy for Israel from 1969-on. He had demonstrated his value to Israel multiple times, been vetted multiple times, and passed every test. The Source was almost certainly Ashraf Marwan, an Egyptian billionaire who married Egyptian President Gamal Abdel Nasser's daughter. So he was son-in-law of the Egyptian leader...and switched to provide information to the Israelis. He continued to be closely involved with Egyptian leaders like Sadat after Nasser's death, and passed plenty of information to Israel.

Continued in a response to my own comment.