While I guess I know a little about Newtons work in Physics, I'll talk about the part I'm most comfortable with, and that is with his discovery of Calculus, and if it would have set us back if he didn't.

Before doing this however, I would like to point out Newton did famously say in a letter (using a quote that can at least be traced back to Bernard of Chartres, the 12th Century French theologian) "If I have seen further it is by standing on the shoulders of giants." This quote shows that while obviously Newton advanced our understanding of the way the universe works hugely, it came as a product of the time, not independent of it. His research was built upon methods, and texts that preceded him, just like it would be in modern sciences. ^1

Anyway, back to Calculus. The answer is no, it wouldn't. If he hadn't made the discovery of calculus it wouldn't have massively set back our understanding of mathematics and physics (in fact, it could have led to more research done in England, but more on that later) because independently at around the same time Gottfried Leibniz, the German Mathematician, had also arrived at a formulation of calculus.

Leibniz

Leibniz made some of the first breakthroughs in this discovery of calculus, beginning in the 1670's. It was on on 11 November 1675, according to his notebooks, and he used integral calculus for the first time to find the area under the graph of a function y = ƒ(x). Around this time Leibniz also introduced many of the symbols that are still used in calculus to this day, such as the integral sign ∫ and dy/dx. ^2 (he used this notation in a letter to Newton in 1677)

By 1676 Leibniz had discovered how to integrate and differentiate any power of x, writing the formula dx^n = nx^n-1 .dx. This was followed by the fact in 1677 he had derived rules for differentiating the sum, product and quotient of two functions and by 1680 had obtained the formula for the length of an arc of a curve and the volume of a solid of revolution as integrals. We know these from his unpublished notebooks, however it was only in 1684 that he published a paper on these discoveries

Newton

Newton claimed to have begun working on a form of the calculus in 1666 (many years before Leibniz), at the age of 23. This is now known to be true, however at the time there was no proof of this other than Newton's word, but even so the majority of participants in the "Calculus Controversy" never doubted this (with the release of Newtons manuscripts after his death, it was shown to be true.) Anyway, in 1666 Newton had begun working on his first form of Calculus, and in 1671 and written "the method of fluxions and fluents" (published 1736) ^3

Newtons method differed vastly from Leibniz, one way being because he attempted to avoid the use of the infinitesimal such as by forming calculations based on ratios of changes, in contrast to Leibniz who made it into the cornerstone of his notation and calculus. Newton developed his Fluxional Calculus in an attempt to evade the informal use of infinitesimals in his calculations.

The Controversy

At first, it didn't look like there would be a controversy at all, and in 1696 it still looked relatively peaceful, with both mathematicians acknowledging each other in some form, and French mathematician L'Hôpital's 1696 book about the calculus from a Leibnizian point of view had also acknowledged Newton's Principia of the 1680s as "nearly all about this calculus" (however he did express a preference Leibniz's notation)

It was only in the 1700's that the controversy started, and the he initial accusations were made by students and supporters of the two scientists, it was only later on that they themselves got involved. This started in 1704, when an anonymous review was published about Newton's tract on quadrature. This review implied that Newton had borrowed the idea of the fluxional calculus from Leibniz, and this was one of the catalysts that led to Newtons supporters doubting that Leibniz had invented the calculus independently of Newton. Newtons supporters such as John Keill jumped to his defense, while continental mathematicians such as Johann Bernoulli did the same for Leibniz.

The support for Newton was much larger, and it appeared there was a large case against him, as was summed up by the Royal Society in the Account of the Commercium Epistolicum. ^4 however they had an bias towards Newton, that tainted the whole affair. The report itself, finding in favour of Newton, was even partially written by Newton! The committee also had never asked Leibniz for his view of the affair.

At the same time, the argument was moving away from the mathematics, and was becoming much more personal. In a letter dated 7 June 1713 Johann Bernoulli attempted to indirectly weaken the evidence by attacking the Newton's personal character (Bernoulli denied having written the letter.) Similar personal attacks came from Newton's side as well, such as attempting to cast Leibniz as a dishonest character.

This came about because when Leibniz made his first visit to England in 1673, he sought out the mathematician John Pell and showed him what he believed was an original discovery. Leibniz was then shocked however when Pell informed him that his "discovery" wasn't, and that it had already been advanced Frangois Regnauld. To avoid the suspicion that he had stolen Regnauld's idea, Leibniz wrote a letter explaining his position and the development of his ideas to Henry Oldenburg, the Secretary of the Royal Society. During the controversy however, this letter had made its way into Newton's hands, and he used this as proof of Leibniz dishonesty. ^5

It has now been shown that while they made the discoveries independent of each other, they were not completely independent at the same time. Leibniz had almost certainly seen some of Newton's manuscripts (which he had admitted were of little or no value.) Both had also leaned heavily on the work of the English Mathematician Isaac Barrow, whose work had contained some ideas of calculus. Newton had also sent a copy of "On Analysis" to Barrow in 1669, which Leibniz may have heard about indirectly through people who knew Barrow.

Results of the controversy

The controversy over who made the first discovery of Calculus led to a rift in the European scientific community that lasted many years, as shown by the fact that the English carried on using Newton's methods, while continental Europe used Leibniz. This set back English development in the area compared to on the continent, as the English stubbornly kept with Newton's geometric style of thinking, which was a much slower and more difficult to use method, while the majority of continental mathematicians used Leibniz algebraic method and pushed the subject ahead at a much more rapid pace, and many subsequent developments in mathematical Physics for a long time came from continental Europe instead.

^1 Gods Philosophers by James Hannam

^2 http://www.math.uh.edu/~shanyuji/History/hh-27.pdf

^3 https://archive.org/details/methodoffluxions00newt

^4 http://www.maths.tcd.ie/pub/HistMath/People/Newton/CommerciumAccount/

^5 http://www.math.rutgers.edu/courses/436/Honors02/newton.html

Other Sources

http://pages.cs.wisc.edu/~sastry/hs323/calculus.pdf

The Calculus Wars: Newton, Leibniz, and the Greatest Mathematical Clash of All Time by Jason S Bardi

In terms of Calculus he wasn't. Both Newton and another mathematician named Gottfried Leibniz are now, after controversy over whether or not Leibniz plagiarized Newton's work, credited with independently inventing calculus at the same time. Newton figured it out c. 1665 while Leibniz arrived at it in 1673 so there is about an 8 year gap between the discoveries. However Leibniz published his discovery before Newton did.

Source: The Newton-Leibniz controversy over the invention of the calculus