There is another answer posted here which touches on the history of negative numbers in mathematical texts, so I want to add a bit about how society understands negative numbers if it is not taught mathematically and when this understanding became prevalent among large segments of people. This occurred when society started educating large swaths of the population in more abstract mathematics, so starting the late 19th and early 20th century, depending on the country.

Why is this? Math is an abstraction that is connected to the real world. This connection is very simple at easy levels (counting) but gets harder the more abstract it is. For instance, fractions make sense when cutting a thing in half but not when looking at discrete items.

Without instruction in abstract education, people generally show only concrete reasoning, which focuses only in how knowledge is connected with the physical world. My favorite example of this was the Soviet psychologist Alexander Luria, who investigated various people groups in Central Asia as the Soviets were spreading their ideology and education throughout Central Asia in the 1930s. These groups had had minimal contact with Russia and the USSR, and they had very limited education.

Luria was interested in how they thought logically and mathematically. He found that there was only concrete thinking and that they could not think abstractly. For example, he gave a logical argument like as follows: "“In a certain Siberian village, all bears are white. Your neighbor visited that village and saw a bear. What was the color of the bear?”.

The people in the cultures without formal logical training would answer "I don't know, I've never been there" or "Why don't you ask the neighbor". They did not see the logical argument and could not conceive of it even when explained. This also applies to negative numbers since they are an abstract category. A negative sheep is not a physical entity but an abstraction, meaning "I owe you a sheep".

It also does not seem like very specific instruction is needed to teach individuals mathematical abstraction. Children in educational settings get this kind of logic pretty easily, before 12 years old, as long as they get to manipulate numbers and see connections between abstract numbers and concrete concepts.

Citations:

Luria: Cognitive Development: Its Cultural and Social Foundations

Also see: https://languagelog.ldc.upenn.edu/nll/?p=481

In many regards, laypeople were actually ahead of mathematicians when it came to accepting negative numbers. The notion of debt requires some sense of negative numbers, and when looking at a ledger that reads “Harry owes 3 sheep. Sarah is owed 2 sheep” it can be very hard to argue that this isn’t a representation of negative numbers regardless of the intent of the writer. If the ledger had said “Harry owns -3 sheep. Sarah owns 2 sheep” then it would be unambiguous that these were negative numbers, and in my personal and scholarly opinion (speaking as a philosopher rather than a historian) there isn’t a good way to systematically draw the line between the two ledgers. I certainly wouldn’t defend the idea that the ledger saying “Harry owes 3 sheep” represents a formal understanding of negative numbers, but I also don’t feel that “Sarah owns 2 sheep” represents a formal understanding of the positive integers! In economies across the globe people got by just fine with notions of credit and debt for thousands of years without any metaphysical angst.

The first recorded use of negative numbers is from a Chinese manuscript called Nine Chapters of the Mathematical Arts that’s hard to date, but is from the Han Dynasty (202 BCE - 220 CE). Some of the material in the manuscript predates the Han Dynasty but there’s no particular reason to think negative numbers do. One very interesting thing about this book is that, in contrast to early western systemic works of mathematics, the focus is on problem solving and not theorem proving. Between that and the fact that the solutions are all “actual quantities” it’s unclear how “actually existing” negative numbers were thought to be or if they were viewed as a useful abstraction for solving real problems. The idea of solving problems with “not actually existing numbers” might sound strange, but we actually have very good documentation of the fact that this was how many mathematicians viewed the complex numbers in Europe for a while: they were useful abstractions for solving polynomial equations, but not “actual” entities. This is where the terms “real numbers” and “imaginary numbers” in their modern usage came from, the terms being coined by Rene Descartes to emphasize that “imaginary numbers” are a mental fiction.

The first systematic account of negative numbers is also from China, from the 3rd century CE. Liu Hui published a book that systematically solved all of the problems presented in Nine Chapters, and developed a relatively robust understanding of negative numbers to do so. To Hui, negative numbers are viewed as a second type of number, in some sense opposite to positive numbers. He describes them using colored rods, a common counting device at the time. Red rods represented positive numbers and black rods represented negative numbers. Black rods could be “added” the same way red rods were (in modern terminology, addition of negative numbers is called subtraction), they cancelled (to use modern terminology) with black rods, and he had an understanding of how to do problems like 5 + -6 as well. I don’t have a copy on-hand and cannot easily find a translation free online, but I do not believe that he had an understanding of statements like 6 x -3 = -18 or -3 x -3 = 9.

The connection between negative numbers and debt is evident in these early sources. While neither Hui nor Nine Chapters tie negative numbers to debt explicitly, the “counting rods” Hui references were real objects used by people to do calculations including using different colored rods to represent debt and surplus. Economic interactions was a major reason to use the rods, especially the black “negative” ones. It’s believed that they were widely used in Hui’s time.

Negative numbers would be independently discovered in India, though the date of the manuscript is unknown. Estimates vary from 200-400 CE to 800-1000 CE. Again, we see negative numbers expressed in terms of the concept of debt. Note that I say “expressed in terms of” very deliberately, as number as an abstract concept wouldn’t be fully developed for a long time. In Ancient Greece, where numbers were geometrically grounded, we don’t see any development of the idea of a negative number despite their intensive investigation of numbers.

Moving forward to Islamic mathematics, in Al-Khwarizmi’s treatise on algebra he discusses numbers as financial values explicitly. He refers to them as “possessions” and uses coins as a unit associated with his numbers, as seen in the following quote (translated “as literally as possible” into English by Jens Høtrup from a “rather faithful” Latin translation by Gherado of Cremona):

But possession and roots that are made equal to a number is as if you say, “A possession and ten roots are made equal to thirty-nine dragmas”. The meaning of which is: from which possession, to which is added ten of its roots, is aggregated a total which is thirty-nine. The rule of which is that you halve the roots, which in this question are five, then multiply them by themselves, and from them 25 are made. To which add thirty-nine, and they will be sixty-four. Whose roots you take, which is eight. Then subtract from it half of the roots, which is five. There thus remain three, which is the root of the possession. And the possession is nine.

The translation I am quoting is from Høyrup 2008. See also Hughes 1986, Høyrup 1998, and Rashed 2007.

Negative numbers would show up ad hoc in Europe starting around the 15th century but not be universally accepted until Leibniz used them foundationally in his invention of calculus.

Ultimately it’s hard to say how laypeople felt about the “actualness” of negative numbers over two thousand years ago, in large part because for most of history the thoughts and words of laypeople were not recorded. People certainly had a working knowledge of negative numbers, at least insofar as they were able to do basic calculations involving debt, before any formal notion of negative numbers was presented by mathematicians. In historical mathematical work, we can often see the impact of everyday mathematics in the way mathematicians framed problems. At the same time, arguments like “show me negative 3 sheep” were advanced by mathematicians in Europe in the 1600s so it seems rather presumptuous to assume that they weren’t considered by laypeople before the year zero.

The history is a well-researched topic, and there isn't much ambiguity or confusion about it. Rather than quoting and rewriting from the sources mentioned here, I'll point to this summary handout:

https://web.stanford.edu/class/me161/documents/HistoryOfNegativeNumbers.pdf

The philosophy behind the question is anything but simple. And the curious thing is that in some ways - but only some - negative numbers were still a conceptual problem as late as the 18th century.

The challenge is how to generalise from concrete arithmetic - take three sheep away from six, or travel so many miles towards home on a return journey - to abstract algebra, where "-x" is a valid representation for certain kinds of information.

Concepts like debt and journey direction have been understood since Babylonian times. So the problem is really about formalising and abstracting the root concepts. Historically there was always an understanding of negative numbers as symbols that could be used in specific kinds of problems, but not as a fully-defined concept map.

It's impossible to explain how this understanding evolved without summarising the history of modern pure mathematics. Unfortunately pure math is very probably - without exaggeration - the most abstract pastime in all of history.

To give a basic and rather misleading taste - there are formal differences between numbers considered as concrete nouns ("three sheep"), numbers considered as verbs (also known as operators - "take away three sheep"), numbers as indices ("the third sheep back starting from here, or forward starting from here"), and numbers as sequence patterns ("the set of countable numbers of all possible sheep or sheep indices").

Pure math is about using symbols to work with these relationships, and academic pure math is a kind of map of the relationships between the relationships. It turns out these meta-relationships have structures of their own, and so on, perhaps indefinitely. So an idea that may seem unnecessarily abstract, like "the infinite set of countable numbers", is a very basic building block for much more complex insights.

So... the full answer is that this wasn't really understood until the 20th century, when mathematicians like Banach [1] defined operator theory and pure math became a coherent and self-propelling academic discipline in its own right, where previously it was more the domain of individual researchers.

The summary near the top gives references for Chinese examples of negative coefficients in simultaneous equations, well before BCE. It took the West a long time to catch up, and negative numbers in basic algebra were only used - with some reluctance - in the 16th century. More complex problems, such as solving equations with negative square roots, were also tackled around the same time [2], but the techniques invented remained a curiosity until they started finding applications in physics in the 19th century.

Overall - negativity in math is a much richer concept than "show me minus three sheep." But it took a long time for the implications to be understood. The problem was making the leap between concrete counting, and a practical philosophy of symbolic operations on the relationships between numbers. The most gifted mathematical minds in history struggled with that leap, so it's no surprise students and teachers still do today.

[1] Théorie des Opérations Linéaires, 1932. [2] Ars Magna, Geralomo Cardano, 1545.