Also, follow up:
How is the Antikythera mechanism an "ancient computer," but not a clock or other mechanisms used back then? Surely somebody didn't"invent the computer" in ancient Greece and then humanity just forgot it for a thousand or two years and never made anything similar.
I feel like I'm missing one of three things: either a "computer" is much simpler thing than I thought, or the Antikythera mechanism is more complicated than I think, or it's just hyped up pop history fun facts
Early computers used a variety of devices and techniques for programming, input, and output, depending on what was available, affordable, and suited to the purpose of the computer. I'll confine myself here to digital computers of the post-war "blinky-lights" type (which, granted, is still pretty vague) because I think that's mainly what you're thinking of. Hopefully someone else will address analog devices and the theory of computing, or perhaps I'll come back later. That's a whole different issue.
The earliest I/O medium for digital computers was punched cards. Descended from automated weaving technology and adapted from mechanical punched-card calculators, these were cardstock paper about the size of a letter envelope with holes punched in it on a grid to represent information in binary. A punch would be a 1, no punch a 0 (or the other way around, depending on the system). One card might carry around 100 modern bytes. A large number of these would be read from a stack to program and/or provide input to the computer. A programmer or data entry tech would carefully punch a sequence of cards with a manual punch machine with a program or data, and then hope that no mistakes were made... and that no one dropped the stack of cards on the way to the computer. Many computers also had a card punch device, and could output data on cards as well. (The output cards might be later run through a separate machine for converting to typed output.)
This technology meant computing was done in "batch processing", where you dropped off a program to run and got the results back later. Interactive use would require a different method of interaction.
Very early machines would be programmed literally by rewiring the computer, or by flipping a series of switches on and off. This was, of course, very cumbersome and better ways were soon found. Input from cards was quickly adapted for programming, though this required fancy features like stored memory!
Output devices could vary according to what the machine was designed to do; while punched cards were common, some had CRTs, panels of lights, or typed output.
The teleprinter (basically a remote controlled typewriter) predated the digital computer, and was also used to interface with early computers. In the earliest models, this would mostly be used for output, but could also be found as an input device where text was the data to be processed (such as in decryption). Eventually the teleprinter would become the main interface to computers, eventually becoming video terminals when those became available. This allowed interactive computing, familiar to us today.
This changed greatly by era.
The earliest digital computers were controlled by punch cards (or tape), which was a data storage medium. Punch cards typically had about 80 columns and 10 or 12 rows that could be punched to encode either character, decimal, or binary data. With a stack of punch cards you could input data into a computer which would be written into some section of memory sequentially. You could encode a program in this way and then execute the program, produce data in the form of values somewhere else in the computer's memory which could then be output via some mechanism such as a printer or into other punch cards via the appropriate device.
Let's say you are a computer programmer in the late 1950s, what might that look like? Well, first you need to figure out what your program is actually going to do, so you might spend some time with a notebook writing out the algorithms and the workflow of your program. The result of this might be some mathematical formulas, some plain English, some "pseudocode", some flow diagrams, etc. You would then need to translate this into stuff that could actually run on the computer. So, you dig up your references for the instruction set of the machine and you write out your program at an "assembly language" level, figuring out the exact sequence of instructions and their binary operands that will be executed. At this stage the code might still be somewhat human readable by someone who was familiar with the machine or the instruction set you were using. For example, an addition instruction might be written as "ADD". Then you need to translate this into raw machine code. So you go take your instruction set reference and your assembly language code and you go through and add all of the numerical values for the instructions to replace the mnemonic versions. You do some work to figure out how your code will be laid out in memory, what the appropriate addresses should be and so on and then you finally have a sequence of numbers that makes up your program. Then you have that code translated into a series of punch cards, which you might do yourself or have someone else do for you. If you are very lucky those punch cards are carefully labelled and individually numbered. Then when you have time allocated on the computer you input your program via the punch card reader, you let it run, and then you collect whatever form of output you specified for it to create.
By the 1960s and especially 1970s this process would change by leaps and bounds, and the positive feedback loop of using computers to program computers started to take off. Computers became much more powerful and also smaller versions were created. The first minicomputers (like the PDP-5) were introduced. These were much more like the personal computers that would come years later, though at a level of cost that restricted them to business and scientific use.
One common way to interface with such machines was through a teletype machine. A teletype is an electric typewriter that can respond to signals and then type characters (or move the type head one space or one line or what-have-you in typical typewriter fashion). Unlike simple teleprinters a teletype also has a keyboard which will type characters and also send signals for those characters across the line. By connecting a teletype machine to a computer you can use the machine interactively instead of just in batch processing mode via dumping in piles of punch cards. By using the teletype you could share a context with the computer because everything the computer sent you or that you typed would be known to the computer (because what you typed would be sent to it) and also seen by you (because it would be printed on the paper roll in the teletype). This was a far cry from the full-screen display interactivity that is so ubiquitous today but compared to the era of offline programming and compilation followed by non-interactive batch processing it was a quantum leap. Instead of having to enter in data via punch cards you could do so directly via the teletype. You could enter character data directly or you could enter binary data via some encoding, such as in the form of hexadecimal numbers (base 16, 0-9 then A-F as 10-15 for each digit). You could edit text as file data in the machine. Oh, and you could even save data on hard disk instead of just on punch cards (though that was also a common data storage medium at the time).
So, what might your workflow look like now as a programmer? Well, you probably will still be doing a lot of work on pencil and paper "offline" figuring out how your program will work and what it will do, but once you get to the stage of having written out what it looks like in that mnemonic "assembly language" (or perhaps the step before that) then things are different. You can now use the teletype terminal to input your program as assembly language, in the form of characters in a "text file". You can even edit the code (line by line, which is a pain but doable) if needed, and you can even print it out (either inline in your "session" on the teletype directly or on some other peripheral printer), save it onto a personal 8" floppy disk, save it to tape, etc. Then you can use the computer to assemble the code for you, doing some fairly straightforward symbol replacement and data processing to translate your assembly code in the form of somewhat human readable text into the underlying binary representations of the various instruction op-codes, doing some math to figure out the code's layout in memory and the relevant address offsets and so forth and finally producing an executable binary "program" which you could then save on a hard disk, on a personal floppy, printed out as a "hex dump", or printed out on punch cards. You could run the program locally to do whatever work you wanted the program to do or you could simply be developing the program to be run on a more powerful machine (being loaded via cards, tape, floppy, or what-have-you).
Through the 1970s this process went through yet another quantum leap as technology advanced and as the feedback loop of using computers to program computers escalated even more. You have a combination of the advent of CRT based terminals to replace teletypes (such as the VT50 introduced in 1974), the advent of timesharing operating systems (like multics and unix), the advent of computer networking, and a massive increase in the adoption of high level programming languages (like C and BASIC). With timesharing operating systems you could have multiple users connected to the same mainframe each with their own terminal, each editing their own files and running their own programs that would cooperatively execute simultaneously (by being run in short bursts in time slices before execution was paused and handed off to the next program that needed it) on the same system. With CRT based terminals you could interact much more responsively with the system, with it able to send characters in far less time than with an old teletype based terminal. By the late 1970s the first screen-oriented text editors came about, making it possible to view a full screen worth of a file at a time, scroll through it, and edit it in real-time the way we're familiar with today.
One important note here is that you could potentially have a mainframe computer in a computer room that people were using remotely. Either via terminals in the same building or even via networked connections over long distances. So just as today even if you saw a mainframe or a rack full of servers with no keyboard and display immediately nearby that doesn't necessarily tell you that nobody is using precisely that kind of interface remotely.
As for the Antikythera mechanism it was not any sort of general purpose computer it appears to have been a kind of calendar that could keep track of the position of the solar, lunar, and planetary cycles. It is a kind of "analog computer" which uses gearing to perform algorithmic functions. In the strictest sense even a simple analog watch with two hands is also an analog computer, but the Antikythera mechanism was considerably more complex than an analog watch. It was likely the most complex analog computer that had been constructed for many centuries after it was made.
I see people mentioning punched cards a lot. It should be pointed out that punched cards was very much an IBM / business based thing as used in a commercial environment. In the scientific and systems development world we always used reels of punched tape.
Here is some tape being read through a tape reader attached to a teletype machine. Tape was used to load programs, the teletype to dump out results. You could actually physically edit tape by covering up holes with mylar tape and punching new ones in different places. With experience, programmers became quite adept at actually reading the binary and correcting errors - a skill I spent years acquiring and of absolutely no value today whatsoever!
Setting up values in specific kernel registers was often done by flipping switches to actually set bits to 0 or 1. This is a picture of an F1600B input console that I used to work with every day. The row of coloured switches across the top correspond to the 24 bits of a single register. Physically switching them up or down would set the value to 0 or 1. You would then use other switches to select a different register to set up.
Here is a picture of the set up used for a typical I/O session. You can see the desk with switch console and tape reader and the cabinets containing core memory in the background. The CRTs to the right are a later addition. What you can't tell from the picture is that it was very noisy and extremely cold. Extreme levels of air filtration were needed to prevent dust impacting the disk drives of the time and the low temperatures were needed to improve efficiency and prevent things from overheating.
Even today I still associate coding with wearing gloves.
I think you need to start by defining what a "computer" is and what it's used for.
Computers
Even before the neolithic revolution (start of what we would recognize as cities and agriculture, about 10,000 years ago) people were interested in the movements of the bright lights in the sky. They built large structures that were celestially-aligned on every continent. Maybe they were used to mark seasons for hunting, planting and migration. Maybe they were part of religious worship. But even before provable writing or walled cities people everywhere calculated the movements of the heavens using large structures and inscribed calendars.
With industrialization (1760-), people had a need to calculate underwriting tables for insurance and interest payable on loans. They employed computers (people) using calculus and statistics to produce these tables. They used the same system to construct ballistics tables for artillery in the 19th century, and anti-aircraft sighting systems during WWI and missile launch vectors during WWII.
With the dawn of the space age computers used learnings from missiles to calculate launch trajectories to get objects into orbit and to the moon. After that, the use of human computers declined with the invention of the von Neumann electronic computer (we'll get to that later).
Mechanical Computers
The Antikythera mechanism is the oldest mechanical computer found and apparently was used to calculate point-in-time positions for celestial bodies. Nothing happens in this field for a long time.
Then in the 19th century we see lots of toys made from clockwork, including a famous singing nightingale which inspired Hans Christian Anderson's story. These inspire mechanical adding machines for calculating payroll, programmable looms for making patterned rugs, and, later, slide rules (which are essentially pre-calculated logarithm tables.)
Each of these machines has a specific and unchangeable purpose.
So it gets really interesting when Charles Babbage proposes a general-purpose mechanical computer called the Difference engine. Ada Lovelace figures out how to write "algorithms" for it. But it's still mostly a parlor trick until WWII.
Electronic Computers
During WWII germans used a mechanical computer to encrypt their communications; decrypting them was beyond the powers of human computers or even mechanical computers. So Alan Turing used a network of electronic switches to brute-force decode the German communications.
He then went on to extend Babbage and Lovelace's theoretical work by formalizing the idea of a "general purpose computer" - a "Turing Complete" computer is one that can be programmed to run any program, using some form of addressable memory that holds both data and instructions.
The next strategic problem that was intractable by contemporary methods was designing nuclear weapons. Fission bombs work by using conventional explosives to compress enriched uranium until it reaches critical mass and fissions off into a massive release of energy. The wave front of the initial explosion has to be just-so, or the uranium just gets scattered and doesn't fission. Since they didn't have much uranium, were operating in secret and real-life tests were too dangerous they needed a good mathematical model of the explosive wave front before building an actual bomb.
All of the available human computers were employed to model the wave front but it was just taking too long. So John von Neumann led a team to design an electronic computer based on Turing's work that could run simulations of the explosion. The first one used CRTs (like in an old TV) as memory, but later they figured out how to use magnetic cores strung on grids of copper lines (hence the term "core dump"). With this electronic computer they designed and built the first atomic bombs.
At first these computers were programmed by switches and gave their answers using lights, but soon they figured out how to write longer programs on punch cards and connect the computer to a teletype machine for output.
Modern computers
Modern computers work the exact same way as a von Neumann machine, they just have a lot more switches. This was enabled by the invention of transistors, which replaced other types of switches, in the late 1940's and then x-ray lithography, which allows transistors to be printed that are many times smaller than the width of a hair.
With computers getting faster, cheaper, and more power efficient, more and more applications have been found for them.
Conclusion
There is definitely a through-line from the Antikythera mechanism to the phone in your pocket: