So this depends a bit on what specific definition of "computer games" you have in mind. For the purposes of this post, I'm going to include any electronic video games, not just those that today we would recognize as being "PC" or "computer" games. So I will be including early consoles and games built out of scientific equipment.
So, probably the first example you'd be interested in is a neat little game called Tennis for Two which was developed by a scientist (William Higinbotham by the by) at Brookhaven in 1958. The game was, as I said, built by a scientist at a national lab. It was built out of an oscilloscope and made use of the lab's equipment's ability to model projectile motion with wind resistance. To play, two analog controllers were designed. You can view footage of the game being played here. It really is a neat little game.
Now, you might think this would be viewed as a waste of time or resources, but this doesn't appear to have been the case. You see, even back in the day, folks in science recognized that it was a good idea to promote science to the general public, and so Brookhaven hosted exhibitions where they would show off interesting stuff to the public, especially students. Tennis for Two was developed for, and exhibited at, such an exhibition and was apparently a big hit (for obvious reasons -- then as now high school and college kids love them some vidya). Of his creation, the designer later said that he hoped the device "could convey the message that our scientific endeavors have relevance for society."^1 So, far from being viewed as a waste of time, the game was basically viewed as marketing tool to get kids interested in science and demonstrate practical applications of scientific discoveries. Also, it is worth noting that the game didn't take too much time to make; it only took a few hours to design and a few weeks to make.
The next big game was Spacewar! which was developed by a former RA at MIT (in conjunction with some other folks) on a newly installed computer at MIT. The game is the granddaddy of early space shooter games like Asteroids; you can watch some gameplay here. Once again, this doesn't seem to have been viewed as a waste of time or resources. While the game reportedly took over 200 hours to make, it was something of a hobbyist project among a group of programmers at (eventually) several universities. The game was designed to push the limits of the hardware and modeled Newtonian physics during gameplay. Basically, it seems to have been developed by a bunch of academics interested in seeing what you could do with emerging computer technology. Part of the point of doing graduate level work, then as now, is completing little projects like this that (i) demonstrate a proof of concept and (ii) help you get familiar with working in a given system. Seriously, if you take some computer science classes today, odds are you may end up programming a simple game as a learning exercise at some point.
Finally, we get to the Magnavox Odyssey, which was the first commercially released video game console. This game is extremely primitive, to the point that even if you are familiar with old games like Pong, you might not even recognize it as a video game without some prior knowledge. The game had no graphics beyond two blocks of light on the screen that you could control (you had to literally put a plastic overlay on the TV screen), no sound, and the programming was primitive to the point that it didn't even stop you from cheating (in the "Pong" game, for example, nothing stopped you from just straight up going over to your opponent's side to hit the "ball"). Here is a video with gameplay and here is a hilarious video made on the Odyssey by the Angry Video Game Nerd that, in between toilet humor, talks a bit about the history of the console and demonstrates basically all of its features.
Was this viewed as a waste of time? Well, I can't speak to the public perception (hopefully someone else can chime in here). But obviously from the creator's perspective, it was designed to be a profitable product, which if your running a business, is the very soul of practicality.
Some Sources
NY Times article on Tennis for Two with quotes from the creator
The Ultimate History of Video Games by Steven Kent
All Your Base are Belong to Us by Harold Goldberg
*Edit: Fixed broken link