'GULAG' is actually an acronym for the body that operated the camps — the Main Camp Administration (Glavnoye upravleniye lagerey). GULAG was initially responsible to/part of the NKVD, the internal security/secret police service, parts of which eventually evolved into the KGB.

At the time Solzhenitsyn was in the camps, and when Ivan Denisovich is set (late '40s/early '50s), GULAG was run by the Ministry of Internal Affairs (MVD), which had taken on the policing/internal security duties of the NKVD after the reshuffling of the USSR's intelligence and state security organs in the late '40s.

Editors note to David Glantz' book (I guess, it's "Colossus Reborn: The Red Army at War" originally) translated to russian (about 1941): "The immediate protection of the camp was carried out paramilitary units were completed on the basis of a free employment; NKVD units were responsible only for the external protection and escort." By NKVD units here escort troops (конвойные войска, konvoyniye voiska) of NKVD are meant. Konvoyniye voyska were later renamed to konvoynaya ohrana (конвойная охрана) - escort guard.

If you're interested in the camps from the point of view of the guards and operators, Mochulsky's Gulag Boss is a pretty interesting read. The basic argument he makes in the memoir is that nobody really wanted to be there — that everyone in a gulag camp is in some sense a prisoner of the same system, though there were big material differences in the circumstances, obviously. The guards and bosses could just as easily end up as actual prisoners, too, if they did the wrong thing, crossed the wrong person, failed to meet the right quota, or just got a bad string of luck when it came to being informed upon. Solzhenitsyn gets this, too, in Gulag Archipelago, though he sees it from the other side, of course.