There’s a particular reason GTA changed in visual identity completely when it entered the HD Universe. Now, we know a little bit more about how Rockstar and Take-Two got there.
The original 2D GTA games were built on Windows with Visual C++ 2.0, as vintage as Rockstar could have gotten at the time.
Entering the PlayStation 2/Xbox era and their first generation of 3D games, Rockstar moved to RenderWare. Built by Criterion Games, it was one of the first 3D game engines available at the time, and immediately brought GTA to the cutting edge of technology.
By the time of GTA IV for PlayStation 3 and Xbox 360, however, Rockstar had moved to their homegrown Rockstar Advance Game Engine.
Obbe Vermejj, who was still in Rockstar for GTA IV, shared his insight on why they dropped RenderWare:
Engines like RW really only handled rendering. Modern engines include art pipeline, physics, an editor, postfix and whatever else. So Renderware as it was during the trilogy would not cut it these days. Especially for Indies.
(We stopped using it after SA because it was bought by EA)