NVIDIA Comments On GeForce GTX 970 Memory Allocation Reports
Nvidia sais that the GeForce GTX 970 is equipped with 4GB of dedicated graphics memory. However the 970 "has a different configuration of SMs than the 980, and fewer crossbar resources to the memory system." To optimally manage memory traffic in this configuration, Nvidia says its engineers segment graphics memory into a 3.5GB section and a 0.5GB section. The GPU has "higher priority access" to the 3.5GB section. When a game needs less than 3.5GB of video memory per draw command then it will only access the first partition, and 3rd party applications that measure memory usage will report 3.5GB of memory in use on GTX 970, but may report more for GTX 980 if there is more memory used by other commands, aaccording to the company. "When a game requires more than 3.5GB of memory then we use both segments."
As you realize, NVIDIA admits that the GTX 970 does materially differ from the GTX 980. Despite the outward appearance of identical memory subsystems, there is an important difference here that makes a 512MB partition of VRAM less performant or otherwise decoupled from the other 3.5GB.