In practice, it usually decides to do a garbage collection. The answer varies depending on lots of factors, like which JVM you're running on, which mode it's in, and which garbage collection algorithm it's using.
I wouldn't depend on it in your code. If the JVM is about to throw an OutOfMemoryError, calling System.gc() won't stop it, because the garbage collector will attempt to free as much as it can before it goes to that extreme. The only time I've seen it used in practice is in IDEs where it's attached to a button that a user can click, but even there it's not terribly useful.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…