There are two branches of the Special Edition, both based off of the original 1997 Special Edition. The first branch was done in HD, with new (worse) color-timing. The first set of changes were done in 2004 adding more prequel tie-ins, the second set in 2011 adding changes that would be included in the 3D release. The second branch was done from a new scan of the 1997 Special Edition in 4K, redoing all of the changes made in 2004 and 2011, but with color-timing closer to the 1997 Special Edition.
More information:
For a great overview of all of the changes made to the movies (including audio) (including the prequels), check out Original Thrillogy on YouTube and the playlist embedded below.
Star Wars Visual Comparisons 