Lets say for some reason the party attempts to dimension door into a 5ft square, such as a closet.
Based on the shunting portion of the spell, is everyone supposed to suffer it? or does one person make it into the closet as intended?
I perused around, and I didn't find anything explicit in the rules for this. Honestly, I think this is a GM call. The only two "applicable" rules I found might be a Medium Creature occupies a 5ft space, and can weigh between 60-500 lbs, and Squeezing. Honestly, if I had to make a GM call on this, I would allow up to a maximum of 500lbs into the closet, and everyone else would get shunted out. And also, they would be subject to the Squeezing rules. None of this is RAW per your specific question though.
Squeezing
In some cases, you may have to squeeze into or through an area that isn’t as wide as the space you take up. You can squeeze through or into a space that is at least half as wide as your normal space. Each move into or through a narrow space counts as if it were 2 squares, and while squeezed in a narrow space, you take a –4 penalty on attack rolls and a –4 penalty to AC.
When a Large creature (which normally takes up 4 squares) squeezes into a space that’s 1 square wide, the creature’s miniature figure occupies 2 squares, centered on the line between the 2 squares. For a bigger creature, center the creature likewise in the area it squeezes into.
A creature can squeeze past a creature while moving but it can’t end its movement in an occupied square.
To squeeze through or into a space less than half your space’s width, you must use the Escape Artist skill. You can’t attack while using Escape Artist to squeeze through or into a narrow space, you take a –4 penalty to AC, and you lose any Dexterity bonus to AC.
Although, some clever use of Reduce Person and/or Shrink Item would probably help.