"Unless otherwise noted, effects caused by the deck are permanent and can only be undone by a miracle or wish spell, the powers of another artifact, or the intervention of a deity."
So, there is certainly an argument for the size change to stick to you.
In fact, there is an argument for him being stuck at large size despite the effects of other size altering magics. Being stuck at large size no matter what you do is part of the "curse" of using those cards.
The actual text of the card:
"The character grows one size category, gaining all the benefits and penalties of increased size. His equipment does not increase in size."
It is unclear if this is meant to be a polymorph effect and use the polymorph size change chart. Or if it is meant to use the monster advancement size change chart. If it is a polymorph effect, then it also means you are immune to almost all other polymorph effects, as the effect can't be trumped by most magics.
Have you changed the size of your weapons/armor on hero labs? They might not change on they're own simply because you changed race/size unless via a spell effect listed in the Adjustment tabs. Try putting in a new medium heavy flail and see if it's still 2d8 (make sure the equipment size drop box has "medium" selected.
Is there a link for online source for these card effects to reference?
I'm not sure where it's stated anywhere if you'd actually get a change to Strength and Dexterity simply for changing size category. Does the card state it's a polymorph effect or enlarge person or something?
http://www.d20pfsrd.com/magic-items/artifacts/minor-artifacts/deck-of-many- things-harrow
The mountain man is the one that changes your size.
I'm tempted to start a FAQ on this, because it also comes up with the Mauler familiar.
When an effect says you gain the benefits or penalties of increased size, what table do you use for the size change? And if an effect uses the polymorph table, is it necessarily a polymorph effect?
We know that Mark intended the Mauler to use a polymorph effect, but if the Mountain Man uses the same logic, you create a character who is immune to being polymorphed. Of course, the polymorph-immune Mountain Man may be intentional.
Ah ok, never knew about the harrow deck of many things.
I don't know if the monster advancement table is ever used for anything besides actual "monster advancement" and creation.
The polymorph stuff I'd assume is used for any magical effect, but I don't really know much about transmutation/polymorph stuff (I stay away from it as a player because it looks too complicated for a fun time for me to use for a character).
But if unstated, I'd assume it's a simple -/+ size modifier to attack/AC/CMB/CMD/skills and no ability score changes, like how one can make a small sized tiefling/aasimar as specified from Blood of Fiends/Angels books, or with kineticists' kinetic form utility wild talent.
Also, the polymorph chart is only intended to bring non-standard sizes to small or medium before modifying them with the actual polymorph spell. Meanwhile, the mountain man card would assume you are going from a standard size to a possibly non-standard.