1 is not always a failure (the best gymnast in the world rolling a 1 when trying to jump over a hedge should not fail) and a 20 is not always a success (because a level 1 commoner trying to jump to the moon should not have a 5% chance of success.)
Java Man's got it right, but PossibleCabbage brings up a valid point: you only roll dice when there is both a chance for failure and for success, as well as consequences for failure.
So yeah, 1 and 20. Automatic failure/success on attack rolls and saving throws only.