I'm totally willing to accept it as a story premise. But then the story ceases to be science and becomes a miracle tale.
Well, some parts of quantum theory do allow for the idea that time is very much not as straightforward as we perceive it and that perception of time is malleable is well-known (the way it seems to speed up as we get older, for example, even when it remains exactly the same).
I'm wary of going down the route of saying that something isn't science fiction just because it features an extreme extrapolation of ideas. In that case, any story involving FTL at all isn't science fiction, any story featuring aliens isn't SF and we're left with very little that is SF (in terms of recent films, not very much at all, as even THE MARTIAN and GRAVITY feature very loose grasps of the laws of physics).