The blog Neurophilosophy has a review of the connection between space and time, and between direction and past or future. (here) It is worth reading in full as I am only talking about a very little part of the review.

We know that imagining a future event is dependent on memory, because patients with amnesia cannot imagine new experiences. It involves piecing together fragments of past experiences to generate a plausible simulation of what might happen.

It seems that memory and imagination/planning use the same basic mental processes. There is the same framework for consciousness of now, of memories and of ‘what if’ scenes. Consciousness is used to create memory, memory is used to produce future imaginings and short term projections are use to produce the consciousness of now. Only what is found in consciousness is stored in episodic memory. We know that stimulations of the future and imaginings are constructed from memory scrapes. And our conscious experience is a slight prediction. The difference between our current consciousness and recalled memories or imaginings seems to be in the amount of detail and the vividness of the experience. I presume that both the detail and the vividness is due to the availability of the actual sensory data in the sensory cortex areas that is only there for a few moments.