Doing History chronilogically for younger siblings

We are doing Genesis – Deuteronomy & Ancient Egypt this year with my 2nd grader.  We will be doing History chronologically and so next year we will be doing Joshua – Malachi & Ancient Greece.  I have 6 children and my 2nd grader is the oldest.  As I think about when my younger children will be joining in, it bothers me that we will not be starting History at the beginning.  I know that probably sound weird 🙂  I have some kids that will be joining us in the next year and others who will join us in 2 & 3 years, which will be towards the end of the 6 year cycle in Modern History!  Do you guys just keep going and not worry about it?  Just curious!  Beginning at the beginning has just been so great 🙂