Charles Dickens said that procrastination is the thief of time.
While I was looking that up, I saw a bunch of other things online about procrastination which pulled me their direction.
You can't write a book when "more important" things are pulling you away.
Just make sure they are more important to you. Is it really how you want to spend your time?
If it is...go for it. You can always, always write that book next year. Well, not that book. You'll have a different idea for a different book eventually. But you can always write that the year after.
Those little characters inside your head might give up on you, and wander off into another person's head.
The answer to procrastination?
Prioritize.