I've got a wife and two kids. I couldn't walk away from my job without something else lined up. It would be irresponsible. Having said that, I've had several occasions in the past 10 years when I have wanted to walk out the door.
