One thing I've noticed that happens an awful lot is that townies will run half way across the map if a higher priority activity opens up, rather than just finishing up what they are doing.
Ideally, I think a good algorithm would be something like this:
1) Townie needs a new task, check to see what is the highest priority in the queue, and check to see which task is closest to his/her current location.
2) If the highest priority task is no more than x times the distance from the closest task, or the townie is the closest townie to the highest priority task, (s)he'll do the highest priority task.
3) Otherwise, just do whatever's nearest to the townie.
X could be a parameter that the user could adjust from 1 (always do the closest task unless townie is the closest townie to the high priority task) or some high number that effectively would have the game work how it does now (townie always does the highest priority task). This would be especially nice for those chopping/mining/killing trips far from your base, since your townies would never return empty handed just to have someone else walk the distance back out to collect stuff.
Later on, you could improve it to look at everything within a certain radius of the closest task and pick the highest priority task it finds there, but that seems like a considerably more difficult and involved change so it's something for the future.