Silver filling fell out, should I be concerned about mercury posioning?

If there is mercury in it there is in such a small amount that it probably won't hurt you. Not getting the tooth fixed will hurt you.