Judge says he won't reconsider order to remove federal building fence