Yeah, I guess master key usage is just a little too broad. Opening doors doesn't seem like a huge issue, but I would at least require a supplementary password entry for crucial functionality like disabling elevators.
And in those life and death situations that emergency workers are involved in... would you accept the loss of a life because the emergency worker couldn't get hold of the password in time to save someone?
A second layer of protection for critical features that could cost lives is always a good idea. Have to balance authorized people using them with preventing unauthorized people from causing chaos, maybe a standardized pass code that changed once a month and all rescue workers received. If for instance 3d printers become household items and the key pattern is widely distributed online it could end up being a big problem.
Yeah, the question is really which potential harm is larger: unauthorized access, or the possible failure of authorized access? Then you need to ensure the system is more likely to fail in the less dangerous direction.