Ideally, not that much material would be returned to Earth from asteroids, and certainly not "whole-ass" asteroids themselves. They would be kept far from an Earth-crossing orbit and mined in some safe orbit for their valuable minerals, and then a lot of those would probably be used in space, such as on the Moon, for constructing manufacturing facilities there.

Ultimately, we should be trying to move a lot of our "dirty" industries offworld, to places where no one cares much about keeping pristine and where nothing lives.

But yeah, The Expanse showed what happens when 1 jerk plays around with asteroid orbits...