An accurate and scalable direction-splitting solver for flows laden with non-spherical rigid bodies