Towards Efficient And Robust Reinforcement Learning Via Synthetic Environments And Offline Data