Simulation & perception for heavy machinery
Isaac Sim and MuJoCo twins of forklifts, wheel loaders and dozers, a fused camera-depth perception pipeline, and a see-through bucket view for operators.
Problem
Heavy machines are big, expensive and blind in all the wrong places. Before a single camera is mounted, you need to know what it will and will not see, and whether an operator sitting kilometres away can load a truck as fast as one in the cab.
What I did
I build Isaac Sim and MuJoCo environments for forklifts, wheel loaders and dozers, with interactive sensor-placement tools that show camera overlap, blind spots and LiDAR coverage live. I synchronise camera images and depth in a modular perception pipeline that serves scene understanding, autonomy and real-time teleoperation from the same data. I also prototyped a multi-camera see-through bucket view and perception overlays, so the operator can see past the bucket while loading.
Result
Sensor layouts get decided with coverage maps rather than opinions. The see-through bucket view targets a 10–15% cut in truck-loading cycle time, measured through simulator trials, operator feedback and productivity metrics from live customer sites.