Research / Featured research

Vision-Based Navigation and Manipulation

Perception that informs physical decisions. A robot with several ways to move should choose its route by what it will cost, not only by whether it is passable. We learn from camera images alone, and without human labels, how much energy the terrain ahead will demand, and plan with it.

IEEE Robotics and Automation Letters · 2025

Self-supervised cost of transport estimation for multimodal path planning

V. Gherold, I. Mandralis, E. Sihite, A. Salagame, A. Ramezani, M. Gharib

Which way costs the least energy? From camera images alone, the multimodal M4 robot learns to predict the cost of transport of the terrain ahead, labelling its own training data as it drives and running the whole pipeline onboard a Jetson Orin Nano.

Paper Code Video

Campus terrain, the robot's trajectory from above, and the resulting cost-of-transport map with two planned paths — **Seeing energy in terrain.** The terrain, the robot's route, and the cost-of-transport map it learned. A planner on this map prefers the red path along the road to the yellow one across the grass: 59 m instead of 56 m, but less energy spent.

Pipeline from RGB and depth images to pixel-wise cost of transport, local and global bird's-eye-view maps — **From pixels to a map.** A network predicts cost of transport pixel by pixel from RGB-D images. Each prediction is projected into a local bird's-eye view and merged into a global map for planning.

In preparation · 2026

Vision-language-action models for drone navigation in synthetic environments

A. Pantalos, supervised by I. Mandralis and M. Kovač

Can a drone be told where to fly in plain language? We rebuild real flight spaces as synthetic Gaussian splat environments — the DroneHub at NEST, and the flight arena at LSR — and fly a drone through them in simulation under an expert model-predictive controller running acados. Those expert flights then train vision-language-action models such as π₀, fine-tuned with imitation learning, until the policy reaches a target named in a prompt from camera images alone. The aim is to carry the same scene understanding from navigation into aerial manipulation. The work began as a semester project.

DroneHub at NEST

“Navigate to the red target.” The fine-tuned policy flying in simulation, choosing its heading from the prompt and what its camera sees inside the reconstructed DroneHub.

Gaussian splat reconstruction of the DroneHub flight space, with coloured targets on the floor and reconstructed camera poses — **A flight space, rebuilt from photographs.** The Gaussian splat reconstruction used for training, with coloured targets placed through the scene. Expert trajectories flown here become the demonstrations the policy imitates.

More research Ground-Aerial Morphing Robotics · Learning-Based Control · Fluid-Body Interaction and Flow Control