Dr Amir Haji Rassouliha
Going deeper with computer vision innovation
In a pool in the Auckland suburb of Remuera, small-scale submarines are getting their first watery glimpses of the world.
The remotely operated underwater vehicles (ROVs) are the creations of Boxfish Research, New Zealand’s only company developing deep-sea submersibles which are used around the world to do the underwater jobs that are too dangerous or difficult for humans.
A recent addition to the Boxfish team is now allowing the ROVs to be equipped with intelligent cameras which leverage computer vision and machine learning to map and measure the underwater world.
In a pool in the Auckland suburb of Remuera, small-scale submarines are getting their first watery glimpses of the world.
Iranian-born Dr Amir HajiRassouliha joined Boxfish around six months ago after helping develop a machine vision platform as a postdoctoral researcher on the Science for Technological Innovation (SfTI) Seed project – ‘A giant leap for small displacements’.
Amir and his SfTI colleagues, Prof. Poul Nielsen, Prof. Martyn Nash, and A/Prof. Andrew Taberner, developed and patented computer vision technology that can measure shape, motion, volume and surface deformation 10 to 20 times more accurately than anything previously described in the scientific literature. The patented technology, supported by Auckland UniServices, is in the process of being commercialised in several applications.
The technology’s potential for intelligent underwater mapping and identification saw Amir snapped up by Boxfish, a six-year-old start-up that was seeking to give its ROVs a competitive edge.
“BoxFish didn’t have the resources, or time to focus on this research. However, because of my work on the SfTI Seed project, I was able to deliver what they needed within a few weeks of joining the company,” Amir says.
Boxfish needed computer vision technologies that accurately captures underwater features ranging from pipelines, dams and oil rig platforms to species of fish, and the spatial dimensions of underwater caves. Developing computer vision to work underwater has its own set of challenges.
“The ROVs use fisheye lenses that have a high level of distortion,” says Amir.
“If you want to do any measurements, you need to correct for that distortion. It was something I could fix quickly, based on my previous experience on the SfTI project.”
The computer vision system sitting behind those fisheye lenses harnesses machine learning algorithms to improve its detection ability as it analyses underwater features.
A video using the Boxfish ROV based on a reconstruction of a train wheel as a test sample.
The ROV cameras could be used in New Zealand to identify fish species and numbers.
Traditionally that has been done with underwater cameras but has required humans to review the footage to extract data manually.
“Often, they’ll put a camera underwater and record video for a few hours to try and see what species are present in a specific region,” Amir says.
“But it requires someone sitting in front of the video watching for species that they know.”
Amir credits his experience as a SfTI researcher with preparing him for applying his knowledge to the Boxfish ROVs.
“The SfTI project gave me an appreciation for how the best research and technology can be applied in the commercial world,” he says.
Working four days a week at Boxfish, Amir spent his fifth day working on projects that also had their genesis in the SfTI project. He has adapted the computer vision system for a medical application with colleagues Prof. Nielsen, Prof. Nash, and A/Prof. Taberner at the Auckland Bioengineering Institute (ABI), and in a collaboration with Dr Yusuf Cakmak and Dr Prashanna Khwaounjoo from the University of Otago.
“Using a camera to record video of a person’s neck, the pulsation of the veins on the neck can be measured using the technology we’ve developed,” says Amir.
That could be used to quickly and accurately measure blood pressure and heart function.
The team is now embarking on a small trial of the technology involving cardiologists including A/Prof Mark Webster and patients at Auckland Hospital.
Another side project between the ABI team and Prof. Simon Holdaway’s team, at the social science department of the University of Auckland, will involve taking images of fragments of Māori artefacts recovered from around the country to map their characteristics and determine whether they share a common origin.
With a community of around 300 researchers working on projects to stimulate New Zealand’s high-tech innovation, SfTI went beyond scientific research to develop capacity in its people to assume leadership roles and to enable commercialisation of science.
“SfTI leadership coaching helped me become more confident in what I was doing, and helped me develop skills to manage teams,” says Amir.
Amir aims to help Boxfish apply its ROV’s computer vision features to a wide range of underwater applications. His side projects show strong potential to be commercialised too, he says.
Top image: Photograph shows UoA Auckland Bioengineering Institute Giant Leap team members: (L-R) Professor Poul Nielsen, Dr Amir HajiRassouliha, Associate Professor Andrew Taberner, and Professor Martyn Nash.