It has often been stated that we know more about the surface of the moon than the deep ocean floor, and that we have explored less than 5% of the Earth’s oceans. This is despite the ocean plays a substantial role in our climate, food, and health.
This lack of knowledge is not without reason as developing underwater systems has significant challenges. Waterproofing and pressure housings require substantial mechanical enclosures. Communication bandwidth is limited as radio frequency waves and optics – the cornerstone of high-speed communications – propagate only short distances underwater. And murky conditions and the lack of lighting in all but the shallowest waters make traditional imaging difficult.
The research in the Kastner Group enables ocean exploration by developing more efficient wireless communications, creating advanced image processing and computer vision techniques, and building the next generation of underwater robots. We describe our research in each of these three domains in the following.
Wireless Communication
A key limiting factor to underwater ecological research is a method for reliable, low cost wireless communication. Acoustic communication is the most widely used method for wireless underwater communication since both radio frequency and optical communications have minimal range. Unfortunately, commercially available acoustic modems are too expensive, limited in data rates, fail to operate in all environments, and/or require extensive amount of our power [22,23]. Our research aims to make better wireless acoustic modems, thus enabling the next generation in underwater networking.
Our research focuses on two aspects: 1) creating an efficient analog frontend [7,11,15,16], and 2) developing an agile digital signal processor [2,12,13,14,18,19,21]. When used in combination, this results in an adaptive acoustic modem that can handle a variety of transducers and wireless communication protocols. The system can be tuned in time and space to efficiently and effectively wirelessly transmit information in an energy efficient manner.
Our research is driven by the development of acoustic modem prototypes [7,13,15,16,24,25]. The first prototype focused on coral reef environments. This utilized a direct sequence spread spectrum (DSSS) modulation that was tolerant to significant multipath [24,26,27]. This was successfully tested in at the National Science Foundation (NSF) Long Term Ecological Research site on the Island of Moorea, in French Polynesia. This site is focused on studying coral reef ecosystems. Our modem successfully transmitted data in this difficult environment at ranges up to 1 km.
Subsequent prototypes aimed to demonstrate the feasibility of creating a sub $500 acoustic modem. This system was carefully crafted around a cheap acoustic transducer. This ceramic cost less than $20 USD; however it was poorly characterized and unreliable. It had a significant variability both across different ceramics and over time. This necessitated the use of agile electronics that could reconfigure to most effectively operate the transducer most efficiently. These modems where shown effective in field deployments off the coast of San Diego, CA [7,15,16]. The technology received an NSF Small Business Innovation Research grant.
This research in underwater wireless communication was funded by the National Science Foundation, Gordon Moore Foundation, and California Institute of Telecommunication and Information Technology Strategic Research Operation grant.
Image Processing and Computer Vision
Underwater imaging is a key technology for archaeology, military operations, and ecological monitoring. Our research in this domain aims to develop image processing and computer vision techniques to quickly and effectively detect, classify, and create large-scale 3D models of underwater objects. Our research in this space is driven by real-life applications from stakeholders including the Atlantic World Marine Archaeology Research Institute, US Navy, and National Oceanographic and Atmospheric Agency (NOAA).
Underwater Archaeology: We work with a number of underwater archaeologists at sites in the Caribbean, Lake Tahoe, and Pacific Ocean to examine and document underwater artifacts. These include sunken barge wrecks from the 19th Century in Emerald Bay, Lake Tahoe, an English galleon off the coast of Bermuda, and artifacts likely from Dutch ships near Tobago. The primary goal is to create 3D models of these artifacts in-situ. Using a technique call structure from motion (SfM), we can transform hundreds of optical images into 3D models. Currently, skilled technical divers laboriously take these pictures. We are studying how to develop automated techniques either replace or aid these divers in taking the pictures. Furthermore, accurate location information is paramount to create precise 3D models. Therefore underwater localization is an important role, which requires fusion of a number of different sensors including inertial measurement unit, Doppler velocity logger, optical flow, and acoustic ranging (we discuss this area of research in more detail later). The key goals in this research are: 1) creating an extensible and flexible underwater imaging platform for archaeological applications, 2) developing more accurate underwater position estimates by fusing information from a variety of sensors including stereo cameras, a Doppler velocity logger, an inertial measurement unit, and buoys, and 3) incorporating location information into existing structure from motion 3D modeling software to create more accurate and georeferenced underwater models.
Military: The task of detecting mine-like objects (MLOs) in side scan sonar imagery has a profound impact on military operations. The current process involves subject matter experts analyzing sonar images searching for MLOs. The automation of this problem has been heavily researched over the years without a definitive solution that outperforms the manual approach in real world scenarios. Our research implements novel solutions for automatically detecting mine like objects in side scan sonar images, leveraging the recent increase in sonar image resolution, advances in combined computer vision features and machine learning for object recognition, and the first application of a brain computer interface using electroencephalography (EEG) to this problem [1,8,9].
Ecological Monitoring: The quantification of abundance, size, and distribution of fish is critical to properly manage and protect marine ecosystems and regulate marine fisheries. Currently, fish surveys are conducted using fish tagging, scientific diving, ROVs, AUVs, and/or capture and release methods (i.e., net trawls). All of these methods require many man hours and ship time, which is costly and time consuming. Therefore, providing an automated way to conduct fish surveys could provide a real benefit to marine managers. We work closely with NOAA to develop advanced technology in order to make this process more accurate and efficient. We have developed both optical and sonar imaging techniques to aid in this process.
We developed a system for automated fish detection, counting, and classifying fish species of interest. By utilizing our hardware accelerated system [10,17,20], we are able to classify video streams from underwater cameras in real-time. This resulted in faster analysis and provided an important step towards the ultimate goal of completely automating the process to enable high throughput classification of fishes [29].
In the acoustic domain, we created a system that enabled the real-time processing of multifrequency and multibeam echosounders by developing a hardware accelerated digital signal processing system for the computationally intensive multi-frequency biplanar interferometric (MBI) algorithm [28]. The visualization system provides the MBI results during data acquisition to enable real-time inspection of seabed and water column targets, and enhance decisions concerning adaptive survey operations and gear deployment, such as ROV navigation and trawl placement. Our system is currently being used on NOAA ships.
The Office of Naval Research, SMART Fellowship, and NOAA provide funding for our research in underwater image processing and computer vision.
Robotics
Underwater robots have explored the depths of the oceans to broaden our understanding of the world’s largest ecosystem. Yet despite their vast successes, these vehicles still remain underutilized due to their cost, size, and usability. They are targeted either for long missions (e.g., gliders and torpedo-shaped vehicles) or short deployments (e.g., cube-shaped remotely operated vehicles). We aim to develop new class of vehicles; ones that can be deployed relatively easily in swarms, low cost yet robust enough for field deployments, and with significant processing power to perform on-board processing for advanced sensor fusion.
The Stingray autonomous underwater vehicle provides a novel, aesthetically pleasing design with a unique propulsion system. It is compact, enabling access into tight spots. Its sleek, carbon fiber hull makes it light and aerodynamic. A novel propulsion system consisting of two Voith-Schneider and three vertical thrusters enables it to easily maneuver at slow speeds and hover in a similar fashion to a helicopter. The Stingray can also take advantage of the lift generated by the wings to glide like a fixed-wing aircraft [8].
An autonomous underwater explorer (or drifters) is a vehicle that can control its depth by changing its buoyancy, but is otherwise carried entirely by the ocean currents. These drifters, developed and pioneered in the Jaffe Lab, are subjected to the same dynamics as ocean phenomena themselves [6]. When equipped with appropriate sensors, an ensemble of drifters forms a dynamic underwater sampling system. Tracking the positions of this swarm of underwater vehicles is key to understand the Lagrangian dynamics of the ocean that they are uniquely built to sense. In collaboration with Dr. Curt Schurgers, we have developed localization algorithms that utilize IMUs, ranging from known buoys, and collaborative ranging from the drifters themselves to collectively localize the swarm [3,4,5].
The National Science Foundation funds our research on underwater robotics.
References
[1] Christopher Barngrover, Ryan Kastner, and Serge Belongie, “Semi-Synthetic Versus Real World Sonar Training Data for the Classification of Mine-Like Objects“, IEEE Journal of Oceanic Engineering, January 2014
[2] Feiyun Wu, Yuehai Zhou, Feng Tong, and Ryan Kastner, “Simplified p-norm-like Constraint LMS Algorithm for Efficient Estimation of Underwater Acoustic Channels“, Journal of Marine Science and Application, Volume 12, Issue 2, Pages 228-234 June 2013
[3] Jingwang Yi, Diba Mirza, Curt Schurgers, and Ryan Kastner, “Joint Time Synchronization and Tracking for Mobile Underwater Systems“, ACM International Conference on UnderWater Networks and Systems (WUWNet), November 2013
[4] Diba Mirza, Paul Roberts, Jinwang Yi, Curt Schurgers, Ryan Kastner and Jules Jaffe, “Energy Efficient Signaling Strategies for Tracking Mobile Underwater Vehicles“, IEEE International Symposium on Underwater Technology (UT), March 2013
[5] Diba Mirza, Curt Schurgers and Ryan Kastner, “Real-time Collaborative Tracking for Underwater Networked Systems”, International Conference on Underwater Networks and Systems (WUWNet), November 2012
[6] Ryan Kastner, Albert Lin, Curt Schurgers, Jules Jaffe, Peter Franks and Brent S. Stewart, “Sensor Platforms for Multimodal Underwater Monitoring“, International Green Computing Conference (IGCC), June 2012
[7] Benson and Ryan Kastner, “Design of a Low-Cost Underwater Acoustic Modem“, Optical, Acoustic, Magnetic, and Mechanical Sensor Technologies, Krzysztof Iniewski (editor), CRC Press, 2012
[8] Christopher Barngrover, Thomas Denewiler, Greg Mills and Ryan Kastner, “The Stingray AUV: A Small and Cost-Effective Solution for Ecological Monitoring“, IEEE Oceans, September 2011
[9] Christopher Barngrover, Serge Belongie and Ryan Kastner, “JBoost Optimization of Object Detectors for Autonomous Underwater Vehicle Navigation“, International Conference on Computer Analysis of Images and Patterns (CAIP), August 2011
[10] Janarbek Matai, Ali Irturk and Ryan Kastner, “Design and Implementation of an FPGA-based Real-Time Face Recognition System“, IEEE International Symposium on Field-Programmable Custom Computing Machines (FCCM), May 2011
[11] Lingjuan Wu, Jennifer Trezzo, Diba Mirza, Paul Roberts, Jules Jaffe, Yangyuan Wang and Ryan Kastner, “Designing an Adaptive Acoustic Modem for Underwater Sensor Networks“, IEEE Embedded Systems Letters, vol. 3, issue 3, December 2011
[12] Ying Li, Xing Zhang, Bridget Benson and Ryan Kastner, “Hardware Implementation of Symbol Synchronization for Underwater FSK“, IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing, June 2010
[13] Feng Tong, Shengyong Zhou, Bridget Benson and Ryan Kastner, “R&D of a Dual Mode Acoustic Modem Testbed for Shallow Water Channels“, International Workshop on Underwater Networks (WUWNet), September 2010
[14] Feng Tong, Bridget Benson, Ying Li and Ryan Kastner, “Channel equalization based on data reuse LMS algorithm for shallow water acoustic communication“, IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing, June 2010
[15] Bridget Benson, Ying Li, Ryan Kastner, Brian Faunce, Kenneth Domond, Donald Kimball and Curt Schurgers, “Design of a Low-Cost, Underwater Acoustic Modem for Short-Range Sensor Networks“, IEEE Oceans, May 2010
[16] Bridget Benson, Ying Li, Brian Faunce, Kenneth Domond, Don Kimball, Curt Schurgers and Ryan Kastner, “Design of a Low-Cost Underwater Acoustic Modem“, IEEE Embedded Systems Letters, vol. 2, issue 3, September 2010
[17] Jung Uk Cho, Bridget Benson and Ryan Kastner, “Hardware Acceleration of Multi-view Face Detection“, IEEE Symposium on Application Specific Processors (SASP), July 2009
[18] Ying Li, Bridget Benson, Ryan Kastner and Xing Zhang, “Bit Error Rate, Power and Area Analysis of Multiple Implementations of Underwater FSK“, International Conference on Engineering of Reconfigurable Systems and Algorithms (ERSA), July 2009
[19] Bridget Benson, Ali Irturk, Junguk Cho and Ryan Kastner, “Energy Benefits of Reconfigurable Hardware for Use in Underwater Sensor Nets“, IEEE Reconfigurable Architectures Workshop (RAW), May 2009
[20] Jung Uk Cho, Shahnam Mirzaei, Jason Oberg and Ryan Kastner “FPGA-Based Face Detection System Using Haar Classifiers”, International Symposium on Field Programmable Gate Arrays (FPGA), February 2009
[21] Bridget Benson, Ali Irturk, Jung Uk Cho and Ryan Kastner, “Survey of Hardware Platforms for an Energy Efficient Implementation of Matching Pursuits Algorithm for Shallow Water Networks”, International Workshop on Underwater Networks (WUWNet), September 2008
[22] Bridget Benson, Frank Spada, Derek Manov, Grace Chang and Ryan Kastner, “Real Time Telemetry Options for Ocean Observing Systems”, European Telemetry Conference, April 2008
[23] Frank Spada, Derek Manov, Grace Chang, Bridget Benson, and Ryan Kastner, “Real-time Telemetry Technologies for Moored Oceanographic Applications“, Ocean Sciences Meeting, March 2008
[24] Daniel Doonan, Tricia Fu, Christopher Utley, Ronald A. Iltis, Ryan Kastner and Hua Lee, “Design and Experimentation with a Software-Defined Acoustic Telemetry Modem”, International Telemetering Conference (ITC), October 2006
[25] Bridget Benson, Grace Chang, Derek Manov, Brian Graham and Ryan Kastner, “Design of a Low-cost Acoustic Modem for Moored Oceanographic Applications”, International Workshop on Underwater Networks (WUWNet), September 2006
[26] Hua Lee, Tricia Fu, Daniel Doonan, Christopher Utley, Ronald A. Iltis and Ryan Kastner, “Design and Development of a Software-Defined Underwater Acoustic Modem for Sensor Networks for Environmental and Ecological Research”, MTS/IEEE Oceans, September 2006
[27] Ronald A. Iltis, Hua Lee, Ryan Kastner, Daniel Doonan, Tricia Fu, Rachael Moore and Maurice Chin, “An Underwater Acoustic Telemetry Modem for Eco-Sensing”, MTS/IEEE Oceans, September 2005
[28] Pingfan Meng, George R. Cutter Jr., Ryan Kastner, David A. Demer, “GPU Accelerated Post-Processing for Multifrequency Biplanar Interferometric Imaging”, Oceans, 2013
[29] Bridget Benson, Junguk Cho, Deborah Goshorn, and Ryan Kastner. “Field Programmable Gate Array Based Fish Detection Using Haar Classifiers “. American Academy of Underwater Sciences, March 2009