FishSense is a collaboration between Engineers for Exploration, Scripps Institution of OceanographySemmens Lab, and REEF citizen scientists. FishSense provides simple yet effect tool for fisheries assessment. FishSense couples a commercial dive camera with an underwater laser and adds a bit of optics and machine learning to automatically determine fish length and biomass. Congrats Chris, Kyle, Avik, and the entire FishSense team!
Skip connections are valuable for training Residual Neural Networks (ResNets), helping them converge to a better solution faster. But skip connections add overhead to the hardware implementation; they require additional on-chip memory and other resources and larger memory bandwidths. Skip connections are like training wheels on a bicycle – they are helpful for learning but get in the way once the learning process is completed.
Tailor is a hardware-software codesign technique that modifies skip connections to be more hardware efficient while maintaining overall accuracy. Tailor gradually transforms a ResNet to remove or shorten the skip connections while iteratively retraining the network. Tailor can remove skip connections on smaller networks. As the network gets larger, removing the skip connections reduces accuracy. In these cases, Tailor makes the skip connections shorter. Perhaps non-intuitively, this reduces resources as the hardware synthesis tool (hls4ml in this case) will implement these shortened skip connections on local memories, which incurs very little additional overhead.
An original ResNet with skip connections in place. The same ResNet model with removed and shortened skip connections.
Tailor started in CSE 237C class in Fall 2020. That was during the height of the pandemic, so the class was totally online. Co-authors Olivia “Liv” Weng, Gabriel Marcano, and Nojan Sheybani were students in this class. Co-author Alireza “Ali” Khodamoradi and Ryan Kastner were course instructors. Ali came up with the idea and pitched it as a potential class project. The project went through several submissions and many rejections, but each time Liv and other co-authors took the reviewer’s comments, added more experiments, and polished the writing. The paper improved and picked up a few other co-authors along the way. Rejections are frustrating but a common (and rarely talked about) aspect of publishing in top venues. It was a journey that resulted in a very strong set of research results.
Andy is a Research and Development Staff in our group working on hardware security and ML hardware acceleration. He plays a critical role in much of our group’s research. These include hardware security verification projects with industry (Intel, Cycuity, OpenTitan, Leidos, the Semiconductor Research Corporation) and academia (Cynthia Sturton’s group and Calvin Deutschbein). As if this isn’t enough, he also works on the hls4ml project on topics related MLPerf Tiny benchmarks and fault tolerance. To date, he has seven(!) publications at top venues with several more in submission.
Andy started with our group as an undergraduate. After graduation, he continued working on research projects and moved into his R&D staff role. Andy is applying for Ph.D. programs and will undoubtedly continue his research career as a Ph.D. student starting next academic year. We hope he stays at UCSD!
The Smartfin project was highlighted in a ThisWeek@UCSD article. The Smartfin holds a microcontroller, temperature sensor, inertial measurement unit, and wireless radio — all embedded into a surfboard fin. This allows surfers to opportunistically gather oceanographic data in the near-shore environment, which is otherwise challenging for more traditionally sensors on buoys and moorings. Engineers for Exploration students are working this summer as part of the NSF-funded REU Site to solidify the data collection process, and develop in-house ability to manufacture Smartfins in a low-cost and open-source manner.
Francesco Restuccia was invited to give a talk at the June 2022 edition of hardware.io — a conference dedicated towards showcasing novel hardware attacks and training the security community to defend against those attacks. Francesco’s talk details how the popular on-chip communication protocol is prone to attacks against the security and safety of on-chip resources. The attacks take advantage of inadequacies in the protocol, which was developed for high-speed communications, and not necessarily designed with safety and security in mind. For more details, check out his talk in its entiretly.
One of the biggest challenges in hardware security verification is developing formal properties that can subsequently be verified by automated tools. This is a difficult and time-consuming task typically assigned to security verification engineers that must manually sort through hundreds of thousands of lines of a hardware description.
Congratulations to Olivia Weng for being awarded a National Science Foundation Graduate Research Fellowship. The NSF GRF is one of the most prestigious graduate fellowships in the US. The fellowship will fund Liv for the remainder of her PhD allowing her to continue her research on the co-design of efficient, fault-tolerant computer architectures for applications in high-energy physics. One example is the Large Hadron Collider, where physicists need hardware that will process millions of particle collisions per second. Her research will allow their hardware, and the machine learning software that runs on it, to meet these intense computing demands while handing faults that are inherent in such sensors.
Smartfin is an oceanographic sensor–equipped surfboard fin and citizen science program aimed to provide an increase of coastal ocean observations. Smartfins are used by surfers and paddlers in surf zone and nearshore regions to provide valuable oceanographic data in these challenging to sample ecosystems. Smartfin measures temperature, motion, and wet/dry sensing, GPS location, and cellular data transmission capabilities for the near-real-time monitoring of coastal physics and environmental parameters.
Over 300 Smartfins have been distributed around the world and have been in use for up to five years. The technology has been proven to be a useful scientific research tool in the coastal ocean—especially for observing spatiotemporal variability, validating remotely sensed data, and characterizing surface water depth profiles when combined with other tools—and the project has yielded promising results in terms of formal and informal education and community engagement in coastal health issues with broad international reach.
Our recent research article in the Continental Shelf Research journal describes the technology, the citizen science project design, and the results in terms of natural and social science analyses. We also discuss progress toward our outreach, education, and scientific goals. Congrats to Phil Bresnahan and all the authors!
Design space exploration aims to quickly determine the design parameters that yield the best results. In software, the designer must set algorithmic and performance parameters, e.g., thresholds, bounds, and other input parameters that provide the best output in terms of accuracy and runtime. In hardware design, the designer must determine parameters related to pipelining, memory architecture, and data types to give the best tradeoff between resource usage and performance. In both cases, one wants to quickly understand the relationship between the input and outputs and find the Pareto set of designs.
Sherlock is a DSE framework that can handle multiple conflicting optimization objectives and aggressively focuses on finding Pareto optimal solutions. Sherlock integrates a model selection process to choose the regression model that helps reach the optimal solution faster. Sherlock designs a strategy based around the Multi-Armed Bandit (MAB) problem, opting to balance exploration and exploitation based on the learned and expected results. Sherlock can decrease the importance of models that do not provide correct estimates, reaching the optimal design faster. Sherlock is capable of tailoring its choice of regression models to the problem at hand, leading to a model that best reflects the application design space
“Sherlock: A Multi-Objective Design Space Exploration Framework” was recently published in the ACM Transactions on Design Automation of Electronic Systems (TODAES). Congrats to the authors Quentin Gautier, Alric Althoff, Chris Crutchfield, and Ryan Kastner. The Sherlock algorithm was also released as open-source. We plan to use it in the future to tune machine learning models for optimized hardware implementations and tune algorithmic parameters for aerial tracking project. We hope that others will find is similarly useful!
Our large-scale 3D modeling work was featured on the National Geographic Docuseries “Ancient China from Above“. We developed 3D models using drones, multispectral cameras, lidar, and other cutting edge technology, which provided archaeologist Alan Maca new insights to better understand ancient Chinese civilizations.
As part of the production, Ryan and Eric Lo traveled to some of the most remote parts of China including Xanadu (Kublai Khan’s summer palace in Inner Mongolia), the Han Great Wall in the Gobi Desert, and Shimao — a new archaeological site known as China’s Pompeii located on the Loess plateau.
The three part series aired on the National Geographic Channel and is available on most video streaming platforms. The full “Secrets of the Great Wall” episode is available on YouTube. We appear around the 26 minute mark.
