Cognex Funds Hardware Accelerated Computer Vision Project

Our group has a long history with Cognex. The company itself is headquartered in Natick, MA, but they have a growing research lab in San Diego. Our group was first “raided” for talent when John McGarry (Sr. VP R&D) hired (my then post-doc) Ali Irturk. Since that time, they have expanded substantially which includes a large number of people with direct ties to the Kastner Research Group (KRG). This includes Isaac Philips (undergraduate KRG researcher and MS student), Janarbek Matai (KRG PhD alumni), and Wireless Embedded Systems (WES) MAS alumni Chris Neuhauser. Several of our current graduate students have also spent time at Cognex including Alric Althoff and Alireza Khodamoradi (also a WES MAS alumni). The picture shows those at Cognex SD research with UCSD ties.

We are happy to continue our research collaboration with Cognex. Their high-speed image sensors (operating at 10,000s frames per second!) provide unique research problems. And we enjoy seeing our ideas transitioned into their products. More specifically, our collaboration has resulted in several major technical contributions including developing hardware accelerated architectures for compressed sensing and investigating novel event based sensors. We look forward to our future research collaborations with Cognex. And hopefully, we will continue to serve as a pipeline for future Cognex employees.

Press releases: UCSD, UCSD Jacobs School , UCSD Computer Science

Using Drones to Automate the Understanding of Mangroves

Our latest Engineers for Exploration (E4E) project aims to use multispectral camera on an unmanned aerial vehicle to help our scientific collaborators better understand the mangrove ecosystems. Research engineer Eric Lo and Nikko Bouck and Brynn Hall (two of our Summer E4E students) accompanied our SIO collaborators from the Aburto Lab to Bahia Magdalena, Baja California Sur to perform an initial survey of the mangroves around the region. This data is currently undergoing processing with the ultimate goal of automatic large scale aerial mangrove classification. We are currently investigating a number of supervised and unsupervised machine learning techniques to perform species level classification. Our ultimate goal is to track changes to the mangrove ecosystems over time.

For more information, see the blog post by Aburto Lab member Astrid Hsu – Mangroves from Above and Below.

This project was featured in thisweek@ucsandiego:

Dr. Dajung Lee Successfully Defends Her PhD Thesis

Dajung is putting the finishing touches on her PhD with a successful defense. Dajung has been working broadly in the realm of hardware acceleration. Her research specifically focused on developing a system that can analyze high-speed cellular images in real-time. Imaging flow cytometry is a technique that enables cellular studies, e.g., determining the presence of cancerous cells, identifying mature stem cells, and characterizing sickle cell anemia. Imaging flow cytometry uses cameras operating at greater than 10,000 frames/sec, and thus requires special hardware to analyze the images. This is especially important to enable cell sorting rather than just cell screening. Dajung thesis details her work that studied algorithmic and hardware design techniques to perform real-time cellular analysis. Her next stop is as a researcher at Intel in San Diego where she will continue work on hardware acceleration (though targeting different application). We will miss both Dajung and Coco (pictured in the “Questions?” slide).

Two Research Papers at Design Automation Conference

"Hi, How Are You?" by Daniel JohnstonOur group had two papers in the Design Automation Conference, which was held in Austin, TX. This is the biggest conference for the design and automation of electronic systems. Our two papers were both related to hardware security.

The first paper “Arbitrary Precision and Complexity Tradeoffs for Gate-Level Information Flow Tracking” was a highly collaborative work spanning N. America (UCSD), Europe (EPFL), and Asia (NPU). This work started during my sabbatical to EPFL (Fall 2015 – Winter 2016). The research looks at the tradeoffs between accuracy and speed for different library elements in our gate level information flow tracking work.

The second paper “An Architecture for Learning Stream Distributions with Application to RNG Testing” described Alric‘s latest research to develop a low complexity hardware cumulative distribution function estimator. This is broadly useful for summarizing the internals of integrated circuits. The paper used monitoring the security of random number generator as an exemplar application, but it is applicable to many other domains.

Both Alric and Andrew did a great job in their presentations. And congrats to all the authors!

Oh, and by the way, next year I’ll be a part of DAC’s “Special Focus Committee”. If you have any thoughts on how to better make the program better (in particular, with respect to security), please get in touch with me. I would like to hear your ideas.

Finally, the picture is an example of “keeping Austin Weird”; it is the “Hi, How Are You?” mural from the great Daniel Johnston.

Wei Hu Begins “Post” Post-Doc Career

“Vinnie” Wei Hu has been in our research group for 4 years across two separate occasions — 2 years as a visiting graduate student (shortly after I moved from UCSB to UCSD) and the past 2 years as a post-doc. During that time he published many of the fundamental papers related to GLIFT and has more recently been the leader of our security research group. As anyone who has worked with him knows, he is a patient mentor and an outstanding researcher. He will certainly be missed. But alas all the best people eventually need to leave and go on to do other great things. Vinnie will be doing this as a Professor at Northwestern Polytechnical University in Xi’an. We look forward to continued collaborations with him and his students.

A Modern Take on Palissy Rusticware

Bernard Palissy was a 16th century French ceramicist known for making large decorative platters from the casts of animals (reptiles, fish, crustaceans, etc.). As part of their class project for CSE 145, Erica Sugimoto & Christopher Chinowth, along with their mentor staff Eric Lo, are recreating his artistic process, but with a modern spin. Instead of using dead animals as a mold, they are using Intel RealSense depth cameras to capture a 3D model of a live animal. This model will then be used by artist Miljohn Ruperto to create a modern version of Palissy ware. This artwork is scheduled to be displayed in the Haus der Kulturen der Welt (House of World Cultures) in Berlin, Germany within the next year.

The first try at capturing the data happened today. We have the cameras and the 3d data capture software ready, we just needed a subject. For that, we enlisted “snake wrangler” Josh Ruffell who brought in three large (4-5 ft) Florida King Snakes. The second of the snakes, who goes by the name “Florida King Snake #2
“, made for the best model. Erica and Chris have a few more weeks before the end of the quarter to post-process the data, and determine the best models for Miljohn.

More information can be found in a slideshow developed by Eric and Chris presented earlier this quarter as part of their CSE 145 class project overview (Palissy Snake Project Overview) or their class project webpage (Palissy Snake Homepage).

The class has 12 other projects. To learn about those, check out the Class Project Webpage.

Special thanks to Intel (in particular Byron Gillespie) for providing the RealSense Cameras.

Increasing the Security of Computer Chip Design

Computer systems are increasingly handling important information and responsible for controlling and protecting critical infrastructures; insuring that they behave in a secure manner is of the utmost importance. Information flow tracking is an important technique for determining the security of a computing system. It works by “labeling” important data and then automatically determining how it will move throughout the microelectronic circuit. This can be used to verify that the hardware and software are behaving in a correct manner, and eliminate costly security flaws.

We have been working on hardware information flow tracking for almost a decade, and we now finally decided it is time to increase the level of abstraction from gate level to register transfer level. Our recent paper at Design, Automation and Test in Europe (DATE) — the premier conference for electronic system design and test — detailed the benefits this approach by showing that formal verification can be made substantially faster, and we can more easily provide tradeoffs between complexity (i.e., verification time) and accuracy. This allows us verify the security of larger circuits which is important as security rightfully becomes a more prevalent consider in the hardware design flow.

The paper “Register Transfer Level Information Flow Tracking for Provably Secure Hardware Design” was authored by Armaiti Ardeshiricham, Wei Hu, Joshua Marxen and Ryan Kastner. Ryan returned to Lausanne, Switzerland to give the talk. It was a beautiful week as evidenced by the picture taken in vineyards in nearby Epesses above Lake Geneva.

Small Underwater Robots Uncover Big Scientific Questions

Our work on localizing small underwater robots was recently published in Nature Communications. The article describes a swarm of little underwater robots, called Mini-Autonomous Underwater Explorers (M-AUE), that can change their depth, but otherwise drift along with currents. These robots were developed in Dr. Jules Jaffe’s Laboratory for Underwater Vision. While they seem very simple, this is allows for experimentation on how ocean currents effect different types of ocean phenomenon. In particular, a swarm of these robots were used to verify, for the first time, the physical–biological interaction leading to plankton patch formation in internal waves. Our role in the project was to determine where each of these drifter moves over the course of the experiment. Typically you can rely on GPS, but that unfortunately does not work underwater. We set up our own version of GPS, using buoys instead of satellites, and acoustic signals instead of radio signals. This complicate things substantially as acoustic signal are very messy when transmitted underwater. Nevertheless, we were able to develop tracking and localization algorithms that played a key role in uncovering these scientific findings.

Technical Paper: A swarm of autonomous miniature underwater robot drifters for exploring submesoscale ocean dynamics, Jules S. Jaffe, Peter J. S. Franks, Paul L. D. Roberts, Diba Mirza, Curt Schurgers, Ryan Kastner & Adrien Boch, Nature Communications 8, Article number: 14189 (2017)

Media coverage: LA Times, KPBS,, Live Science, Cosmos, Extreme Tech, New Atlas, Digital Trends.

MTV Invited Talk

Ryan’s childhood dream has come true – speaking at MTV! Well, not exactly as it isn’t that MTV rather the Microprocessor Test and Verification Conference. But it was still an honor to give an invited talk at this conference in December. Our paper “Towards Property Driven Hardware Security” covers some of our group’s most recent security work and gives a glimpse at some future research directions. The paper defines a paradigm for a hardware security design flow that is focused on specifying important security properties early in the design process, and then using tools to test and verify that these properties hold during the entire design process. This work highlights some of the research efforts by Wei, Alric, and Armita. And given that MTV has moved far away from the good ole’ days of playing music videos (and now mostly plays mind numbing reality TV shows seemingly aimed at those with limited intelligence) this is probably a lot more prestigious. We miss you Matt Pinfield!

Spector: OpenCL to FPGA Benchmarks

FPGAs are notoriously hard to use largely due to the lack of programming infrastructure. Recent synthesis tools that compile OpenCL code directly to an FPGA platform are attempting to reduce this programming burden. As part of our research in this domain, we developed the Spector benchmarks. These provide a number of different applications with tunable “knobs” that can be used to change the resource usage and performance. Furthermore, we have fully compiled each of these benchmarks with different knob setting and reported the results. The benchmarks are available in a repo under an open-source license. See our FPT paper for more information. And congrats to the authors Quentin, Alric, Pingfan, and Ryan.

Links: Spector paper, Spector repo