CSE 237D – Embedded System Design
Instructor: Prof. Ryan Kastner (kastner@ucsd)
Time: Tuesday-Thursday, 12:30-2pm
Location: CSE 2154
Office Hours: After class or by appointment
This is a project oriented class on the broad topic of embedded systems. As such, the grading is primarily based on your project, and the homeworks are set up as project milestones to insure that you make progress over the course of the quarter.
Current Class (warning some of these files are quite large):
Topic #1 – Overview of Embedded Computing Systems
Topic #2 – Underwater Sensor Networks
Topic #3 – Green Computing
Topic #4 – Image Processing
Topic #5 – Embedded System Security
Topic #6 – Embedded Computing Processors
Topic #1 – Overview of Embedded Computing Systems
Topic #2 – Image Processing
Topic #3 – Embedded Computing Processors
Topic #4 – Underwater Sensor Networks
Topic #5 – Embedded System Security
Topic #6 – Operation Scheduling
Topic #7 – High Level Synthesis
Topic #8 – Digital Arithmetic
Vision applications are appearing in many embedded systems including robotics, security and human computer interaction. They utilize a number of different computing platforms including FPGAs, GPUs, DSPs, etc.
FPGAs are an excellent device for image computation. My post doc has developed an FPGA platform that can hook up to a variety of image inputs (LVDS, composite, component) with a camera. There are many things that can be here, most revolving around real time image processing. We have a variety of camera to play with (PZT, CMOS) including the monitors displays that are set up around the building and there are many interesting things one could do – real time tracking, filtering, image distortion, etc.
More specific ideas include:
- GPU implementations of different vision algorithms using Jacket – GPU engine for MATLAB. I am particularly interested in object detection, recognition and tracking.
- Utilizing Mentor Graphics Catapult C to implement different vision algorithms.
- Creating a Simulink-like library for Jung’s existing image processing implementations and a simple application using them as building blocks.
- Performing fish detection and classification using Jung’s existing hardware platform. This would be useful to NOAA, Birch Aquarium and Reef Check
Autonomous Underwater Vehicle
Mentors: Chris Barngrover, Thomas Denewiler
The Stingray is an unmanned underwater vehicle that was initially developed by a bunch of undergraduates at UCSD and has since evolved into a much bigger project. Their main goal is to compete in the annual AUVSI AUV contest held every summer. My interests are to turn this into a vehicle that can be used for coastal and harbor explorations. Example applications include mapping coral reefs topology, detect and track harmful chemicals, search boats for unidentified objects (e.g., mines), etc. There are a number of research goals needed for both of these pursuits. These include:
- Develop a more efficient networking software infrastructure that allows the various components of the Stingray (control loop, planning, vision, data logging, battery monitoring, etc.) run more effectively. Much of the initial software is already built but with hacks that are not scaling well. One example would be to utilize the Boost libraries.
- Re-write the networking/messages code. It would be nice to have a proper TCP and UDP server-client library. Then we could use that to build a messaging library on top of the networking library. Right now the networking and messaging is a bit buggy and the two pieces are dependent on each other.
- 2) Re-write the GUI. Clean up the Gtk code or (preferably) re-write using OpenGL. Add the ability to embed video from the robot in the GUI. This would work for live and recorded video/images because the video server takes care of that part. But it would be nice to have two window widgets for displaying that video. Add the ability to display real-time graphs of data in the GUI. This would be data like target yaw vs. current yaw. Then we could see how well our controller was working. Maybe even have a tab (or something similar) where we had ~6 graphs of different things. Or a way to turn on the graphs of certain data when you wanted to see it.
- Improving the camera: Calibration – There are offline methods for determining the intrinsic camera properties using known objects in the scene. That could be useful for future video work. Image collection and labeling – This would be taking the cameras out into the water and recording images/video and then labeling them based on what objects are in the scene. Different light and water conditions would be useful as well.
- Integrate a chemical sensor into the Stingray. Prof. Joe Wang has developed a number of chemical sensor for tracking harmful chemical (e.g., explosives, chemical from runoff). I would like to integrate on of these into the Stingray enabling the Stingray to utilize the chemical sensor to find and track chemicals of interest
- Tune the current image processing algorithms for underwater object detection. We have created a number of algorithms that can find buoys and pipes (for the AUVSI contest) using color classification. We need to enhance these algorithms and further integrate them with the Stingray control systems.
- Enhanced battery monitoring and energy saving schemes. We have code that will monitor the batteries, in theory. I would rather see this re-written so that it hooks into the Linux ACPI system, just like laptops monitor their batteries. Then the Stingray computers will shut down when they are supposed to and we can still get access to the battery information but would be using system calls instead of our own program. Based on this you could turn parts of the system off to save energy when necessary
- Figure out how to use quaternions instead of Euler angles so that we can do flips and barrel rolls. Most IMUs (ours included) have a singularity when some of the angles (pitch and roll, not yaw) become too large because the trig they use ends up dividing by zero. Quaternions avoid this. For example, if the Stingray is sitting level and we want to roll 45 degrees clockwise I just tell the roll control loop to go to 45 degrees and I am done. How would this maneuver be accomplished using quaternions? We can log both Euler angles and quaternions from the IMU and just rotate the Stingray in every dimension so that we could look at when the IMU breaks down (documentation says it’s around 70 degrees) and see if we determine smooth 360 degree rotations using the quaternions.
- Create a image processing box for use in the Stingray and other AUV/UUVs. The Navy is interested in developing a combined optical/acoustic sensing system for detecting things like mines. This would use both visual and acoustic information to identify and inspect potential targets.
Mentors: Dr. Feng Tong, Bridget Benson
We are building a hardware platform that interfaces with a variety of sensors, store and process sensor data and wirelessly transmit the data. The end result is something akin to an “underwater mote”. There are a number of potential projects here including:
- Experiment with different physical layer communication protocols. Currently we are using frequency shift keying (FSK). Other more advance protocols include phase shift keying (PSK), DSSS, OFDM, etc.. We are working on a digital acquisition system that enables us to transmit and record different signals underwater. The first part of the project would be to get this system working. Then you could transmit different signals in a tank, pool, Mission Bay, etc. and analyze their performance. Finally, you could implement this protocol in software/hardware
- Develop and simulate network protocols for wireless communication to conserve energy.
- Implement the DataTurbine software – a streaming data server into the system.
This area has seen a lot of interest in recent years. I have been working on security for FPGAs for the past few years as part of two NSF funded projects called RCSec and 3DSec. The goal of the RCSec project is to investigate security primitives for FPGAs. The 3DSec project aims to use 3D integrated circuit technology for the purpose of securing modern processors. An example of this would be to use a separate computational layer to monitor and insure separation of cache resources. It has been shown that a process can modify the cache entries of another process running some encryption scheme and eventually find the key. I believe that we can use 3D technology to mitigate or eliminate such attacks. There are many other interesting problems that could be solved, including data tagging and tracking, monitoring of shared resources to eliminate side channel attacks, etc.
Mentor: Dr. Ali Irturk
Many financial institutes that trade commodities (stocks, options, …) perform a wide range of analysis on prices and trends in trading. These are often done overnight on large servers. There is the potential for huge benefits (aka making a lot of money) if one could do such analysis faster, e.g. in real time. This is a new interest of mine, so projects in this area are somewhat undefined at this point. I would expect you to study the area in great detail and propose, new, interesting opportunities for research.
Signal Processing for Embedded Devices
Mentors: Dr. Ali Irturk
Design, simulate and synthesize an signal processing core. I’m particularly interested in things like matrix multiplication, matrix inversion, FFT, correlation, QR Decomposition (QRD), QRD-M, Kalman Filters GSIC/MP, JPEG/MPEG compression, etc. I’m also open to other applications. There are many different ways to develop these cores. One natural, incremental way is to first develop C, Java, Matlab or your other favorite high level language. Then you an port that code to some acceleration engine, e.g. GPU or FPGA. This would require architecture specific to optimization, e.g. extracting task level parallelism and add custom functions to speed up computationally intensive parts. The FPGA implementation could use Catapult C to create the RTL HDL. Another route (for those who are more savvy with HDL) is to directly develop, simulate and synthesize a RTL HDL core.
Mentors: Dr. Ali Irturk, Dr. Gang Wang
Part of the material presented in class will focus on the scheduling problem, formulation and effective algorithms to solve the problem. One of my former PhD students has developed a instruction scheduling framework that you can enhance for part of your project. This framework has a set of benchmarks and many different methods for time and resource constrained scheduling. There are several project ideas including:
- Multiple operation to resource mapping. The current framework does not handle the case where a resource can handle multiple operations (e.g. an ALU can do add, subtract, shift, OR). Develop an algorithm to effectively partition operations to resources.
- Integer Linear Program (ILP) formulations: Develop code to generate an ILP solutions to various scheduling problems. The input to your program is a DFG and the output is an ILP formulation that can be solved by CPLEX – a commercial ILP solver. Compare the performance of the various benchmarks and scheduling formulations.
- Enhance the visualization tools for the scheduling results
- Scheduling is a fundamental problem in a variety of areas. The existing framework is highly flexible and is easily adapted. It would be interesting to utilize the framework to tackle other scheduling problems, e.g. multiprocessor scheduling, distributed scheduling, etc.
Dr. Ali Irturk
I am working on a number of preliminary projects related to accelerating biomedical imaging. These include optical mapping of heart tissue, stem cell identification and selection, and radiotherapy. All of these applications are for the most part, currently done in a tedious, non-optimized manner, and involve hours/days of time for manual “computation”. Automation of these techniques could have profound effects on experiments, treatments and therapies for ailments ranging from heart disease to cancer. These projects are all in their early stages and involve understanding the algorithm and translating onto a computation device (GPU, FPGA, multicore processor) with hopes of creating a real-time implementation.
I am open to other projects. However, these should not be “off the cuff”. They should be well developed ideas that you have experience working on. Please contact me early to discuss potential ideas.
UCSD CSE237D – Spring 2011
Nathan Manning, Julian Fessard – webpage
Cory Li – website
Danny Anderson – website
Aruna Ravinagarajan – website
Joon Lee, Karthik Nagabhushana Sanji, Nameeta Patil – website
Eric Yip, Michael Chin, Seemanta Dutta – website
Jason Oberg – website
Ming Jia, Xia Zhang – website
Avinash Anathakrishnan – website
Yashar Asgarieh – website
UCSD CSE237D – Winter 2010
Per Magnus Østhus, Per Christian Corneliussen – webpage
Krishnam Raju Indukur – webpage
Harshit Chitalia, Po-Chao Huang – webpage
Liang Cheng – webpage
Digvijay Dalapathi – webpage
Janarbek Matai – webpage
Christopher Lei – webpage
Germán Alfaro – webpage
Laura Pina – webpage
Derick Johnson – webpage
Hayden Gomes – webpage
Hemanth Meenakshisundaram – webpage
Nima Nikzad – webpage
Kaisen Lin – webpage
Yuan Wang – webpage
UCSD CSE237D – Spring 2009
Priti Aghera – webpage
Chris Barngrover – webpage
John Kooker – webpage
Maulin Patel, Grace Wang, Tyler Netherland – webpage
Edoardo Regini – webpage
Erik Rubow – webpage
Ishani Selvaratnam – webpage
UCSD CSE237D – Spring 2008
Liang (Leo) Chen – webpage
Chun Chen Liu – webpage
Jamie Bradley Steck and JunJie Su – webpage
Frank Zhang – webpage
UCSB ECE253 – Winter 2007
Prerna Prerna, Ajay Ramji -webpage
Sydney Pang – webpage
Vivek Padi – webpage
Brett Brotherton, Nick Callegari, Ted Huffmire – webpage
Rami Yassine, Tyler Barton – webpage
Sean Gordoni – webpage
Adam Volk – webpage
UCSB ECE253 – Winter 2006
Ali Irturk, Shahnam Mirzaei – webpage
Jimmy Lin – webpage
Morten Engen – webpage
Dongwoo Hong – webpage
Banit Agrawal – webpage
Ricardo Sanfelice – webpage
Ryan Avery – webpage
Roopa Chari, Justin Kane – webpage
David Ansari – webpage
Kai Yang – webpage
Adam Brill – webpage