Secure Autonomous Systems

Instructor: Prof. Sibin Mohan, The George Washington University

CSCI 6907/3907 | Fall 2022 | TR 12:45PM - 02:00PM ET | SMTH 115

This course will focus on the state-of-the-art on security for autonomous and cyber-physical systems. The goal is to provide a deep and broad understanding of the design of such systems as they interact with the real world. Autonomous cars, unmanned aerial/ground/water vehicles, robots, etc. are examples of systems that we intend to explore.


Smart City with Autonomous Systems


We will cover topics related to:

Hands-on exercises will explore real hardware rovers as well known, open-source simulation and control platforms such as:

The course will also include an exploration of the state-of-the-art research papers in these domains.

Prerequisites: students should be familiar with one or more modern programming languages such as Python, C++, etc. The hands-on machine problems will involve the setting up of open-source simulation frameworks (detailed instructions will be provided).


Course Staff

Course Details

[* Note: Please message ahead of time and set up alternate slot in case of conflicts for office hours.]

Communication

We will use Blackboard for all course related communications.

Target Audience

Graduate and undergraduate students in CS, ECE, Robotics, AI and even other departments who want to explore topics related to the security, privacy and ethics of autonomous and cyber-physical systems.

Covid

If you suspect you have been exposed to someone with Covid, or are feeling sick yourself, please let me know and I will provide a Zoom link for you to attend remotely. And get some rest!

Grading Criteria

In-class participation 5 %
MP1 [V2X/VEINS] 10 %
MP2 [UAV Security/Gazebo] 10 %
MP3 [V2X Attack/Leaderboard/VEINS] 20 %
MP4 [Hardware Rover Security] 25 %
Paper Reading/Critique 30 %

Conversion from numerical to letter grade is as follows:

6907 3907
95 - 100 A 90 - 100 A
90 - 94 A- 86 - 90 A-
87 - 89 B+ 82 - 84 B+
83 - 86 B 78 - 81 B
80 - 82 B- 75 - 77 B-
77 - 79 C+ 72 - 74 C+
73 - 76 C 68 - 71 C
70 - 72 C- 65 - 67 C-
67 - 69 D+ 62 - 64 D+
60 - 66 D 55 - 61 D
50 - 59 D- 45 - 54 D-
Below 50: F Below 45: F

In Class Participation

I encourage you to ask questions and ask often. Participate in discussions as well. Take notes since that helps with the learning.

Schedule


Week Date Topic Links
1 Aug 30, 2022
Intro to Autonomy, Cyber-Physical Systems and Real-Time
Systems. Discussion on course syllabus and requirements
Lecture Slides
Sep 01, 2022 Lecture Slides
2 Sep 6, 2022
Machine Problem MP I-A: Introduction to VEINS/FMD
Due Date: Sep 16, 2022 [11:59 PM ET]
MP I-A
Sep 6, 2022
Image Classification, YOLO and Attacks

Additional Information:
Lecture Slides
Sep 8, 2022 Lecture Slides
3 Sep 13, 2022
Kalman Filters

"A Novel Side-Channel in Real-Time Schedulers"
by C. Y. Chen, A. Ghassami, S. Mohan, R. Bobba,
R. Pellizzoni and N. Kiyavash, IEEE RTAS 2019.

Additional Information:
Lecture Slides

Scheduleak Slides
Sep 15, 2022 Lecture Slides
4 Sep 20, 2022
"Guaranteed Physical Security with
Restart-Based Design for Cyber-Physical Systems
"
by F. Abdi, C. Y. Chen, M. Hasan, S. Mohan
and M. Caccamo, ACM/IEEE ICCPS 2018.

"TaskShuffler: A Schedule Randomization Protocol
for Obfuscation Against Timing Inference Attacks
in Real-Time Systems
"
by Man-Ki Yoon, Sibin Mohan, Chien-Ying Chen
and Lui Sha, IEEE RTAS 2016.
ReSecure Slides




TaskShuffler Slides
Sept. 22, 2022 Lecture Slides
Sep 22, 2022
Machine Problem MP I-B:Misbehavior Detection in VEINS
Due Date: Oct. 07, 2022 [11:59 PM ET]
MP I-B
5 Sep 27, 2022
Class Discussion on MP I-B.
Sep 29, 2022
Guest lecture by Prof. Bhagi Narahari.

"Integrated Hardware/Software Approaches to
Software Security"
Lecture Slides
6 Oct 03, 2022
Paper Reading List | Attack Papers
Papers List
Oct 04, 2022
Security Classification [Attacks and Defenses]
Lecture Slides
Oct 06, 2022
Behavior-Based Intrusion Detection
for Cyber-Physical Systems

Behavior-based IDS Papers: Additional Related Papers:
Behavior IDS Slides
7 Oct 11, 2022
Machine Problem MP II: Autopilot/Flight Controller
Security
Due Date: Oct. 21, 2022 [11:59 PM ET]
MP II
Oct 11, 2022
Student Paper Presentation:
  • Samantha McDonald's presentation of Miller, Charlie,
    et al. "Remote exploitation of an unaltered passenger
    vehicle.." Black Hat USA 2015.S 91. (2015).
Class Discussion on MP II.



Samantha McDonald's
Slides
Oct 13, 2022
Student Paper Presentation:
  • Connor Burnett's presentation of :
    X. Ji, et al. "Poltergeist: Acoustic Adversarial Machine
    Learning against Cameras and Computer Vision."
  • Ellis Thompson's presentation of :
    J. Lin, et al. "Data Integrity Attacks Against Dynamic
    Route Guidance in Transportation-Based
    Cyber-Physical Systems: Modeling, Analysis,
    and Defense."



Connor Burnett's
Slides


Ellis Thompson's
Slides
8 Oct 18, 2022
Student Paper Presentation:
  • Colton Asnes's presentation of :
    Sathaye, Harshad, et al. "Wireless attacks on aircraft
    instrument landing systems." 28th USENIX Security
    Symposium 2019
  • Yuan Gao's presentation of :
    Wen, H., Zhao, Q., Chen, Q. A., & Lin, Z.
    "Automated cross-platform reverse engineering
    of CAN bus commands from mobile apps".
    In Proceedings 2020 Network and Distributed
    System Security Symposium (NDSS'20).



Colton Asnes's
Slides



Yuan Gao's
Slides
Oct 20, 2022
Student Paper Presentation:
  • Danning Ma's presentation of :
    Eykholt, Kevin, et al. "Robust physical-world attacks
    on deep learning visual classification." Proceedings of
    the IEEE conference on computer vision and pattern
    recognition. 2018.
  • Jonathan Lee's presentation of :
    S. Jha et al. , "ML-Based Fault Injection for
    Autonomous Vehicles: A Case for Bayesian Fault
    Injection"



Danning Ma's
Slides.



Jonathan Lee's
Slides.
9 Oct 25, 2022
Fall Break.
Oct 27, 2022
Machine Problem MP III: V2X Misbehavior Detection
Contest in VEINS
Due Date: Nov. 07, 2022 [11:59 PM ET]
MP III
Oct 27, 2022
Student Paper Presentation:
  • Mushary Alghamd's presentation of :
    Garcia, Flavio D., et al. "Lock It and Still Lose It—on
    the ({In) Security} of Automotive Remote Keyless
    Entry Systems." 25th USENIX security symposium
    (USENIX Security 16). 2016.
  • Miles Grant's presentation of :
    Sitawarin, C., Bhagoji, A. N., Mosenia, A., Chiang, M.,
    & Mittal, P. (2018). "Darts: Deceiving autonomous
    cars with toxic sign". arXiv preprint
    arXiv:1802.06430.



Mushary Alghamd's
Slides.



Miles Grant's
Slides.
10 Nov 1, 2022
Student Paper Presentation:
  • Dev Shah's presentation of :
    Patel, Naman et al. “Adaptive Adversarial Videos on
    Roadside Billboards: Dynamically Modifying
    Trajectories of Autonomous Vehicles.” 2019
    IEEE/RSJ International Conference on Intelligent
    Robots and Systems (IROS) (2019): 5916-5921.
  • Marshall Thompson's presentation of :
    Yeom, Samuel et al. "Overfitting, Robustness, and
    Malicious Algorithms: A Study of Potential
    Causes of Privacy Risk in Machine Learning".
    Journal of Computer Security 28.1 (2020): 35-70.



Dev Shah's
Slides.



Marshall Thompson's
Slides.
Nov 3, 2022
Student Paper Presentation: :
  • Deng Pan's presentation of :
    Jing, Pengfei, et al. "Too Good to Be Safe:
    Tricking Lane Detection in Autonomous Driving
    with Crafted Perturbations." 30th USENIX
    Security Symposium, 2021.

Class dicussion on MP-4 with TA.



Deng Pan's
Slides.


MP-4 Class Dicussion
Slides.
Nov. 04, 2022
Paper Reading List [Updated]
  • Defense Papers
  • Ethics, Privacy, Law Papers
Papers List
11 Nov 8, 2022
Student Paper Presentation (Defense Papers):
  • Danning Ma's presentation of:
    Gruebler et al "An intrusion detection system against
    black hole attacks on the communication network of self
    -driving cars." 2015 sixth international conference on
    emerging security technologies (EST). IEEE, 2015.
  • Yuan Gao's presentation of :
    Abera, T. et al. "DIAT: Data Integrity Attestation for
    Resilient Collaboration of Autonomous Systems". In
    Proceedings 2020 Network and Distributed System
    Security Symposium (NDSS) 2019.
  • Colton Asnes's presentation of :
    Miller, Charlie, and Chris Valasek. "Securing self-driving
    cars (one company at a time)." Black Hat (2018)





Danning Ma's
Slides.



Yuan Gao's
Slides.



Colton Asnes's
Slides.
Nov 10, 2022
Student Paper Presentation (Defense Papers):
  • Ellis Thompson's presentation of:
    (D) Noise Matters: Using Sensor and Process Noise
    Fingerprint to Detect Stealthy Cyber Attacks and
    Authenticate sensors in CPS.
  • Connor Burnett's presentation of:
    Madan, Bharat B., Manoj Banik, and Doina Bein.
    "Securing unmanned autonomous systems from cyber
    threats." The Journal of Defense Modeling and
    Simulation 16.2 (2019): 119-136.
  • Samantha McDonald's presentation of:
    Aniculaesei, Adina, et al. "Toward a holistic software
    systems engineering approach for dependable autonomous
    systems." 2018 IEEE/ACM 1st International Workshop on
    Software Engineering for AI in Autonomous
    Systems (SEFAIAS). IEEE, 2018.




Ellis Thompson's
Slides.



Connor Burnett's
Slides.



Samantha McDonald's
Slides.
Nov 11, 2022
Machine Problem MP IV: Attacks on Rover
Due Dates:
  • Setup: Nov 18, 2022 [6:00 PM ET]
  • Final Demo: Dec 09, 2022
MP IV
12 Nov 15, 2022
Student Paper Presentation (Defense Papers):
  • Deng Pan's presentation of:
    Feth, P. et al. "Dynamic risk assessment for vehicles of
    higher automation levels by deep learning". In International
    Conference on Computer Safety, Reliability, and Security
    (pp. 535-547). Springer.
  • Marshall Thompson's presentation of:
    J. Kamel et al. "CaTch: A Confidence Range Tolerant
    Misbehavior Detection Approach," 2019 IEEE Wireless
    Communications and Networking Conference (WCNC),
    2019, pp. 1-8, doi: 10.1109/WCNC.2019.8885740.



Deng Pan's
Slides.



Marshall Thompson's
Slides.



Nov 17, 2022 Class Cancelled, a Make-up Class will be scheduled later.
13 Nov 22, 2022
Student Paper Presentation (Ethics, Privacy, and Law Papers):
  • Marshall Thompson's presentation of :
    T. Holstein and G. Dodig-Crnkovic, "Avoiding the
    Intrinsic Unfairness of the Trolley Problem" 2018
    IEEE/ACM International Workshop on Software Fairness
    (FairWare), 2018, pp. 32-37, doi: 10.23919/FAIRWARE.
    2018.8452918.
  • Dev Shah's presentation of :
    Zhang, Heng, et al. "Privacy and performance trade-off in
    cyber-physical systems." IEEE Network 30.2 (2016): 62-66.




Marshall Thompson's
Slides.



Dev Shah's
Slides.
Nov 24, 2022 Thanksgiving Break
14 Nov 29, 2022
Student Paper Presentation (Ethics, Privacy, and Papers):
  • Miles Grant's presentation of :
    M. Keshk, et al.,"An Integrated Framework for Privacy-
    Preserving Based Anomaly Detection for Cyber-Physical
    Systems," in IEEE Transactions on Sustainable Computing,
    vol. 6, no. 1, pp. 66-79, 1 Jan.-March 2021,
    doi: 10.1109/TSUSC.2019.2906657.
  • Jonathan Lee's presentation of :
    Thierry Fraichard. "Will the Driver Seat Ever Be
    Empty?" [Research Report] RR-8493, INRIA.
    2014. hal-00965176v2



Miles Grant's
Slides.



Jonathan Lee's
Slides.
Dec 1, 2022
Student Paper Presentation (Ethics, Privacy, and Law Papers):
  • Danning Ma's presentation of :
    Fleetwood, Janet. "Public health, ethics, and autonomous
    vehicles." American journal of public health 107.4
    (2017): 532-537.
  • Yuan Gao's presentation of :
    S. Karnouskos and F. Kerschbaum, "Privacy and Integrity
    Considerations in Hyperconnected Autonomous Vehicles,"
    in Proceedings of the IEEE, vol. 106, no. 1, pp. 160-170,
    Jan. 2018, doi: 10.1109/JPROC.2017.2725339.
  • Colton Asnes's presentation of :
    Berendt, Bettina. 2020. “(De)constructing Ethics for
    Autonomous Cars: A Case Study of Ethics Pen-Testing towards
    ‘AI for the Common Good’”. The International Review of
    Information Ethics 28 (June). Edmonton, Canada.



Danning Ma's
Slides.



Yuan Gao's
Slides.



Colton Asnes's
Slides.
15 Dec 5, 2022
[Virtual makeup class for Nov 17th]
Virtual Student Paper Presentation (Defense Papers):
  • Miles Grant's presentation of :
    Yang, Y., & Huang, G. (2020). "Map-based localization
    under adversarial attacks". In Robotics Research
    (pp. 775-790). Springer
  • Mushary Alghamd's presentation of:
    A. Singandhupe and H. M. La, "MCC-EKF for Autonomous
    Car Security," 2020 Fourth IEEE International Conference
    on Robotic Computing (IRC), 2020, pp. 306-313,
    doi: 10.1109/IRC.2020.00056.
  • Jonathan Lee's presentation of:
    Q. Liu et el. "Secure Pose Estimation for Autonomous
    Vehicles under Cyber Attacks," 2019 IEEE Intelligent
    Vehicles Symposium (IV), 2019, pp. 1583-1588,
    doi: 10.1109/IVS.2019.8814161




Mushary Alghamd's
Slides.



Jonathan Lee's
Slides.
Dec 6, 2022
Student Paper Presentation (Ethics, Privacy, and Law Papers):
  • Ellis Thompson's presentation of :
    Héder, M. "The epistemic opacity of autonomous systems
    and the ethical consequences". AI & Soc (2020).
  • Connor Burnett's presentation of :
    Schellekens, Maurice. "Self-driving cars and the chilling
    effect of liability law." Computer Law & Security Review
    31.4 (2015): 506-517.
  • Samantha McDonald's presentation of :
    Kulicki, Piotr et al. (2018). Towards a Formal Ethics for
    Autonomous Cars", Deontic Logic and Normative Systems.
    14th International Conference, DEON 2018.



Connor Burnett's
Slides.



Samantha McDonald's
Slides.
Dec 7, 2022
Virtual Student Paper Presentation (EPL & Defense Papers):
  • Dev Shah's presentation of (Defense):
    Lu, J. ey al. "Standard detectors aren't (currently)
    fooled by physical adversarial stop signs". arXiv
    preprint arXiv:1710.03337.
  • Deng Pan's presentation of (EPL) :
    Gogoll, Jan, and Julian F. Müller. "Autonomous cars:
    in favor of a mandatory ethics setting." Science and
    engineering ethics 23.3 (2017): 681-700.
  • Mushary Alghamd's presentation of (EPL) :
    Jardim, et al. "A study of public acceptance of autonomous
    cars." Worcester Polytechnic Institute: Worcester,
    MA, USA(2013).



Dev Shah's
Slides.


Deng Pan's
Slides.


Mushary Alghamd's
Slides.

Policies

Late Submissions for MPs

Submission/due dates are announced along with the MP descriptions.

You have a total of 3 "grace" days to use during the term. You can use those to submit whatever assignment(s) you want late. E.g. you can turn in one assignment 2 days late, the other 1 day late. Or turn in 3 assignments, each 1 day late. If you want to use these late days, indicate so in the comment box when you submit it on Canvas.

You start losing 20% of the points per day after the grace days you use. E.g., if the assignment is due on Tuesday, and you turn it in on Wednesday without saying anything, you lose 20 points. If you use 1 grace day, and turn it in on Wednesday, you don't lose any points; but if you turn it in on Thursday, your final grade will be Your Earned Grade - 20. On Friday, it will be Your Earned Grade - 40.

An assignment can be at most three days late: so in our example, if you turn in the assignment Saturday, the grade is 0.

The point of this policy is to balance the need to be fair to those who turn in their assignments on time, with the need to recognize that sometimes, stuff happens and you just can't do all that's expected of you. Why have deadlines at all? Because almost everything in life does, and this is a small-stakes environment to practice that. Also, we can't grade everything at once.

Collaboration and the Honor Code

You are encouraged to work together. You may discuss the MPs with other people to understand the problem and reach a solution. However, each student/group must write down the solution independently, without referring to written notes from others. Hence, you must understand the solution well enough to discuss it yourself. In addition, each student/group must explicitly mention the names of the people with whom they collaborated. If I suspect cheating, I might have the student(s) come and answer questions in my office. If my suspicions are confirmed, I will refer the student to the disciplinary committee.

The purpose of problem sets in this class is to help you think about the material, not just give us the right answers. You are encouraged to use online resources for learning more about the material covered in class; however, you should not look for or use found solutions to questions in the problem sets. Specifically, you must not look at any code that has been created to solve the assignment, including solutions found on the internet to questions in the problem sets, code created by a student in a previous class or code created by a current classmate. (Though frankly, I seriously doubt you'll find anything, I regularly change the assignments).

Academic Integrity

Academic Integrity is an integral part of the educational process, and GW takes these matters very seriously. Violations of academic integrity occur when students fail to cite research sources properly, engage in unauthorized collaboration, falsify data, and in other ways outlined in the Code of Academic Integrity. Students accused of academic integrity violations should contact the Office of Academic Integrity to learn more about their rights and options in the process. Outcomes can range from failure of assignment to expulsion from the University, including a transcript notation. The Office of Academic Integrity maintains a permanent record of the violation.

Familiarize yourself with the standards set forth in the GWU Code of Student Conduct and specifically the University’s “Guide of Academic Integrity in Online Learning Environments". If there is any question about whether an act constitutes academic misconduct, it is your responsibility to seek clarification and approval from the instructor prior to acting. When in doubt, ask, or don't do it.

Contact information:

Absence Policy

If you miss a class, for whatever reason, you are still responsible for that class' material. Go through the lectures, consult the scribe notes, come to office hours (in that order), and talk to your fellow students, to learn the material.

Grade Review Policy

You have one week after a grade is released to ask me questions about it and seek a correction. After the week has passed, the grade is finalized. When releasing a grade, I also post comments where appropriate, explaining where you lost points, made mistakes, etc.

In case you are seeking a correction to the grade, you need a specific reason: e.g., your reasoning on MP 1.a was essentially correct but your written explanation, you now realize, was ambiguous; my comments say that your model had errors and I couldn't run it, but you can run it fine on your machine; etc. I cannot accommodate general requests that are a variation on "I think my grade was too low". Why do you think that? Based on our discussion, I might increase the grade, decrease it, or leave it as is.

Of course, you can stop by anytime during the term to ask questions about any part of the material. The one-week deadline applies to grade-specific questions.

Use of Electronic Course Materials and Class Recordings

Students are encouraged to use electronic course materials, including recorded class sessions, for private personal use in connection with their academic program of study. Electronic course materials and recorded class sessions must not be shared and must not be used for non-course related purposes unless express permission has been granted by the instructor. Students who impermissibly share any electronic course materials are subject to discipline under the Student Code of Conduct. Please contact the instructor if you have questions regarding what constitutes permissible or impermissible use of electronic course materials and/or recorded class sessions. Please contact Disability Support Services if you have questions or need assistance in accessing electronic course materials.

Academic support

Writing Center

GW’s Writing Center cultivates confident writers in the University community by facilitating collaborative, critical, and inclusive conversations at all stages of the writing process. Working alongside peer mentors, writers develop strategies to write independently in academic and public settings.  Appointments can be booked online at: gwu.mywconline.

Academic Commons

Academic Commons provides tutoring and other academic support resources to students in many courses. Students can schedule virtual one-on-one appointments or attend virtual drop-in sessions. Students may schedule an appointment, review the tutoring schedule, access other academic support resources, or obtain assistance at academiccommons.gwu.ed.

COVID-Related Topics

The university's COVID-19 Safety and Success website serves as the primary communication channel for the GWU regarding COVID-19 logistics and classroom guidelines.

Face Coverings

The university has a requirement to use a face covering when in indoor spaces in order to contribute to the health and safety of the GWU community during the ongoing COVID-19 pandemic. Accordingly, you are expected to use a face covering when attending class, including when you speak. I will do the same.

Acceptable face coverings include:

If you don't have your own, face coverings are available at various campus locations.

Requests for exemption from wearing a face covering in indoor spaces require prior approval. Please follow this link for more information.

Students with Disabiities [202-994-8250]

Accommodations for students with disabilities are determined and approved by Disability Support Services(DSS). If you, as a student, feel that you need accommodations but have not obtained approval please contact DSS immediately at 202-994-8250 or at https://disabilitysupport.gwu.edu. DSS notifies students and faculty members of approved academic accommodations and coordinates implementation of those accommodations. While not required, students and faculty members are encouraged to discuss details of the implementation of individual accommodations.

Ethical and Respectful Behavior

We expect all students to act in an respectful and ethical way, both with respect to the treatment of their peers in the classroom during discussion but also in the design and execution of their course projects. Actions should meet the expectations of ethical research and follow the norms and proper behavior of the George Washington University community. We strive for an inclusive classroom, which includes but is not limited to anti-racism, anti-sexism, accessibility, LGBTQ+ inclusivity. If you feel like you are unable to use inclusive language and behave in a non-discriminatory way, please drop the class.

University policy on observance of religious holidays

In accordance with University policy, students should notify faculty during the first week of the semester of their intention to be absent from class on their day(s) of religious observance. For details and policy, see “Religious Holidays”.

Reach Out for Success/Wellness

University students encounter setbacks from time to time. If you encounter difficulties and need assistance, it's important to reach out. Consider discussing the situation with an instructor or academic advisor. Learn about resources that assist with wellness and academic success at Health and Wellness Center. If you or someone you know are in immediate crisis, please contact one of the following resources immediately:

Safety and Security