Lectures will take place in Mechanical Engineering 212. Lectures can also be attended remotely on Zoom. If you are attending the course through UNITE, you can join live lectures either on Zoom or through UNITE. Lecture recordings will be provided and linked to this Canvas page.
Office hours will start from the second week of the course. The instructor's office hours are group office hours and will be on Zoom. The instructor's office hours will be recorded and linked to this Canvas page. The TA's office hours are individual and will be on Zoom.
| Week | Date | Type | Topic | Remote Attendance |
Slides | Recording |
| 1 | - | - | - | - | - | - |
| 01/19/2022 | Class | Introduction | Zoom | Slides | Video | |
| 2 |
01/24/2022 | Class | Software Verification and Validation | Zoom | Slides | Video |
| 01/26/2022 | Class | Software Systems | Zoom | Slides | Video | |
| 01/27/2022 | Office Hours | - | Zoom | - | Video | |
| 3 |
01/31/2022 | - | - | - | - | - |
| 02/02/2022 | Class | Building Software Systems | Zoom | Slides | Video | |
| 02/03/2022 | Office Hours | - | Zoom | - | - | |
| 4 |
02/07/2022 | Class | Unit testing | Zoom | Slides | Video |
| 02/09/2022 | Class | Test Doubles | Zoom | Slides | Video | |
| 02/10/2022 | Office Hours | - | Zoom | - | Video | |
| 5 |
02/14/2022 | Class | System Testing | Zoom | Slides | Video |
| 02/16/2021 | Class | Acceptance Testing and CI |
Zoom | Slides | Video | |
| 02/17/2022 | Office Hours | - | Zoom | - | Video | |
| 6 |
02/21/2022 | Class | Program Representations I | Zoom | Slides | Video |
| 02/23/2022 | Class | Program Representations II | Zoom | Slides | Video | |
| 02/24/2022 | Office Hours | - | Zoom | - | Video | |
| 7 |
02/28/2022 | Class | Control-flow Coverage | Zoom | Slides | Video |
| 03/02/2022 | Class | Mutation Coverage | Zoom | Slides | Video | |
| 03/03/2022 | Office Hours |
- |
Zoom | - |
Video |
|
| 8 |
03/07/2022 | - | - | - | - | - |
| 03/09/2022 | - | - | - | - | - | |
| 03/10/2022 | - |
- | - | - | - | |
| 9 |
03/14/2022 | Office Hours |
- |
Zoom | Video |
|
| 03/14/2022 | Class | Data-flow Coverage | Zoom | Slides | Video | |
| 03/16/2022 | Class | Random Testing | Zoom | Slides | Video | |
| 03/17/2022 | Office Hours | - | Zoom | - | Video | |
| 10 |
03/21/2022 | Class | Partition Testing | Zoom | Slides | Video |
| 03/23/2022 | Class | Combinatorial Testing | Zoom | Slides | Video | |
| 03/23/2022 | Office Hours | - | Zoom | - | Video | |
| 03/24/2022 | Office Hours | - | Zoom | - | Video | |
| 11 |
03/28/2022 | Class | Model-based Testing | Zoom | Slides | Video |
| 03/30/2022 | Class | Search-based Testing | Zoom | Slides | Video | |
| 03/30/2022 | Office Hours | - | Zoom | - | Video | |
| 03/31/2022 | Office Hours | - | Zoom | - | Video | |
| 12 |
04/04/2022 | Class | Symbolic Execution | Zoom | Slides | Video |
| 04/06/2022 | Class | Fuzzing | Zoom | Slides | Video | |
| 04/06/2022 | Office Hours | - | Zoom | - | Video | |
| 04/07/2022 | Office Hours | - | Zoom | - | Video | |
| 13 |
04/11/2022 | Class | Test Input Minimization | Zoom | Slides | Video |
| 04/13/2022 | Class | Flaky Tests |
Zoom | Slides | Video | |
| 04/13/2022 | Office Hours | - | Zoom | - | Video | |
| 04/14/2022 | Office Hours | - | Zoom | - | Video | |
| 14 |
04/18/2022 | Class | Regression Testing I | Zoom | Slides | Video |
| 04/20/2022 | Class | Regression Testing II | Zoom | Slides | Video | |
| 04/20/2022 | Office Hours | - | Zoom | - | Video | |
| 04/21/2022 | Office Hours | - | Zoom | - | Video | |
| 15 |
04/25/2022 | Class | Testing Tech Talks | Zoom | Slides | Video |
| 04/27/2022 | Class | Testing Tech Talks | Zoom | Slides | Video | |
| 04/27/2022 | Office Hours | - | Zoom | - | Video | |
| 04/28/2022 | Office Hours | - | Zoom | - | Video | |
| 16 | 05/02/2022 | Class | Testing Tech Talks | Zoom | Slides | Video |
You should refer to this course site and our Piazza site for finding information about course meetings, assessments, and grades. We are going to meet in class for lectures but there will also be the option to join lectures synchronously on Zoom. (If you enrolled through UNITE, you can also use the UNITE live stream). We will also provide video recordings for the lectures a few hours after each lecture ends. The recordings are made by UNITE and will be available to both the on-campus and the UNITE sections. We will use Piazza for all communications between students, the instructor, and the TA. (We will add you to Piazza using your UMN email.) We use Piazza as it is a great place to promote interactions among students, and the course staff (instructor and TA) can keep track of all course-related discussions in one place. You will be using Gradescope to complete participation assessments and quizzes. You will be using GitHub, GitHub classroom, and Gradescope to complete coding assignments. You will use Gradescope to submit the link to your testing tech talks.
You can find all the assessment tasks on the Assignments page of this Canvas site and on Gradescope. Please check this page at the end of each course meeting for updated information.
Final course grades will be calculated based on the following percentages:
| Percentage | Assessment Type | Submission Type |
| 10% | Participation | Individual |
| 40% | Quizzes | Individual |
| 40% | Coding Assignments | Team |
| 10% | Testing Tech Talks | Team |
Participation: Due to the current situation, attending lectures synchronously is not required but highly encouraged. Although attendance is not required, there will be a participation grade. The participation grade will be based on offline assessments that include questions already answered in the class meetings. Specifically, at the end of each class meeting, you will need to complete a participation assessment on Gradescope. The assessment will be open for 48 hours. For people that cannot attend course meetings synchronously, we will provide meeting recordings, and link the recordings in the Topics table above. Participation assessments are individual.
Quizzes: There will be six quizzes. However, the quiz with the lowest score will be automatically dropped from the final grade computation. Quizzes will be available for 48 hours but once you start them they will be open for a limited amount of time. Quizzes need to be performed individually.
Coding Assignments: There will be six coding assignments. Coding assignments are based on teams of two students. In the coding assignments, your team will use technologies that implement the theoretical concepts discussed in the course. We will use peer evaluations to help us assess how much the individual team members contributed to the results of the team in the assignments. Please note that we will (1) give a lower weight to outliers (e.g., discount one negative rating when all other ratings for that student are positive), (2) consider individual cases (e.g., specific problems that might have affected the performance), and (3) provide the opportunity to detail the individual contributions in the assessments. In general, we will use a grain of salt when taking the assessments into account.
Testing Tech Talks: Your team will create a video discussing a testing technology. This video will be played to the class in one of the last three lectures. The testing tech talk is a team-based assessment. Peer evaluations will also cover the tech talks.
Exams: There is no midterm or final exam.
Late work is not accepted in this class. There are a few reasons for this policy. First, the assessment schedule follows a tight timeline and we would like to grade assessments as quickly as possible. Second, you will have ample time to complete the assessments. Finally, the majority of the assessments will help you complete immediately following assessments.
Requests for items to be regraded must be made within seven calendar days of the marks being posted. Be aware, this means that students may not ask for an assignment from early in the term to be regraded after they receive their final grade. If a mistake has been made in recording a student's marks, please bring this to the attention of the course staff prior to the end of the term.
Final grades will be assigned based on the following scale. Grading for this course is on an absolute scale (i.e. no curve, no rounding up, etc.).
| Weighted Score (x) | Letter Grade | S/N |
| 94.0% ≤ x ≤ 100.0% | A | S |
| 90.0% ≤ x < 94.0% | A- | S |
| 87.0% ≤ x < 90.0% | B+ | S |
| 83.0% ≤ x < 87.0% | B | S |
| 80.0% ≤ x < 83.0% | B- | S |
| 77.0% ≤ x < 80.0% | C+ | S |
| 73.0% ≤ x < 77.0% | C | S |
| 70.0% ≤ x < 73.0% | C- | S |
| 65.0% ≤ x < 70.00 | D+ | N |
| 60.0% ≤ x < 65.0% | D | N |
| 0% ≤ x < 60.0% | F | N |
I also would like to share some of my thoughts on final grades. First and foremost, your grade isn't a judgment of who you are as a person. It is not an overarching statement about your fitness for work in this major or this field. It is not even necessarily a statement about how much of the course material you know or how hard you tried. It is a summary of the record of how you did on the required assessments for this course.
Second, if you mention to me that you think that your grade doesn't reflect your effort or your understanding, I want to let you know that I understand that. I want to let you know that you are so much more than that number or letter on a page. But if you ask me whether it is possible to change your grade, please understand that you are asking me to falsify the record of how you did on the required assessments for this course. I cannot do that. I can tell you that even if your grade is not what you would have hoped it would be, I still believe in you and in the dream behind that hope. I 100% believe that you can go on to be successful in this major and in the field throughout your life.
To ensure the health and safety of everyone during these difficult times, the following precautions will be taken in this course.
Masking, vaccination, and absences: When indoors on campus, students, faculty, staff and guests are currently required to wear a face covering (mask). You must wear your mask so that it covers both your nose and mouth. This will help protect all members of the community, and especially those who are immunocompromised and/or who are caretakers of others (e.g., young children) who are not yet vaccinated. Even though vaccinations are highly protective and required for all students and employees, breakthrough infections do occur; therefore, indoor masking continues to be one of our most important tools for ensuring sustained in-person learning. With the high transmissibility of the recent variants it is strongly recommended that you use an enhanced mask— a surgical mask either alone or in combination with a cloth mask, or an N95, KN95, AirPop or similar mask. Surgical masks are widely available throughout campus, and you can get free high-quality masks by following these instructions.
The University requires all students and employees to be vaccinated or have a valid exemption; more information is at safe-campus website. On January 5, 2022 President Gabel announced an update on COVID-19 and campus operations which strongly encourages all community members to get a booster as soon as they are eligible. For information about getting a booster and how to schedule an appointment, please refer to the University’s Get the Vax 2.0 initiative.
Please stay at home if you experience symptoms of COVID-19 or have a positive COVID-19 test result and consult with your healthcare provider about an appropriate course of action. For COVID-19 excused absences, I will work with you to find the best course of action for missed work and/or class experiences.
Stable seating: I encourage everyone to sit in the same seat for every lecture. This will facilitate contact tracing in the unfortunate event someone tests Covid-positive during the semester.
Interactions: Other than the lectures, the interactions with the instructor will be over Piazza or Zoom. During the lectures, I would greatly appreciate it if we could practice social distancing while interacting. I am very happy to answer questions at end of the class while maintaining the CDC recommended social distancing guidelines and I will make sure to stay a few minutes longer at the end of class so that you can ask questions also from your seat. If you would like to discuss something in private, please contact me on Piazza, and I will take it from there.
Eating and drinking: Eating during class might make the masking precaution ineffective. For this reason, it will not be possible to consume food (with the exception of drinks) during the class and lab meetings. When drinking, masks must be worn between sips. If you have a medical exemption that requires you to eat or drink in class, do not hesitate to contact the instructor and/or provide an accommodation letter through DRC.
While this setup is far from ideal, every effort will be made to make the course as accessible, engaging, and useful as possible. Thank you for your understanding and cooperation.
CSCI5801 is the formal prerequisite for this class. You must be accepted to the CS upper-division, be a CS graduate student, or receive department permission to take this class. You are expected to have previous programming experience in Java, know how to use Unix-like command-line tools (e.g., cp), and be familiar with Git. You are expected to know basic data structures (such as lists), algorithms (such as search), recursion, and data abstraction. You are expected to be proficient in English and in using a computer to produce text documents and figures as required by the assignments. Please contact the instructor on Piazza if you have any questions about whether the course is a good fit for your background and academic goals.
In this course, you will learn the theory, skills, and tools to test real-world software systems. This course will prepare you for some important aspects associated with a career in software engineering. The topics covered in the course include:
In this course, you will:
This course covers a medium-large amount of material, requires good programming skills, and needs a significant amount of time spent on using software testing tools. Therefore the course will require a good amount of time to do all the reading, programming, and studying for quizzes. Students should expect to spend an average of 9 to 12 hours a week on this course.
In previous courses, you were introduced to software testing. In this course, you will learn software testing techniques used in industry and learn how to use the tools that implement these techniques. In the course, you will be given access to real-world software systems and you will use the systems in the coding assignments of the course.
The first priority of the TA is to ensure that each student receives the support they need to understand the course content. Please see the TA first with any questions about the course content. The course staff also forms the first line of support when it comes to the assignments.
There is no required textbook and the course meetings together with the course's slides will help you prepare for the assessments. There are a few books that I suggest if you would like to further explore some of the concepts we will cover throughout the course. The books are:
All work submitted for this course is required to be your original work or that of your group in the case of group work. You are expected to do your own thinking about how to solve an assessment. You are encouraged to discuss the content of the lectures with your peers. If you have any questions about whether discussing something with peers might go beyond what is permitted, then stop and ask us first on Piazza for clarification on the policy.
You are also not allowed to post your solutions to a repo that is not the one we assigned to you. We will also use code plagiarism detectors in this class.
The web is a fantastic resource for learning about testing, but it is also a potential source for solutions to assignments, so it is important that we are all on the same page with regard to what is permitted use of an online resource for this class and what is not. The use of the web in a way that supplements the type of information you will find in your class materials is great. This is encouraged. For example, if we talk about dynamic symbolic execution in class, you are encouraged to search for "dynamic symbolic execution" online and read more about this as a generic technique that can be used to test many programs. However, when it comes to a question in a quiz, you are not allowed to search online for something like "what is the result of symbolically executing if (x==3)". This crosses a line into searching out the solution to an assessment rather than learning about dynamic symbolic execution.
Scholastic dishonesty includes any deceptive means whereby a student attempts to gain an unfair advantage. Examples of scholastic dishonesty include violating the course policies outlined here, especially its “Academic Integrity” section; plagiarizing; cheating on assignments; or engaging in unauthorized collaboration on academic work, either with other students or via the internet. In order to be as clear as possible about your scholastic conduct responsibilities and how these relate specifically to the types of courses that we teach in the Department of Computer Science & Engineering, the faculty have prepared a CS&E Department Academic Conduct Policy. Our course will follow this policy, which stands alongside the broader Board of Regents Student Conduct Code.
Within the course, a student responsible for scholastic dishonesty can be given a penalty, including an "F" or "N" for the course. I am also required to report the incident to the Office for Student Conduct and Academic Integrity, and further disciplinary action may occur.
You are responsible for knowing and following the policies on scholastic conduct that are described in the syllabus and in the related documents discussed above (see especially the CS&E Department Academic Conduct Policy).
University policy is to provide, on a flexible and individualized basis, reasonable accommodations to students who have documented disability conditions (e.g., physical, learning, psychiatric, vision, hearing, or systemic) that may affect their ability to participate in course activities or to meet course requirements. Students with disabilities are encouraged to contact Disability Services and their instructors to discuss individual needs for accommodations. Disability Services, McNamara Alumni Center, Suite 180, 200 Oak Street, East Bank. Staff can be reached by calling (612) 626-1333 (voice or TTY).
As a student you may experience a range of issues that can cause barriers to learning, such as strained relationships, increased anxiety, alcohol/drug problems, feeling down, difficulty concentrating, and/or lack of motivation. These mental health concerns or stressful events may lead to diminished academic performance or reduce your ability to participate in daily activities. University of Minnesota services are available to assist you with addressing these and other concerns you may be experiencing. You can learn more about the broad range of confidential mental health services available on campus here.
In this course, we are committed to the University's equal access and opportunity policy. The University of Minnesota shall provide equal access to and opportunity in its programs, facilities, and employment without regard to race, color, creed, religion, national origin, gender, age, marital status, disability, public assistance status, veteran status, sexual orientation, gender identity, or gender expression.
In this course, we are committed to the University's Sexual Harassment, Sexual Assault, Stalking and Relationship Violence policy. The University of Minnesota is committed to taking prompt and effective steps intended to end sexual harassment, sexual assault, stalking, relationship violence, and related retaliation, prevent their recurrence, and, as appropriate, remedy their effects.
The University of Minnesota holds the copyright to the recordings - your access is strictly limited to your enrollment in this course. Accessing the media through either the UNITE Media Portal or within the Canvas course site obligates the student to the UNITE media agreement posted in the UNITE Media Portal. Individual access may be discontinued if found in violation of the agreement.
For troubleshooting assistance with the recordings, use the UNITE Troubleshooting FAQ or “Submit a Trouble Report to UNITE” link found on all pages within the UNITE Media Portal.
Zoom: We will use Zoom for classes and labs to allow remote attendance. We will also use Zoom for office hours meetings. To enhance the student learning experience, UNITE will record the meetings, use Kaltura to generate subtitles, and provide the recordings on this site. We will not make the recordings publicly available. If you have concerns, please contact the instructor on Piazza for further information.
Websites: In this course, our use of technology will involve sharing students' names, UMN usernames, and/or coursework information to course-related websites (e.g., Gradescope). All web sites might be affected by security vulnerabilities and it could be possible for them to be the target of software attacks. If you have concerns about the visibility of your name, username, or other information, please contact the instructor on Piazza for further information. Rest assured that it is our priority to keep this information within the boundaries of these websites.
All students are expected to behave as scholars at a leading institute of technology. This includes not talking over each other during the meetings. Disruptive students will be warned and potentially dismissed from the meetings.
We reserve the right to make changes to the syllabus to accommodate for exceptional circumstances that are out of the course staff's control. We will announce any change if it needs to happen.