Lectures will take place in Keller Hall 3-125 Mon, Wed 2:30 pm - 3:45 pm. Although I encourage to attend letures in person, lectures can also be attended remotely through Zoom. If you are attending the course through UNITE, you can join live lectures either through UNITE or Zoom. The course team will also do its best to provide Zoom recordings through this page. (However, please note that recordings might not be available due to unexpected recording issues).
Office hours will start from the second week of the course. The instructor's office hours are group office hours and will be on Zoom. The instructor's office hours will be recorded and linked to this Canvas page. The TA's office hours are individual and will be on Zoom.
| Week | Date | Type | Topic | Remote Attendance |
Slides | Recording |
| 1 | - | - | - | - | - | - |
| 01/17/2024 | Class | Introduction | Zoom | Slides | Video | |
| 2 |
01/22/2024 | Class | Software Verification and Validation | Zoom | Slides | Video |
| 01/24/2024 | Class | Software Systems | Zoom | Slides | Video | |
| 01/24/2024 | Office Hours | - | Zoom | - | Video | |
| 3 |
01/29/2024 | Class | Building Software Systems | Zoom | Slides | Video |
| 01/31/2024 | Class | Team Building Activity | Zoom | - | - | |
| 01/31/2024 | Office Hours | - | Zoom | - | Video | |
| 4 |
02/05/2024 | Class | Unit testing | Zoom | Slides | Video |
| 02/07/2024 | Class | Test Doubles | Zoom | Slides | Video | |
| 02/07/2024 | Office Hours | - | Zoom | - | Video | |
| 5 |
02/12/2024 | Class | System Testing | Zoom | Slides | Video |
| 02/14/2024 | Class | Acceptance Testing and CI |
Zoom | Slides | Video | |
| 02/14/2024 | Office Hours | - | Zoom | - | Video | |
| 6 |
02/19/2024 | Class | Program Representations (Part I) | Zoom | Slides | Video |
| 02/21/2024 | Class |
Program Representations (Part II) |
Zoom | Slides | Video | |
| 02/21/2024 | Office Hours | - | Zoom | - | Video | |
| 7 |
02/26/2024 | Class | Control-flow Coverage | Zoom | Slides | Video |
| 02/28/2024 | Class | Mutation Coverage | Zoom | Slides | Video | |
| 02/28/2024 | Office Hours |
- |
Zoom | - |
Video | |
| 8 |
03/05/2024 | - | - | - | - | - |
| 03/07/2024 | - | - | - | - | - | |
| 03/07/2024 | - |
- | - | - | - | |
| 9 |
03/11/2024 | Class | Data-flow Coverage | Zoom | Slides | Video |
| 03/13/2024 | Class | Random Testing | Zoom | Slides | Video | |
| 03/13/2024 | Office Hours | - | Zoom | - | Video | |
| 10 |
03/18/2024 | Class | Partition Testing | Zoom | Slides | Video |
| 03/20/2024 | Class | Combinatorial Testing | Zoom | Slides | Video | |
| 03/20/2024 | Office Hours | - | Zoom | - | Video | |
| 11 |
03/25/2024 | Class | Model-based Testing | Zoom | Slides | Video |
| 03/27/2024 | Class | Search-based Testing | Zoom | Slides | Video | |
| 03/27/2024 | Office Hours | - | Zoom | - | Video | |
| 12 |
04/01/2024 | Class | Symbolic Execution | Zoom | Slides | Video |
| 04/03/2024 | Class | Fuzzing | Zoom | Slides | Video | |
| 04/03/2024 | Office Hours | - | Zoom | - | Video | |
| 13 |
04/08/2024 | Class | Test Input Minimization | Zoom | Slides | Video |
| 04/10/2024 | Class | Flaky Tests |
Zoom | Slides | Video | |
| 04/10/2024 | Office Hours | - | Zoom | - | Video | |
| 14 |
04/15/2024 | Class | Group Work on Testing Tech Talks | Zoom | - | - |
| 04/17/2024 | Class | Group Work on Testing Tech Talks | Zoom | - | - | |
| 04/17/2024 | - | - | - | - | - | |
| 15 |
04/22/2024 | Class | Regression Testing |
Zoom | Slides | Video |
| 04/24/2024 | Class | Software Testing Research |
Zoom | Slides | Video | |
| 04/24/2024 | Office Hours | - | Zoom | - | Video | |
| 16 | 04/29/2024 | Class | Software Quality Panel |
Zoom | Slides | Video |
You should refer to this Canvas site and our Piazza site for finding information about course meetings, assessments, and grades. We are going to meet in class for lectures but there is also an option to join lectures synchronously on Zoom. If you enrolled through UNITE, you can also use the UNITE live stream. We will do our best to also provide Zoom recordings for the lectures a few hours after each lecture. UNITE students can also access UNITE recordings. We will use Piazza for all communications between students, the instructor, and the TA. (We will add you to Piazza using your UMN email.) We use Piazza as it is a great place to promote interactions among students, and the course staff (instructor and TA) can keep track of all course-related discussions in one place. You will be using Gradescope to complete participation assessments and quizzes. You will be using GitHub, GitHub classroom, and Gradescope to complete coding assignments. You will use Gradescope to submit the link to your testing tech talks.
You can find all the assessment tasks on the Assignments page of this Canvas site and on Gradescope. Please check this page at the end of each course meeting for updated information.
Final course grades will be calculated based on the following percentages:
| Percentage | Assessment Type | Submission Type |
| 10% | Participation | Individual |
| 40% | Quizzes | Individual |
| 40% | Coding Assignments | Team |
| 10% | Testing Tech Talks | Team |
Participation: The course has a participation grade. The participation grade will be based on offline assessments that include questions already answered in the class meetings. Specifically, at the end of each class meeting, you will need to complete a participation assessment on Gradescope. The assessment will be open for 48 hours and is based on content presented in the lecture. Participation assessments are individual.
Quizzes: There will be six quizzes. The quiz with the lowest score will be automatically dropped from the final grade computation. Quizzes will be available for 48 hours but once you start them they will be open for 40 minutes. Quizzes need to be performed individually.
Coding Assignments: There will be six coding assignments. Coding assignments are based on teams of two students. In the coding assignments, your team will use technologies that implement the theoretical concepts discussed in the course. We will use peer evaluations to help us assess how much the individual team members contributed to the results of the team in the assignments. Please note that we will (1) give a lower weight to outliers (e.g., discount one negative rating when all other ratings for that student are positive), (2) consider individual cases (e.g., specific problems that might have affected the performance), and (3) provide the opportunity to detail the individual contributions in the assessments. In general, we will use a grain of salt when taking the assessments into account.
Testing Tech Talks: Your team will create a video discussing a testing technology. The testing tech talk is a team-based assessment. Peer evaluations will also cover the tech talks.
Exams: There is no midterm or final exam.
Late work is not accepted in this class. There are a few reasons for this policy. First, the assessment schedule follows a tight timeline and we would like to grade assessments as quickly as possible. Second, you will have ample time to complete the assessments. Finally, the majority of the assessments will help you complete immediately following assessments.
Requests for items to be regraded must be made within seven calendar days of the marks being posted. This means that students may not ask for an assignment from early in the term to be regraded after they receive their final grade. If a mistake has been made in recording a student's marks, please bring this to the attention of the course staff prior to the end of the term.
Final grades will be assigned based on the following scale. Grading for this course is on an absolute scale (i.e. no curve, no rounding up, etc.).
| Weighted Score (x) | Letter Grade | S/N |
| 94.0% ≤ x ≤ 100.0% | A | S |
| 90.0% ≤ x < 94.0% | A- | S |
| 87.0% ≤ x < 90.0% | B+ | S |
| 83.0% ≤ x < 87.0% | B | S |
| 80.0% ≤ x < 83.0% | B- | S |
| 77.0% ≤ x < 80.0% | C+ | S |
| 73.0% ≤ x < 77.0% | C | S |
| 70.0% ≤ x < 73.0% | C- | S |
| 65.0% ≤ x < 70.00 | D+ | N |
| 60.0% ≤ x < 65.0% | D | N |
| 0% ≤ x < 60.0% | F | N |
I also would like to share some of my thoughts on final grades. First and foremost, your grade isn't a judgment of who you are as a person. It is not an overarching statement about your fitness for work in this major or this field. It is not even necessarily a statement about how much of the course material you know or how hard you tried. It is a summary of the record of how you did on the required assessments for this course.
Second, if you mention to me that you think that your grade doesn't reflect your effort or your understanding, I want to let you know that I understand that. I want to let you know that you are so much more than that number or letter on a page. However, if you ask me whether it is possible to change your grade, please understand that you are asking me to falsify the record of how you did on the required assessments for this course. I cannot do that. I can tell you that even if your grade is not what you would have hoped it would be, I still believe in you and in the dream behind that hope. I 100% believe that you can go on to be successful in this major and in the field throughout your life.
For more information about COVID-19 response measures, please visit this page.
CSCI5801 is the formal prerequisite for this class. You must be accepted to the CS upper-division, be a CS graduate student, or receive department permission to take this class. You are expected to have previous programming experience in Java, know how to use Unix-like command-line tools (e.g., cp), and be familiar with Git. You are expected to know basic data structures (such as lists), algorithms (such as search), recursion, and data abstraction. You are expected to be proficient in English and in using a computer to produce text documents and figures as required by the assignments. Please contact the instructor on Piazza if you have questions about whether the course is a good fit for your background and academic goals.
In this course, you will learn the theory, skills, and tools to test real-world software systems. This course will prepare you for some important aspects associated with a career in software engineering. The topics covered in the course include:
Students will learn theoretical concepts behind software testing and get hands-on experience with key testing technologies. Students will learn to identify and solve software testing problems. Students will learn the theory behind unit testing, test doubles, system testing, acceptance testing, program representations, coverage criteria, black-box and white-box testing techniques, and test management. Students will get hands-on experience with test automation tools and frameworks like JUnit, Mockito, JaCoCo, Randoop, and EvoSuite. Students will learn time management skills to coordinate group work and be held accountable for group commitments. Students will learn how to interact as a team member and to work and collaborate together with shared goals.
This course covers a medium-large amount of material, requires good programming skills, and needs a significant amount of time spent on using software testing tools. Therefore the course will require a good amount of time to do all the reading, programming, and studying for quizzes. Students should expect to spend an average of 9 to 12 hours a week on this course.
In previous courses, you were introduced to software testing. In this course, you will learn software testing techniques used in industry and learn how to use the tools that implement these techniques. In the course, you will be given access to real-world software systems and you will use the systems in the coding assignments of the course.
The first priority of the TA is to ensure that each student receives the support they need to understand the course content. Please see the TA first with any questions about the course content. The course staff also forms the first line of support when it comes to the assignments.
There is no required textbook and the course meetings together with the course's slides will help you prepare for the assessments. There are a few books that I suggest if you would like to further explore some of the concepts we will cover throughout the course. The books are:
All work submitted for this course is required to be your original work or that of your group in the case of group work. You are expected to do your own thinking about how to solve an assessment. You are encouraged to discuss the content of the lectures with your peers. If you have any questions about whether discussing something with peers might go beyond what is permitted, then stop and ask us first on Piazza for clarification on the policy.
You are also not allowed to post your solutions to a repo that is not the one we assigned to you. We will also use code plagiarism detectors in this class.
The web is a fantastic resource for learning about testing, but it is also a potential source for solutions to assignments, so it is important that we are all on the same page with regard to what is permitted use of an online resource for this class and what is not. The use of the web in a way that supplements the type of information you will find in your class materials is great. This is encouraged. For example, if we talk about dynamic symbolic execution in class, you are encouraged to search for "dynamic symbolic execution" online and read more about this as a generic technique that can be used to test many programs. However, when it comes to a question in a quiz, you are not allowed to search online for something like "what is the result of symbolically executing if (x==3)". This crosses a line into searching out the solution to an assessment rather than learning about dynamic symbolic execution.
Scholastic dishonesty includes any deceptive means whereby a student attempts to gain an unfair advantage. Examples of scholastic dishonesty include violating the course policies outlined here, especially its “Academic Integrity” section; plagiarizing; cheating on assignments; or engaging in unauthorized collaboration on academic work, either with other students or via the internet. In order to be as clear as possible about your scholastic conduct responsibilities and how these relate specifically to the types of courses that we teach in the Department of Computer Science & Engineering, the faculty have prepared a CS&E Department Academic Conduct Policy. Our course will follow this policy, which stands alongside the broader Board of Regents Student Conduct Code.
Within the course, a student responsible for scholastic dishonesty can be given a penalty, including an "F" or "N" for the course. I am also required to report the incident to the Office for Community Standards, and further disciplinary action may occur.
You are responsible for knowing and following the policies on scholastic conduct that are described in the syllabus and in the related documents discussed above (see especially the CS&E Department Academic Conduct Policy).
University policy is to provide, on a flexible and individualized basis, reasonable accommodations to students who have documented disability conditions (e.g., physical, learning, psychiatric, vision, hearing, or systemic) that may affect their ability to participate in course activities or to meet course requirements. Students with disabilities are encouraged to contact Disability Services and their instructors to discuss individual needs for accommodations. Disability Services, McNamara Alumni Center, Suite 170, 200 Oak Street, East Bank. Staff can be reached by calling (612) 626-1333.
As a student you may experience a range of issues that can cause barriers to learning, such as strained relationships, increased anxiety, alcohol/drug problems, feeling down, difficulty concentrating, and/or lack of motivation. These mental health concerns or stressful events may lead to diminished academic performance or reduce your ability to participate in daily activities. University of Minnesota services are available to assist you with addressing these and other concerns you may be experiencing. You can learn more about the broad range of confidential mental health services available on campus here.
In this course, we are committed to the University's equal access and opportunity policy. The University of Minnesota shall provide equal access to and opportunity in its programs, facilities, and employment without regard to race, color, creed, religion, national origin, gender, age, marital status, disability, public assistance status, veteran status, sexual orientation, gender identity, or gender expression.
In this course, we are committed to the University's Sexual Harassment, Sexual Assault, Stalking and Relationship Violence policy. The University of Minnesota is committed to taking prompt and effective steps intended to end sexual harassment, sexual assault, stalking, relationship violence, and related retaliation, prevent their recurrence, and, as appropriate, remedy their effects.
For UNITE students, the University of Minnesota holds the copyright to the UNITE recordings - your access is strictly limited to your enrollment in this course. Accessing the media through either the UNITE Media Portal or within the Canvas course site obligates the student to the UNITE media agreement posted in the UNITE Media Portal. Individual access may be discontinued if found in violation of the agreement. For troubleshooting assistance with the recordings, use the UNITE Troubleshooting FAQ or “Submit a Trouble Report to UNITE” link found on all pages within the UNITE Media Portal.
Zoom: We will use Zoom for classes to allow in class participation and remote attendace. We will also use Zoom for office hours meetings. If you have concerns, please contact the instructor on Piazza for further information.
Websites: In this course, our use of technology will involve sharing students' names, UMN usernames, and/or coursework information to course-related websites (e.g., Gradescope). All web sites might be affected by security vulnerabilities and it could be possible for them to be the target of software attacks. If you have concerns about the visibility of your name, username, or other information, please contact the instructor on Piazza for further information. Rest assured that it is our priority to keep this information within the boundaries of these websites.
All students are expected to behave as scholars at a leading institute of technology. This includes not talking over each other during the meetings. Disruptive students will be warned and potentially dismissed from the meetings.
We reserve the right to make changes to the syllabus to accommodate for exceptional circumstances that are out of the course staff's control. We will announce any change if it needs to happen.