Spring 2021 CSCI 5563
3D Computer Vision



Instructor: Hyun Soo Park (hspark at umn.edu)
Office hour: Wed 2:00pm-3:00pm (Zoom)

TA: Jingfan Guo (guo00109 at umn.edu), Tien Do (doxxx104 at umn.edu)
Office hour: Mon/Tue/Thr 2:00pm-3:00pm (Zoom)

Teaching mode: online

Textbook: Not required but the following books will be frequently referred:
+ "Multiple View Geometry in Computer Vision", Hartley and Zisseman

Important Dates

HW #1 due: Feb 5 midnight
HW #2 due: Feb 19 midnight
HW #3 due: Mar 12 midnight
HW #4 due: Apr 9 midnight
HW #5 due: May 7 midnight
Project registration: Feb 19 midnight
Project proposal presentation: Mar 2, 4
Project proposal report: Mar 5 midnight
Project final presentation: Apr 27, 29
Project final report: Apr 30


Lecture Readings
Camera Model Ch 6
Projection Matrix Ch 6
Projective Lines Ch 8
Criminisi et al., Single View Metrology, IJCV, 2000
Camera Localization via Vanishing Points Ch 8
Image Transformation Ch 2
Linear Estimation Appendix 4, 5
Single View Metrology Ch 8
Criminisi et al., Single View Metrology, IJCV, 2000
Reid and Zissermann, Goal-directed Video Metrology, ECCV, 1996
Single View Camera Calibration
Multiview Camera Calibration Zhengyou Zhang, A Flexible New Technique for Camera Calibration, TPAMI, 1998
Tour Into Photo Horry et al., Tour into the picture: using a spidery mesh interface to make animation from a single image, SIGGRAPH 1997
Rotation Wikipedia resources
Quaternion to Rotation
Quaternion interpolation (Slerp)
Learning Based Single View Geometry Hoiem et al., Automatic Photo Pop-up, SIGGRAPH 2005
Saxena et al., Learning Depth from Single Monocular Images, NeurIPS 2005
Ladicky et al., Pulling Things out of Perspective, CVPR 2014
Eigen et al., Depth Map Prediction from a Single Image using a Multi-Scale Deep Network, NeurIPS 2014
Qi et al., GeoNet: Geometric Neural Network for Joint Depth and Surface Normal Estimation, CVPR 2018
Jafarian and Park, Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos, CVPR 2021
Epipolar Geometry Ch 9, 10
Camera Pose Estimation Ch 9, 10
Nonlinear Estimation Appendix 6
Point and Pose Refinement Ch 12, 14
Perspective-n-Point Ch 7
Martin A. Fischler and Robert C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography", Comm. ACM 1981
Bundle Adjustment Ch 18
B. Triggs, P. McLauchlan, R. Hartley and A. Fitzgibbon, "Bundle Adjustment A Modern Synthesis", Vision Algorithms: Theory and Practice 1999
M.I.A. Lourakis and A.A. Argyros, "SBA: A Software Package for Generic Sparse Bundle Adjustment", ACM Transactions on Mathematical Software, 2009.
Depth Fusion R. Newcombe et al., "KinectFusion: Real-Time Dense Surface Mapping and Tracking", IEEE International Symposium on Mixed and Augmented Reality, 2011.


HW #1: Image Panorama
HW #2: Single View Image Navigation
HW #3: Single View Depth Prediction
HW #4: Structure from Motion
HW #5: Depth Fusion

Scholastic misconduct

Scholastic misconduct is broadly defined as "any act that violates the right of another student in academic work or that involves misrepresentation of your own work. Scholastic dishonesty includes, (but is not necessarily limited to): cheating on assignments or examinations; plagiarizing, which means misrepresenting as your own work any part of work done by another; submitting the same paper, or substantially similar papers, to meet the requirements of more than one course without the approval and consent of all instructors concerned; depriving another student of necessary course materials; or interfering with another student's work."