The summer homeworks are optional and all can be done on a local machine, without any fancy equiptment. The first parts of each homework have an autograder, while the second part is meant to be evaluated by yourself. Only summer homework 2 part 2 has a Kaggle option.
Number | Part | Topics | Links |
---|---|---|---|
HW1 | 1 & 2 | A Basic Introduction to Neural Networks | Write-up (PDF) Part 1, Code and Data Files (ZIP) Part 2, Code and Data Files (ZIP) |
HW2 | 1 | Feedforward Convolutional Neural Networks (CNNs) | Write-up (PDF) Code and Data Files (TAR) |
2 | Image Recognition with CIFAR-10 | Write-up, Code, and Data Imports (PDF) Kaggle Classification (LINK) |
|
HW3 | 1 | TBD | Write-up (PDF) Code and Data Files (TAR) |
2 | TBD | Write-up (PDF) Code and Data Files (TAR) |
|
HW4 | 1 | TBD | Write-up (PDF) Code and Data Files (TAR) |
2 | TBD | Write-up (PDF) Code and Data Files (TAR) |
“Deep Learning” systems, typified by deep neural networks, are increasingly taking over all AI tasks, ranging from language understanding, and speech and image recognition, to machine translation, planning, and even game playing and autonomous driving. As a result, expertise in deep learning is fast changing from an esoteric desirable to a mandatory prerequisite in many advanced academic settings, and a large advantage in the industrial job market.
In this course we will learn about the basics of deep neural networks, and their applications to various AI tasks. By the end of the course, it is expected that students will have significant familiarity with the subject, and be able to apply Deep Learning to a variety of tasks. They will also be positioned to understand much of the current literature on the topic and extend their knowledge through further study.
Petr Ermakov and Artem Trunov are mirroring the course at OpenDataScience (ODS.ai). The mirrored course follows the CMU course in its entirety, quizzes, homeworks, piazza, discussion boards and all, and runs roughtly 3 weeks behind the CMU schedule. . There are currently about 1300 students signed up for it. If you are interested in the full course experience, you too can sign up for it at this site.
If you are only interested in the lectures, you can watch them on the YouTube channel listed below.
The course is well rounded in terms of concepts. It helps us understand the fundamentals of Deep Learning. The course starts off gradually with MLPs and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. We get a complete hands on with PyTorch which is very important to implement Deep Learning models. As a student, you will learn the tools required for building Deep Learning models. The homeworks usually have 2 components which is Autolab and Kaggle. The Kaggle components allow us to explore multiple architectures and understand how to fine-tune and continuously improve models. The task for all the homeworks were similar and it was interesting to learn how the same task can be solved using multiple Deep Learning approaches. Overall, at the end of this course you will be confident enough to build and tune Deep Learning models.
Instructor: Bhiksha Raj
TAs:
* -- contingent on registration
Lecture: Monday and Wednesday, 9.00am-10.20am
Location: TBD
Recitation: Friday, 9.00am-10.20am
Location: TBD
Office hours:
This course is worth 12 units.
Grading will be based on weekly quizzes (24%), homeworks (51%) and a course project (25%).
Policy | ||
Quizzes | There will be weekly quizzes.
|
|
Assignments | There will be five assignments in all. Assignments will include autolab components, where you must complete designated tasks, and a kaggle component where you compete with your colleagues.
| |
Project | All students are required to do a course project. The project is worth 25% of your grade | |
Final grade | The end-of-term grade is curved. Your overall grade will depend on your performance relative to your classmates. | |
Pass/Fail | Students registered for pass/fail must complete all quizzes, HWs and the project. A grade equivalent to B- is required to pass the course. | |
Auditing | Auditors are not required to complete the course project, but must complete all quizzes and homeworks. We encourage doing a course project regardless. |
The course will not follow a specific book, but will draw from a number of sources. We list relevant books at the end of this page. We will also put up links to relevant reading material for each class. Students are expected to familiarize themselves with the material before the class. The readings will sometimes be arcane and difficult to understand; if so, do not worry, we will present simpler explanations in class.
We will use Piazza for discussions. Here is the link. You should be automatically signed up if you're enrolled at the start of the semester. If not, please sign up.
You can also find a nice catalog of models that are current in the literature here. We expect that you will be in a position to interpret, if not fully understand many of the architectures on the wiki and the catalog by the end of the course.
Kaggle is a popular data science platform where visitors compete to produce the best model for learning or analyzing a data set.
For assignments you will be submitting your evaluation results to a Kaggle leaderboard.
All recitations and lectures will be recorded and uploaded to Youtube. Here is a link to the Youtube channel. Links to individual lectures and recitations will also be posted below as they are uploaded. All videos for the Fall 2019 edition are tagged “F19”. CMU students can also access the videos on Panopto from this link.
Lecture | Date | Topics | Lecture notes/Slides | Additional readings, if any | Quizzes/Assignments | Shadow Instructor | |
---|---|---|---|---|---|---|---|
0 | - |
|
Slides |
||||
1 | August 26 |
|
Slides |
|
|||
2 | August 28 |
|
Slides |
||||
3 | September 2 |
|
Slides |
.. | |||
4 | September 4 |
|
Slides |
||||
5 | September 9 |
|
Slides |
||||
6 | September 11 |
|
Slides |
||||
7 | September 16 |
|
Slides |
||||
8 | September 18 |
|
Slides |
||||
9 | September 23 |
|
Slides |
. |
|||
10 | September 25 |
|
Slides |
||||
11 | September 30 |
|
Slides |
||||
12 | October 2 |
|
Slides |
How to compute a derivative | |||
13 | October 7 |
|
Slides |
Cascade-Correlation | |||
14 | October 9 |
|
Slides |
Superposition of many models into one | |||
15 | October 14 |
|
Slides |
||||
16 | October 16 |
|
Slides |
||||
17 | October 21 |
|
Slides |
. |
|||
18 | October 23 |
|
Slides |
||||
19 | October 28 |
|
Slides | ||||
20 | October 30 |
|
Slides |
||||
21 | November 4 |
|
Slides |
. | |||
22 | November 6 |
|
|||||
23 | November 11 |
|
Slides |
||||
24 | November 13 |
|
Slides |
||||
25 | November 18 |
|
|||||
26 | November 20 |
|
|||||
27 | November 25 |
|
|||||
28 | November 27 |
|
|||||
29 | Decmeber 2 |
|
. | ||||
30 | December 4 |
|
Recitation | Date | Topics | Lecture notes/Slides | Notebook | Videos | Instructor |
---|---|---|---|---|---|---|
0 - Part 1 | August 15 | Python coding for the deep learning student | TBD | |||
0 - Part 2 | August 15 | Python coding for the deep learning student | Notebook | TBD | ||
1 | August 30 | Amazon Web Services (AWS) |
Slides |
Parth Shah, Kangrui Ruan | ||
2 | September 6 | Your First Deep Learning Code | Slides | TBD | ||
3 | September 13 | Efficient Deep Learning/Optimization Methods | Slides | Notebook | < | TBD |
4 | September 20 | Debugging and Visualization | Slides | Notebook | TBD | |
5 | September 27 | Convolutional Neural Networks | Slides | Notebook | TBD | |
6 | October 4 | CNNs: HW2 | Slides |
Notebook | TBD | |
7 | October 11 | Recurrent Neural Networks | Slides |
Notebook | TBD | |
8 | October 18 | RNN: CTC | Slides | Notebook | TBD | |
9 | October 25 | Attention | Slides | Notebook | TBD | |
10 | November 1 | Variation Auto Encoders |
Slides |
TBD | ||
11 | November 8 | Attention | Slides | Video | TBD | |
12 | November 15 | GANs | TBD | |||
13 | November 29 | Reinforcement Learning | TBD |
Most homeworks require submissions to autolab. If you are an autolab novice here is an “autolab for dummies” document to help you.
Number | Part | Topics | Release date | Early-submission deadline | On-time deadline | Links |
---|---|---|---|---|---|---|
HW0 | - | Python coding for DL | none | pdf html |
||
HW1 | 1 | An Introduction to Neural Networks | - | pdf html pdf (summer version) zip (summer version) |
||
2 | Frame level classification of speech |
Kaggle html zip (summer version) |
||||
HW2 | 1 | CNN | - |
pdf pdf (summer version) tar (summer version) |
||
2 | Face Classification/Verification via CNN |
pdf Classification Kaggle Verification Kaggle pdf (summer version) Kaggle (summer version) |
||||
HW3 | 1 | GRU | - | |||
2 | Utterance to Phoneme Mapping | pdf Kaggle Slack Kaggle Code Submission Form |
||||
Hw4 | 1 | Language Modeling using RNNs | - | pdf |
||
2 | Attention | pdf Kaggle |
||||
Project | Template |