Hi hi,
I think I am speaking for a large group of students when I say I feel underprepared for the current speed and scope of this class.
I have fulfilled all the pre-reqs: CS61A (Python and Programming Basics), STAT 134 (Probability),STAT 135 (Statistics), Math 54 (Linear Algebra) and have done well in all of these classes. However, I am struggling to keep up in lecture and finish homework… ESPECIALLY as they pertain to computational complexity and advanced linear algebra.
Runtime analysis is only a major topic in CS61B (Data Structures), and for most iterations of Math 54 at Berkeley, we barely touch upon decompositions (not to mention positive semidefinite matrices, tensors, matrix calculus). To better inform our instructors, I have attached the syllabi or class websites as hyperlinks above.
I recognize that there are probably quite a few students in this class who have taken classes like CS61B (Data Structures), Math 110 (actually serious linear algebra), CS 188 (AI), CS 189 (ML) and that’s probably why some students are able to follow the lecture and ask meaningful questions. But sometimes I just sit in class not knowing what I don’t know and I feel frustrated because I have technically fulfilled the pre-reqs and have completed those classes with good grades.
Classmates who feel similarly please feel free to +1 or like this post. I have expressed my concerns to @gold_piggy in person during office hour, and I am making the post hoping to get @mli and @smolix 's attention on this issue.
Regardless, I am still committed to this class and am willing to put in the extra time and work. If instructors have good resources to share / or pointers on which areas of pre-requisite knowledge they think are critical to this class, I will greatly appreciate them all!
As for advanced linear algebra, please be more specific of what you mean to be advanced. Feel free to ask question on the forum since more students can benefit from this.
Also, try to be “Googleful”,there are tons of resource online nowadays! I believe if you work hard, at the end of this course, you will not only learn “deep learning”, but also learn “how to learn” more efficiently!
Thanks for the feedback. We’re trying not to go too deeply into any one of these subjects. To answer your questions more specifically:
You don’t really need to know about the various matrix decompositions such as Cholesky, eigenvector, eigenvalue, QR or anything the like to answer the linear algebra question in HW1. In fact, that’s the very reason that the question only asked for the forward part of the decomposition. To prove it you need to show that a matrix M = V^\top D V is positive semidefinite whenever D is a diagonal matrix with nonnegative entries. So we only need to show that x^\top V^\top D V x \geq 0 for all x. This is straightforward using the fact that we can write D = E E, where E is also a diagonal matrix, but with entries E_{ii} = \sqrt{D_{ii}}. Once you do that, you get that x^\top V^\top D V x = \|E V x\|^2, which proves the claim. No fancy linear algebra required, just square roots.
As for runtime complexity analysis, no such thing was required. In fact, the various linear algebra operations all have the same computational complexity as far as a complexity theorist is concerned. The only difference is that vectorization makes things go faster. And, unfortunately, it’s vital for everyone to understand this, since inefficient code can easily be 10x to 100x slower than it should be.
In terms of data structures, we’re only really asking for matrices and arrays. No fancy structures needed. In fact, for the current sampler homework you can do the inefficient thing using a cumulative sum and an array rather than a heap. That said, you obviously can do the fancy thing using an alias method, or a heap, but you don’t need to. A simple sampler will suffice to pass the bar.
We’ll add more suggestions to narrow down the solutions a bit more. I think that your concerns are mostly due to overestimating the level of sophistication that’s required for the homework. The questions are such that you could submit a PhD level answer but that isn’t required. Maybe when we release the solutions it will become a bit more clear. Also - please feel free to use the office hours to ask questions. We’re here to help.
and I feel frustrated because I have technically fulfilled the pre-reqs and have completed those classes with good grades.