Bonkoski University: Fall/Winter Semester 2024
I want to do a couple more retrospectives, in line with the theme of the previous post.
Since September I've been focused on self-studying lots of (mostly) applied mathematics:
- Scientific Computing (Heath)
- Linear Optimization (Bertsimas)
- Convex Optimization (Boyd)
- Statistical Decision Theory (Berger)
- Statistical Rethinking (McElreath)
- Functional Analysis (Kreyszig) [Pure Math]
How it Started
I started self-studying mathematics in 2016-ish after feeling like I had significant gaps (compared to some colleagues). My technical background has generally been low-level computer engineering / compilers / operating-systems / machine-code / etc (e.g. SectorC, Dis86, Reversing a Mystery Function, etc)
I've always had a rather "brutalist" approach to my own education. I want to learn the foundations and internal structure rather than the pretty finish / wallpaper. So, in 2016, I started with the material that is terrorizes/hazes many who elect to study undergraduate pure mathematics: Real Analysis.
I went through Terrance Tao's excellent book Analysis I, which is really a Foundations course that starts from Peano Axioms and builds all the way to Real Analysis. Somewhere I had read that if you really want to learn math, you should essentially re-create the entire book: all proofs, all details, etc. That fit my brutalist approach, and so it began. I think I hand-wrote that entire book. And 3 months later I had developed chronic back pain from being hunched over in perpetual confusion by some $\epsilon-\delta $ proof.
While I don't think I've ever directly used anything I learned from Real Analysis, I'm oddly found of those studies. Perhaps, its some Sisyphean or Stockholm Syndrome way of cherishing ones own memories of Type 2 or Type 3 fun? But, those struggles had made pure math an epsilon less intimidating which is probably worth its weight in gold.
Over the years I've come to view mathematics as a language. In many respects its no different than any other human language: vocabulary, grammar, conventions, nuance, ambiguity, etc. The ambiguity point is one that is surprising to many new practitioners. After all, isn't the goal of mathematics to be precise and rigorous? We might think so, but every time we go to formalize mathematics, we seem to encounter big problems. This shouldn't be too surprising, mathematics is a language for communicating abstract logical ideas to other humans, not computers.
Anyways, I think my main take-away from my studies of Tao's book was improving my Mathematics Fluency (as a language).
That has been well worth the price of admission.
How its Going
But its now 2024 and I'm diving into it again.
In some sense, this time is arguably both no different and more brutal:
- I'm still doing it all self-study: having pre-curated lectures and an engaged professor really is under-appreciated by most students
- I'm still basically writing all the books out (or at least the most technical parts) but now I do it directly in LaTex
- I'm studying four-ish texts simultaneously
But in another sense, its more mature:
- I don't fret over every detail of every proof as long as I fully understand the big picture
- Its no longer important for me to do every exercise
- I'm focused on subjects that have a strong overlap with computing and applications
- I lean on open course materials where possible (e.g. Boyd's Stanford Convex Optimization Lectures)
I've learned over the years that writing is my key to learning. I can read the book five times and retain nothing. If I write out the arguments/proofs once, I just get it. I had a very similar experience with learning programming languages actually. Until 2022, I had read and followed the Rust project for a while but had written all of maybe 10 lines of code. It wasn't until I started using it to build a large project in 2022 that everything clicked.
Conclusion: Writing is the key and synthesizes understanding for me. Reading is only a tool to assist the end-goal of writing. I'm curious in what other areas I can apply a "writing approach" profitably.
Learnings
I've learned so much in 3 months that its hard to even make a list. Overall, its all much more applicable than previous studies. I started writing up takeaways on each, but it just got too long and too triggering to my perfectionism 😬 I may do deeper dives in the future posts. TBD.
Some short thoughts:
- Scientific Computing: Wildly useful, should have studied this in depth long ago. Its all just Taylor Series. Its all just Newton's Method. Everything is sort of obvious but also impossibly subtle and error-prone. Kind of everything just works but also kind of nothing works. Its much better than analytical solutions, but decidedly worse. (This makes sense, trust me)
- Linear Optimization: Beautiful geometric connections, duality is confusing but very useful. Deep respect for Dantzig after studying the Simplex Method: lots of connections.
- Convex Optimization: This is the way (The Mandalorian). Seems deeply useful. Quite remarkable how many non-trivial problems turn out to be convex. Somewhat obvious extension from Linear Programming, given the Interior Point methods, but with very non-obvious conclusions. Boyd gives a better explanation/motivation of duality than Bertsimas.
- Statistical Decision Theory: (1) Decision theory is just optimizing a loss-function over a posterior, all the hard work is in forming the posterior, (2) I don't like utility theory, (3) I don't like "Bayesian Apologetics"
- Statistical Rethinking: An answer to why classical statistics always bothered me: I'm just a Bayesian. I like thinking in prior/posterior rather than applying dubious frequentist rules. Causal Inference is completely new to me: very curious. Also, just use MCMC rather than the dubious approximations to the posterior.
- Functional Analysis: Less practical. But cleared up a lot of missing theoretical links for me: e.g. Riesz Representation: duality of operators and vectors
I have many more thoughts, especially about Bayesian Decision Theory, Utility Theory, and a suppressed rant about the so-called "St. Petersburg Paradox". This probably is all because of an alarming conversation I had with some Effective Altruists shortly after FTX imploded (story for another time...)
Maybe I'll write more. Or not.