Christopher Olah's Blog

In the last few years, deep neural networks have dominated pattern recognition. They blew the previous state of the art out of the water for many computer vision tasks. Voice recognition is also moving that way. But despite the results, we have to wonder… why do they work so well? This post reviews some extremely […]

On a website called fanfiction.net, users write millions of stories about their favorite stories. They have diverse opinions about them. They love some stories, and hate others. The opinions are noisy, and it’s hard to see the big picture. With tools from mathematics and some helpful software, however, we can visualize the underlying structure.   […]

Recently, there’s been a great deal of excitement and interest in deep neural networks because they’ve achieved breakthrough results in areas such as computer vision. However, there remain a number of concerns about them. One is that it can be quite challenging to understand what a neural network is really doing. If one trains it well, it […]

Functions of the form or , where is a group, arise in lots of contexts. One very natural way this can happen is to have a probability distribution on a group, . The probability density of group elements is a function . Another way this can happen is if you have some function and has […]

(Trigger warning: descriptions of severe animal injury.) Today a squirrel was hit by a car a few feet away from me while I was walking down the side walk. Three of its legs kept twitching. I thought it had a broken leg. I came out of my stupor and went to grab it and pull […]

What is the distribution of the maximum of random variables? What started out a utilitarian question in my exploration of some generalized versions of the secretary problem turns out to be quite a deep topic. (Note that I have little background in probability and statistics. Please forgive (and inform me of, so I can fix!) […]

I’ve been talking about writing a topology textbook introductory notes on topology for years. Basically since I wrote my Rethinking Topology (or a Personal Topologodicy) post 2 years ago — it’s hard to believe it’s been that long! In any case, I finally started writing it. I’ve done a mild review of existing introductions to general topology (ie. I […]

Previously, I discussed training a neural net to clean up images. I’m pleased to say that, using more sophisticated techniques, I’ve since achieved much better results. My latest approach is a four layer convolutional network. Sadly, the convolution throws away the sides of the images, so we get a black margin. In any case, compare […]

For the last few months, every time someone hears that I work with 3D printers they bring up 3D printed guns. I can’t say how many times it has happened in this month alone. And I’m getting really really tired of it. “They’re the killer app of 3D printers.” What a great pun. You don’t know […]

For the last few weeks, I’ve been taking part in a small weekly neural net study group run by Michael Nielsen. It’s been really awesome! Neural nets are very very cool! They’re so cool, I had to use them somehow. Having been interested in mathematical handwriting recognition for a long time, I decided to train […]

As readers of this blog are probably aware, I’m a rather big fan of multiplicative calculus, an obscure mathematical tool that is very useful in reasoning about quasi-exponential trends. And I’m rather sad that I get few occasions to apply it. So, it should come as no surprise that when I was given an opportunity […]

At a meeting of the Toronto Haskell User Group a few months back (Jan 9, 2013), after going over the basics of the combinatoric parsing library Parsec, I talked about some clever tricks that made my work with it much cleaner. (See also Alber Lai’s write up of his talk, Parsec Generally.) Note that I […]

One thing I’ve realized in the last year is that I really deeply misunderstood what romantic attraction was. Most people I’ve tried to describe this to have been incredulous, since the things I misunderstood seem trivial to them. Nevertheless, I think that for a certain type of person this may actually be a really difficult […]

It seems like everyone writes a monad tutorial in the Haskell community… Well, here’s mine. People learning Haskell generally seem to get tripped up about monads. There are good reasons for this. Monads are, really, needed to write a serious Haskell program. As a consequence, people generally try to learn them fairly shortly after beginning […]

A while back, I wrote a post on some unusual math notation I was playing with. I actually took it much further than that and drafted up a paper full of different ideas over the following months. At the time, I was hoping it might be publishable but then got discouraged and never did anything […]

One interesting detour I took this last month was writing a toy Haskell Symbolic Algebra library, HaskSymb. It takes advantage of Quasi-Quoters and View Patterns to do awesome math pattern matching and enable one to write beautiful code. An example of its use: The interesting part, however, is the code for expand, collectTerms and friends. […]

I was the organizer of the most recent meeting of the Toronto RepRap User Group. It went excellently, with a turn out of about twenty people and five printers. We had a number of talks that were filmed by Socrates from the Singularity Weblog. In all, an awesome outcome! Attendees also saw the first public […]

I’m pleased to announce the second release of ImplicitCAD: 0.0.1. (The first release was 0.0.0 because 0 is the true first ordinal.) The point of this release is somewhat arbitrarily chosen. We were over due for one and no clean break was in sight. Then I woke up with a nasty headache and couldn’t seem […]

On January 11th, we had the first DIY Bio Toronto Meetup at hacklab. To quote the description of DIY Bio Toronto: Do-it-yourself biological innovation! We’re inspired by the California group “biocurious”, like them, we believe in the power of open source, open access, and learning in community. We believe this philosophy should be applied to […]

It’s that time of the year again: I’m making my fractal Holiday cards! I slacked last year and only made a few, but I’m back at it. Previous years, I’d made up fractals for my cards. This year I just explored the Mandelbrot set and found cool looking regions. Since I obsessed over my choice […]