Kyle Roth

On 2022-11-03 a class action lawsuit was announced against GitHub Copilot on the basis of copyright infringement, and now (2023-01-13) there’s one for stable diffusion (against StabilityAI and friends). Browsing through r/StableDiffusion, I’m seeing lots of posts like this making the very memeable point that 5 billion images can’t be stored in a 4 GB model. From the original poster:The thumbnail for this post was generated with stable diffusion. See the alt text for details. Yes, I’m not great at this.

I’m writing these things down over time because I’d like to figure out how to make artificial neural networks better at storing and retrieving knowledge, and the human brain is pretty good at that. When knowledge is new, it sometimes only comes back to me when I’m in the right context. For example, right now I can only think vaguely of how to adjust a chain derailleur on a bike, but I know that if I walked up to my bike and looked closely at the derailleur some more knowledge would appear. And we all have the familiar experience of being able to predict whether we’d recognize the name of a person or a song once it’s been said to us, but we can’t think of what it is until then.

Thanks to the crash of FTX and Scott Aaronson’s subsequent post about SBF, I read a very interesting deep-dive into Effective Altruism by the New Yorker. I’m seeing a lot of important characters show up that I’ve seen before: Eliezer Yudkowsky, earn-to-donate, 80,000 hours, etc. It’s really fascinating to see this all coming together in one narrative so I can understand a little better the inspiration for these ideas and the way that the movement has interacted with the world up till now. Here are my notes and critiques of the ethical ideas presented in the article.

I think it’s valuable to be working in the open whenever possible, so I’m going to keep my research notes here. These notes will hopefully be full of good (and bad) ideas, so if someone borrows a good idea and publishes on it, that’s great! This post contains my research notes as I try to understand how model scaling affects worst-group performance. This started as a group project in the neural scaling laws course at Mila in winter 2022. We presented about an existing paper and presented our preliminary results in class. The repository for this project is here.

Here I’m going to document my efforts to learn French. I speak Spanish pretty well, which together with English gives me a strong base for comprehension. 2021-08-01: I started using Duolingo every day, and got into the XP challenges to the point where I was getting like hundreds of points most days. I’ve currently (2022-06-01) got 15821 XP and 103 lesson crowns in the French course, and most of that came from Fall 2021. That really gave me a good sense for basic grammar and function words.