Kyle Roth

AI research is dead, long live AI

Wed, 30 Sep 2026 10:41:41 -0400

inspired by Mike Cook’s 2026-09-22 article “Why I love AI” (MHTML archive)MHTML is a web archiving file format that combines an HTML page and its associated resources into a single file.

This may be showing my youth, but I am surprised to learn that route optimization algorithms were once considered a part of AI research; I’ve always seen it treated as a highly distinct field, sometimes even outside the realm of informatics/CS. My department is named Département d’informatique et de recherche opérationelle, or “department of informatics and operations research”, where “operations research” is about such optimization algorithms. If in a different life I’d had the time to become invested in AI research for 20 years before things became the way they are in 2026, I could totally see myself feeling left behind like Cook feels. But critically, just as he was seeing his field left behind, I was seeing my field of interest, natural language processing, becoming subsumed by this latest instantiation of “AI research”.

Nonviolent communication: a language of life

Fri, 25 Oct 2024 15:39:56 -0400

feelings over judgments

To communicate nonviolently, we need to clearly communicate our feelings, rather than our judgments. Maturity means differentiating our feelings and experiences into as many parts and distinctions as a well-trained orchestral listener. “I feel that” or “I feel when” is expressing an opinion, rather than a feeling. Even “I feel inadequate” would be better expressed as “I feel disappointed with my performance…” “I feel good about …” is not specific enough.

Thinking in systems: a primer

Mon, 15 Jul 2024 11:17:09 -0400

A system is something that is composed of parts that have interrelations between them.

Each system has purposes, and those purposes may have nothing to do with stated goals. For example, if a government says it wants to protect the environment but allocates few resources to that goal, its purpose is not to protect the environment, but something else. A university may have the purpose of teaching students, while the components of the university have different goals: the student is trying to get good grades, the professor is trying to produce good research, and the administrator is trying to balance the budget.

Bowling alone: the collapse and revival of American community

Mon, 15 Jul 2024 10:36:57 -0400

the trends

politics: There is now more money in politics, and less individual involvement. There are more total associations, but they look more like the AARP and Greenpeace, instead of unions, churches, bridge clubs, rotary.
religion: Attendance has definitely gone down in the last 40 years, even if the number of people who profess stays nearly the same. (This has changed since the book was published on January 1, 2000.) The 60s and 70s saw people make their religion more private, more personal, making people less concerned with the communal support of a shared faith. Fundamentalist religion (including evangelical) results in more service and giving, but primarily within the church community. So as the mainline middle falls out, religion is becoming a less helpful societal lubricant overall, despite the fact that religiosity has generally meant more volunteerism and giving.
workplace: Unionization has declined, and people primarily see the union as a means for collective bargaining rather than a social movement. People are likely to have friends at work, but those friendships are not usually intimate and they don’t form the center of one’s social life. And now all jobs are more contingent; people are less likely to spend many years at a single job. Job precarity leads to anxiety, and less rootedness in one’s financial and community situation. People are less satisfied with their work time. At the end of the day, employers have control over work time and communications, especially with modern monitoring.
personal social connections: Informal social connection is just as important. Informality peaks in young and old adulthood, and formality in middle age. People thought urbanization would destroy social life, but instead of a city fostering a single social community, it fosters many smaller dispersed networks. You don’t know your neighbors though.But is this due to the lack of third spaces at a scale that makes it easy to meet your neighbors? My high-rise complex in Montreal provides an area with tables and chairs, grills, basic outdoor exercise equipment, and water fountains that kids can splash in. But the complex is large enough that this feels like a public place, like a park. If there were common spaces on the scale of a single building or even a single floor, I would probably know my neighbors. I think a lot of this comes down to some missing priorities in 20th century urban design. But hanging out with friends has really dropped off for average Americans, as has going out to sit-down restaurants. People spend 2/3 as much time in informal social activities as we used to, even while people are having fewer kids and are more educated (which in the past have been associated with greater social capital). Watching sports, going to museums, observing is up, and playing sports, creating music, and doing is down.
altruism/philanthropy: Donating and volunteering are predicted by one’s social connectedness, rather than defining connectedness itself. Joiners of any organization give ten times more philanthropy. We’re making more now than our parents did, but giving has not kept up. Volunteering is reported more frequently, but community service is reported less, meaning the volunteer work we do is more one-on-one, probably partly because people are not doing it through church as often. Volunteering has been propped up by the Silent generation doing more in old age.
trustworthiness: When we trust each other we can get more done because transaction costs are lower. Thick trust is when you’re on a first name basis, thin trust is the kind of trust you have in the anonymous other. Cities are less trusting, but more free. Thin trust peaked in the early 60s and has declined since. People are becoming angrier drivers, and hitchhiking is gone. Spending on lawyers, police, security, etc. have gone up. The increase in lawyers has mostly come from preventative efforts, where we “get it in writing” to establish a cool trust dependent on contracts.
grassroots movements: Political activism has turned from the “ground-based” Civil Rights Movement to “air-based” Greenpeace et al., due to the destruction of social capital. Meanwhile, evangelical, right-leaning political movements have managed to remain based in social capital and become a big political force. Evangelicals are 3-4 times more politically active than average.
telecommunications: People call a lot and keep up with distant friends by voice instead of by writing now. The social impact of the telephone was hard to judge, but was probably mostly positive for families and personal relationships, but reduced the creation of new friendships and the existence of personal relationships in public space. I’d be interested to see what he would report in 2023 about the internet. His ideas about it at time of writing were actually pretty reasonable. “Most online groups have the structure of either an anarchy, if unmoderated, or a dictatorship, if moderated.” (I wonder if we could design an algorithm to allow participants to represent the natural discomfort/dislike people feel with someone and then act accordingly.) Internet communication removes a lot of information necessary for social capital formation. In-person dialog allows speakers to constantly mediate their statements as they get facial feedback from listeners. This section was quite prescient:

Experiments that compare face-to-face and computer-mediated communication confirm that the richer the medium of communication the more sociable, personal, trusting, and friendly the encounter.

Computer-mediated communication is, to be sure, more egalitarian, frank, and task-oriented than face-to-face communication. Participants in computer-based groups often come up with a wider range of alternatives. However, because of the paucity of social cues and social communication, participants in computer-based groups find it harder to reach consensus and feel less solidarity with one another. They develop a sense of “depersonalization”, and are less satisfied with the groups accomplishments. Computer-based groups are quicker to reach an intellectual understanding of their shared problems—probably because they are less distracted by extraneous social communication—but they are much worse at generating the trust and reciprocity necessary to implement that understanding.

Cheating and reneging are more common in computer-mediated communication, where misrepresentation and misunderstanding are easier. Participants in computer-based settings are less inhibited by social niceties and quicker to resort to extreme language and invective. “Flaming” is the commonly-used term among cybernauts; the compelling image of communication is hand-to-hand combat with flamethrowers. Computer-mediated communication is good for sharing information, gathering opinions, and debating alternatives, but building trust and goodwill is not easy in cyberspace. John Seely Brown and Paul Duguid point out that “interactions over the Net, financial or social, will be as secure not as its digital encryption, which is a relatively cheap fix, but as the infrastructure—social as well as technical—encompassing that interaction.”

For these reasons, Nohria and Eccles suggest, widespread use of computer-mediated communication will actually require more frequent face-to-face encounters: “an extensive, deep, robust social infrastructure of relationships must exist so that those using the electronic media will truly understand what others are communicating to them.” (pp. 176-177)

He wondered whether the internet would become the next telephone, or the next TV. Very interesting, because it’s kind of been both, although I would say it definitely has fallen more on the side of TV even though it connects people somewhat intimately across long distances, and allowed important modern communities to flourish in a way they couldn’t have before.

The whole-brain child: revolutionary strategies to nurture your child's developing mind

Mon, 15 Jul 2024 10:21:23 -0400

The “refrigerator sheet” found here has a nice summary of the 12 strategies presented in this book.

This was a great book full of really insightful strategies to approaching mental integration. My only complaint is that they claim the strategies are neuroscientific, when in fact what they’re doing is using brain metaphors to teach real concepts. The left- and right-brain metaphor is taken too far in popular culture, and I think this book over-emphasizes the importance of these metaphors in order to argue for the parenting strategies here. I was especially bothered by how they used the concept of mirror neurons to argue for the sociality of the brain, as if that alone were a sufficient argument that we should teach kids to think about other people. The recommended strategies for teaching kids to see others are “enjoy each other” and “connect through conflict” which, while probably useful, do not depend on mirror neurons in any particular way.

Speaking as from the Dust: ideologies of AI and digital resurrection in Mormon culture

Wed, 08 May 2024 15:03:18 -0400

I wrote this paper with Stephen Betts, a friend of mine doing Mormon Studies at the University of Virginia. (Click the title to download the preprint.) Our interest in the topic was initially piqued when we stumbled across the Wilford Woodruff AI Learning Experience, and this paper explores the unique intersection of Mormonism, transhumanism, and machine learning that makes such a thing possible.

We’re currently looking for a venue to publish this work. Any feedback on the preprint is appreciated!

Better, Nicer, Clearer, Fairer: a critical assessment of the movement for ethical artificial intelligence and machine learning

Tue, 24 Oct 2023 12:53:31 -0400

I will present this paper in the FATE (fairness, accountability, transparency, ethics) reading group tomorrow (2023-10-25). You can view the slides I’ll use here.

There are unresolved tensions in the algorithmic ethics world. Here are two examples:

Is inclusion always good?
- Gebru: “you can’t have ethical A.I. that’s not inclusive… [a]nd whoever is creating the technology is setting the standards”
- Nelson: “… I struggle to understand why we want to make black communities more cognizant in facial recognition systems that are disproportionately used for surveillance.”
academic activism
- O’Neil: why is there a lack of academic efforts to inform policymakers and regulators?
- PERVADE: Academics have been doing this work for a while but it is underfunded, marginalized, and at odds with a US political apparatus generally favorable towards Silicon Valley.

Ethics manifestos or value statements mask these tensions behind a business ethics lens.

The right side of history: how reason and moral purpose made the West great

Thu, 12 Oct 2023 13:36:03 -0400

The main text below is my summary of the points in the book, and I’ve put my commentary in sidenotes. If you want a Marxist, postmodern response to these ideas from the devil himself, check out Jordan Peterson & the meaning of life by Philosophy Tube.

Reason and moral purpose come from Athens and Jerusalem, and without those things the West would not be where it is today. Socialism means taking handouts from the nanny state, and while our society continues to function with capitalism bearing the weight of socialist programs, it’s in the process of crumbling. People on the left like to blame institutions and systems for current woes, when in reality this is the freest, most egalitarian society that has ever existed. That’s why we see the conflict over political divides strengthening in America. The West is losing its attachment to reason and moral purpose, instead shifting to intersectionality, hedonism, and scientific materialism. Happiness is built on a sense of moral purpose, and is achieved by maximizing individuals’ ability to pursue that happiness. Scientific materialism and atheism are problematic because they take away that moral purpose.

Killing sacred cows: overcoming the financial myths that are destroying your prosperity

Thu, 12 Oct 2023 13:06:34 -0400

This book made me cringe pretty often. I’ll leave exclamation points (!) on particularly cringe-worthy ideas, just so you can see how based I am.

This book functions on the idea that if we accept a new set of words: poverty mindset, abundance mindset, etc, we’ll become wealthy.

Zero-sum mindset: any critique of “success”, including environmental devastation
Scarcity/poverty mindset: saving your way to wealth, judging or being jealous of those who have things, being too safe or too risky with your money, avoiding the risk of seeking your dreams, any ideas that make you make bad decisions

If more people live with an abundance mindset, we will all experience less hardship.

Raising an emotionally intelligent child: the heart of parenting

Thu, 12 Oct 2023 12:57:20 -0400

Really fantastic book. Here is a quote that was extremely key for me:

Understand your base of power as a parent. By base of power, I mean the element in the parent-child relationship that makes it possible for parents to set limits on children’s misbehavior, something all kids want and need. For some parents, the base of power is threats, humiliation, or spanking. Others, who are overly permissive, may feel they have no base of power at all. For emotion coaching parents, the base of power is the emotional bond between parent and child. When you are emotionally connected to your child, limit-setting comes out of your genuine reactions to your child’s misbehavior. Your child responds to your anger, disappointment, and worries, so you don’t have to resort to negative consequences such as spanking and time-outs to amplify your feelings. The respect and affection you and your child have for each other become your primary vehicle for limit-setting.

Why we sleep: unlocking the power of sleep and dreams

Wed, 11 Oct 2023 22:00:32 -0400

There is controversy around some of the claims made in this book, so I don’t take any single point of evidence extremely seriously. But the person who critiqued the book also has some weird ideas about sleep,For instance, he believes that we evolved in a sleep-deprived environment so sleep deprivation must be healthier for us. so in the end I think I probably side with the sleep scientist for most issues except when he seems extreme. Isn’t epistemology fun?

Team Human interview with Dennis Yi Tenen

Mon, 09 Oct 2023 10:34:12 -0400

You can view this episode here, and you can download a transcript I made with whisper-medium here. Please let me know if there are errors in the transcript.

writing technology

DYT: You know, it took humans like centuries to perfect the technology of a dictionary and it took hundreds, thousands of people, probably like millions of hours to actually get to the point where you can easily look up a word.

InstructEval: systematic evaluation of instruction selection methods

Mon, 25 Sep 2023 11:38:24 -0400

This was a paper I presented about in Bang Liu’s research group meeting on 2023-09-25. You can view the slides I used here.

thoughts on technology ownership

Thu, 07 Sep 2023 12:11:06 -0400

Having heard about the terrible privacy policies of car manufacturers and Ford’s new patent to automate vehicle repossession, I was finally able to formulate some thoughts I’ve had brewing for a while about ownership in this world of ubiquitous computing.

Before the industrial revolution, humans could essentially understand what a thing did by looking at it, and ownership was about possession of the object. Now we don’t understand our tools, and so possession does not imply control.

"Low-resource" text classification: a parameter-free classification method with compressors

Mon, 24 Jul 2023 11:35:13 -0400

I presented this paper in Bang Liu’s research group meeting on 2023-07-24. You can view the slides I used here.

It seems like the authors made a mistake that inflated the scores for the multilingual experiments, according to Ken Schutte.

the NYT AI explainer misses the point

Mon, 17 Apr 2023 10:07:35 -0400

In late March 2023, the NYT released a series of explainer articles about AI. The first article in the seriesYou can also read it on Archive.org if you don’t have a subscription. characterizes the recent history of AI as a progression of new technological ideas appearing over time. Of course that’s partially true, but it gets the order wrong and misses important non-technical events that are key to understanding our current position.

the inherent subjectivity of reality

Fri, 14 Apr 2023 14:45:44 -0400

These are some thoughts I’ve had while listening to a Lex Fridman interview with Edward Frenkel, a mathematician at UC Berkeley working on mathematical quantum physics.

In the information age, we like to see everything as computation. But what do we mean when we say that something is computation? We mean that a physical system with predictable interactions has a meaningful result. If we somehow learned that the universe was computational in nature, the only thing that adds is that the universe’s state is meaningful somehow.

Winners take all: the elite charade of changing the world

Mon, 20 Mar 2023 14:48:27 -0400

This book was a good one for quotable critique of modern capitalism. Here are some good ones:

These elites believe and promote the idea that social change should be pursued principally through the free market and voluntary action, not public life and the law and the reform of the systems that people share in common; that it should be supervised by the winners of capitalism and their allies, and not be antagonistic to their needs; and that the biggest beneficiaries of the status quo should play a leading role in the status quo’s reform.

for a socially beneficial and responsible development of AI

Mon, 20 Mar 2023 14:04:51 -0400

These are my notes from a conversation between Yoshua Bengio and Kate Crawford held at Mila on 2023-03-20, announcing the release of a new book created as a joint report between Mila and UNESCO called Missing links in AI governance (link). There were news articles in French (Le Devoir), but not as many in English unfortunately (Datanami).

Bengio: What motivates you to do what you do? This topic has turned from an academic move to a societal one really quickly, and it’s scary. We need to figure out what to do.

Think again: the power of knowing what you don't know

Wed, 08 Mar 2023 15:08:24 -0500

The book is divided into 3 parts, covering the value of rethinking, how to help others rethink, and how to help communities rethink.

the value of rethinking

Who you are should be a question of what you value, not what you believe.

When people change their answers on a test, they’re far more likely to change to the right answer than a wrong answer. Rethinking is effective!

helping others rethink

Good families allow for healthy conflict, rather than avoiding it. Orville and Wilbur Wright fought a lot but it wasn’t relational conflict, it was task conflict. Having a good relationship with colleagues is important because it’s what keeps task conflict from bleeding into relational conflict. You need a network of people who will disagree with you. Silence disrespects the value of your views, and our ability to have civil disagreement.

the AI art debate

Wed, 08 Mar 2023 12:09:25 -0500

On 2022-11-03 a class action lawsuit was announced against GitHub Copilot on the basis of copyright infringement, and now (2023-01-13) there’s one for stable diffusion (against StabilityAI and friends). Browsing through r/StableDiffusion, I’m seeing lots of posts like this making the very memeable point that 5 billion images can’t be stored in a 4 GB model. From the original poster:The thumbnail for this post was generated with stable diffusion. See the alt text for details. Yes, I’m not great at this.

Army of none: autonomous weapons and the future of war

Tue, 14 Feb 2023 13:33:19 -0500

The examples in this book make it clear that there is no easy line we can draw between autonomous and non-autonomous weapons (and by extension, autonomous AI agents). There is a smooth gradient of autonomy, which makes the question of allowing autonomous weapons much more nuanced. It’s probably the case that higher-level alignment becomes important proportionally to the level of autonomy and intelligence.

He analyzes the Patriot fratricides,In a military context, the word fratricide means the killing of someone on the same side of a conflict. and ends up blaming the individuals involved for automation bias. I would say that these humans in the system were set up to fail by the training and the functioning of the system. They’re expected to decide whether the computer is right, with only seconds to decide. He acknowledges this later when he talks about Aegis.

Conscious: a brief guide to the fundamental mystery of the mind

Mon, 13 Feb 2023 16:57:49 -0500

The first thing you should know is that this book is only 2 hours long in audio form. That’s short! She managed to get into the difficult details while keeping the jargon accessible. (It helps that we literally know nothing about consciousness.)

This book argued alternately for neuroscientific, illusionist, and panpsychist theories of consciousness. It explains the hard problem of consciousness, and then later really explains why it’s so hard. As a science-oriented person, t’s so easy to forget why it’s hard and start to explain things purely physically. When I do that, I end up thinking there’s got to be a neuroscience explanation. When I pay attention once again to my own experience, I lean towards some kind of panpsychism. The author really lays bare her own thoughts, and carefully spends time walking us through simple thought experiments to fight off the assumptions that we bring to the word “consciousness”.

Artificial intelligence, values, and alignment

Fri, 10 Feb 2023 08:09:15 -0500

I presented this paper in Bang Liu’s research group meeting in two installments on 2023-02-20 and 2023-02-27, and also in Irina Rish’s scaling and alignment course (IFT6760A) on 2023-03-07. You can view the slides I used here.The thumbnail for this post was generated with stable diffusion! See the alt text for details.

Behind each vision for ethically-aligned AI sits a deeper question. How are we to decide which principles or objectives to encode in AI—and who has the right to make these decisions—given that we live in a pluralistic world that is full of competing conceptions of value? Is there a way to think about AI value alignment that avoids a situation in which some people simply impose their views on others?

Unsolved problems in ML safety

Mon, 06 Feb 2023 11:39:33 -0500

This was a paper we presented about in Irina Rish’s neural scaling laws course (IFT6760A) in winter 2023. You can view the slides we used here, and the recording here (or my backup here).

Being mortal: medicine and what matters in the end

Wed, 01 Feb 2023 09:04:11 -0500

Read this book before you turn 60. In fact, read it before you’re 40. Read it before your parents are 60. Read it if you have parents! Read it if you’re a person who is likely to die sometime in the future.

This book gave me tools for the hard decisions that exist for people in the last decade(s) of their lives. Before reaching this stage of life ourselves, we don’t think about it much because we’ve tended to separate the aged from our communities. We’ve lost a cultural understanding of what a good death looks like. This book brings that knowledge back to a world with modern medicine.

The big three in economics: Adam Smith, Karl Marx, and John Maynard Keynes

Tue, 24 Jan 2023 13:28:01 -0500

This book was published in 2007, before the Great Recession. It definitely reads that way. Very capitalist, very Christian, very neo-liberal.

I enjoyed learning more about Adam Smith. I feel like Skousen does a good job painting the importance of his ideas as an invention that drove the Industrial Revolution.

According to Skousen, Marx is the devil incarnate, and his ideas are a dangerous disease infecting the minds of intellectuals and workers. Skousen takes a strange interest in painting Marx and Keynes as deranged, sexual deviants, etc. (“The Truth about Keynes’ Homosexuality” is a section in this book.)

The drunkard's walk: how randomness rules our lives

Tue, 24 Jan 2023 13:23:22 -0500

This is a good intro for non-statisticians to avoid some common pitfalls and misconceptions that the public often has. It’ll help you understand what it means when something is statistically significant, why sometimes studies contradict each other, and how to avoid believing in patterns that don’t exist. You’ll understand the following biases and fallacies:

confirmation bias
gambler’s fallacy
hot hand fallacy
“hindsight is 20/20”: the illusion of hindsight for explaining historical performance/events

One mistake I want to point out:

facts I've observed about how my brain stores knowledge

Thu, 19 Jan 2023 09:47:24 -0500

I’m writing these things down over time because I’d like to figure out how to make artificial neural networks better at storing and retrieving knowledge, and the human brain is pretty good at that.

When knowledge is new, it sometimes only comes back to me when I’m in the right context. For example, right now I can only think vaguely of how to adjust a chain derailleur on a bike, but I know that if I walked up to my bike and looked closely at the derailleur some more knowledge would appear. And we all have the familiar experience of being able to predict whether we’d recognize the name of a person or a song once it’s been said to us, but we can’t think of what it is until then.

The birth order book: why you are the way you are

Thu, 01 Dec 2022 09:48:05 -0500

This book really gave me the chance to think about what being the firstborn did to my personality and my life outlook. I think it’s pretty clear I fit the mold of a perfectionist firstborn, thanks to having usually met the high expectations of my parents growing up. It’s been especially valuable to think about how that will affect my own parenting, especially toward my first child. I think one of the most useful things about the book is that it prepares a parent to try to mediate some of the negative consequences of birth order. Here are the main tasks for a parent for each category:

thoughts on Effective Altruism

Thu, 17 Nov 2022 15:53:44 -0500

Thanks to the crash of FTX and Scott Aaronson’s subsequent post about SBF, I read a very interesting deep-dive into Effective Altruism by the New Yorker. I’m seeing a lot of important characters show up that I’ve seen before: Eliezer Yudkowsky, earn-to-donate, 80,000 hours, etc. It’s really fascinating to see this all coming together in one narrative so I can understand a little better the inspiration for these ideas and the way that the movement has interacted with the world up till now. Here are my notes and critiques of the ethical ideas presented in the article.

AlphaTensor: discovering faster matrix multiplication algorithms with reinforcement learning

Fri, 21 Oct 2022 14:18:00 -0400

This was a paper I presented about in Bang Liu’s research group meeting on 2022-10-21. You can view the slides I used here.

Whisper: robust speech recognition via large-scale weak supervision

Fri, 30 Sep 2022 09:46:29 -0400

This was a paper I presented about in Bang Liu’s research group meeting on 2022-09-30. You can view the slides I used here.

Selective annotation makes language models better few-shot learners

Tue, 13 Sep 2022 11:26:21 -0400

Selective annotation chooses a pool of samples to annotate from a large set of unlabeled data. The main result of the paper is that when this is combined with item-specific prompt retrieval the performance drastically improves (>10% relative gain and lower performance variance). Interestingly, selective annotation does not help for finetuning, or when the prompts are randomly selected. They call their selective annotation method “vote-$k$”.

selective annotation method

Vote-$k$ essentially creates a network of similaraccording to Sentence-BERT unlabeled instances, and then selects from them with a network importance score that is discounted to promote diversityThe discounting is performed by iteratively adding to the selection set, each time penalizing new nodes for being close to nodes that are already in the selection set..

Trivial or impossible—dichotomous data difficulty masks model differences (on ImageNet and beyond)

Mon, 15 Aug 2022 14:21:56 -0400

We observe that 48.2% [of] images [in ImageNet] are learned by all models regardless of their inductive bias; 14.3% [of] images are consistently misclassified by all models; only roughly a third (37.5%) of images are responsible for the differences between two models’ decisions. We call this phenomenon dichotomous data difficulty (DDD).

The authors varied hyperparameters, optimizers, architectures, supervision modes, and sampling methods, finding that models only varied in performance on about a third of the images in the dataset. And this isn’t specific to ImageNet; they found similar results for CIFAR-100 and a synthetic Gaussian dataset. They use this measure to divide the dataset into “trivials”, “impossibles”, and “in-betweens”.

Beyond neural scaling laws: beating power law scaling via data pruning

Thu, 11 Aug 2022 14:09:08 -0400

In this paper they show that we can achieve exponential performance scaling over dataset size, when the samples added are pruned to be only the best examples. This beats power law scaling in a big way. There is still no free lunch, in some sense, because in most cases it will become progressively harder to add new useful samples as the dataset gets bigger. But this is a big deal for computation, because it means that the number of samples in the dataset is not nearly as important as the coverage and quality that the dataset provides.This also means that scaling laws for compute (usually expressed as a function of dataset and model size) are dataset-specific and not generalizable, because of how much sample quality affects data scaling.

LocoProp: enhancing backprop via local loss optimization

Fri, 05 Aug 2022 09:52:06 -0400

This was a paper I presented about in Bang Liu’s research group meeting on 2022-08-05. You can view the slides I used here.

the effects of scale on worst-group performance

Mon, 18 Jul 2022 15:31:12 -0400

I think it’s valuable to be working in the open whenever possible, so I’m going to keep my research notes here. These notes will hopefully be full of good (and bad) ideas, so if someone borrows a good idea and publishes on it, that’s great!

This post contains my research notes as I try to understand how model scaling affects worst-group performance. This started as a group project in the neural scaling laws course at Mila in winter 2022. We presented about an existing paper and presented our preliminary results in class. The repository for this project is here.

Continual-T0: progressively instructing 50+ tasks to language models without forgetting

Thu, 02 Jun 2022 15:28:55 -0400

This was a paper I presented about in Bang Liu’s research group meeting on 2022-06-06. You can view the slides I used here.

Continual-T0 (CT0) extends T0 by progressively training it on 8 unseen language generation tasks, while retaining a replay buffer of 1% of the original training data to preserve performance. The result is a model that maintains nearly all of its performance on previous tasks while learning the new tasks. In addition, CT0 maintains the original T0’s performance on unseen tasks (which is a big deal because those tasks could not appear in the replay buffer) and it extends the compositionality of T0 to even more unseen tasks.

learning French

Wed, 01 Jun 2022 12:19:08 -0400

Here I’m going to document my efforts to learn French. I speak Spanish pretty well, which together with English gives me a strong base for comprehension.

2021-08-01: I started using Duolingo every day, and got into the XP challenges to the point where I was getting like hundreds of points most days. I’ve currently (2022-06-01) got 15821 XP and 103 lesson crowns in the French course, and most of that came from Fall 2021. That really gave me a good sense for basic grammar and function words.
2021-08-18: I moved to Québec and started experiencing the French language multiple times a week.
2021-09-01: The fall semester started at UdeM, so the number of emails (in French) from the university increased dramatically.
2021-10-06: I was assigned a speaking partner by the programme de jumelage interlinguistique, and we started meeting weekly to practice speaking (her English and me French). This was the first time I was actually producing output, which was scary but fun.
2022-01-15: I stopped using Duolingo as often because it feels repetitive and this semester I’m planning to write an app to study vocab more directly.
2022-02-01: My French speaking partner finished her program and moved back to France, so we stopped meeting (although we do plan to continue calling once she’s done traveling).
2022-04-27: I finished the alpha version of the WordBurner app and gave it to friends and family to test. I used it myself for about a week, and it felt like a very time-efficient way to increase vocabulary especially at the beginning. It’s rough around the edges in terms of the example sentences and the card recurrence was off, so I stopped using it after the semester project was over.
2022-05-03: I started attending a weekly French conversation workshop at the local community center.
2022-05-29: We listened to the first episode of the Duolingo French podcast, and really enjoyed it. We understood like probably 40% of what the French speaker was saying, thanks to the context provided by the English-speaking host. This seems like a great way to get interesting input at a level we can understand.
2022-05-31: The session for the French conversation workshop ended. I’d like to continue when it picks up again in the fall.
2022-07-18: I started a government-sponsored French course. It’s 3 hours of class time per day, 4 days per week. So far it’s been very useful for me! I think what I most need to provide for myself outside of class is vocabulary memorization and more time listening and speaking. I need to get back in the habit of using my flashcard app. Even though it’s still rough around the edges, it will help my vocabulary keep up with and go beyond the class. I would feel more like doing that if I spent one work day fixing the issues I had with it during the trial run in April. I should do a 5 minute sprint where I write down exactly what I want to change about the app before I am willing to start using it.

As far as comprehension and production goes, I’m planning on speaking more on the phone with my speaking partner from last year. I should also start consuming more content in French, like news and things like that. Maybe change some of my devices to French?
2022-07-23: I changed my phone’s language to French, and that’s actually been really good. I’m noticing more French language YouTube videos popping up, and some videos that are in English now come with French titles? Not sure how that works. Anyway, I’m finding on the whole that the combination of the class and the phone are making me think about French a whole lot more throughout the day.
2023-02-06: I haven’t done much studying of French, but I feel like my conversation ability has continued to slowly improve. I’m missing a lot of useful vocab because of this, so I often find myself stopping to come up with a way to say what I want with the words I know.
2023-10-12: Since stopping the French courses, I’ve been doing two group conversation workshops per week. I feel my speaking ability has improved despite not putting any other effort in, which is nice. For the fall session I decided to trade one of those workshops for a one-on-one weekly session at the same community center. I’ve done that for three weeks now and I find it immensely helpful. I actually had my first job interview entirely in French last week, which I was super proud of!

Multitask prompted training enables zero-shot task generalization (T0)

Fri, 27 May 2022 17:05:02 -0400

T0 builds on T5 by fine-tuning on more natural prompts and testing the model’s generalization to held-out tasks.

Compare the training format diagrams for T5 (top) and T0 (bottom):

Intuitively, the T0 prompts are more likely to be similar to implicit/explicit prompting that’s present in the pretraining data. The authors created several prompts for each dataset.

WordBurner beta

Mon, 18 Apr 2022 23:08:52 -0400

Update 2022-04-27: The beta is over, but the apk is still installable with the instructions below and any feedback sent from inside the app will be received by me. I’m going to be working on this more over the summer, and eventually publishing it on the app store. :)

Ever since learning Spanish, it has been a dream of mine to create a vocabulary study app that meets my needs. Duolingo won’t cover advanced vocabulary, Anki requires manually-generated decks, and other apps have expensive subscription plans.

PaLM

Mon, 11 Apr 2022 12:17:25 -0400

This was a paper I presented about in Bang Liu’s research group meeting on 2022-04-11. You can view the slides I used here.

QA-GNN: reasoning with language models and knowledge graphs for question answering

Tue, 05 Apr 2022 22:54:43 -0400

This post was created as an assignment in Bang Liu’s IFT6289 course in winter 2022. The structure of the post follows the structure of the assignment: summarization followed by my own comments.

The authors create a novel system for combining an LM and a knowledge graph by performing reasoning over a joint graph produced by the LM and the KG, thus solving the problem of irrelevant entities appearing in the knowledge graph and unifying the representations across the LM and KG.

ethics drift within bubbles

Fri, 01 Apr 2022 08:35:54 -0400

Here are some snippets from a Lex Fridman interview with John Abramson, outspoken critic of Big Pharma.

Lex: Are people corrupt? Are people malevolent? Are people ignorant that work at the low level and at the high level, at Pfizer for example? How is this possible? I believe that most people are good, and I actually believe if you join Big Pharma your life trajectory often involves dreaming, wanting, and enjoying helping people. And then we look at the outcomes that you’re describing and that’s why the narrative takes hold that Pfizer CEO Albert Bourla is malevolent. The sense is that these companies are evil. So if the different parts are people that are good and they want to do good, how are we getting these outcomes?

notes about neuroscience

Thu, 31 Mar 2022 10:02:44 -0400

How much of brain structure is coded for in the genome? For example, the hippocampus is generally thought to be responsible for consolidating long-term memories. Is the specialization of this region an epigenetic phenomenon due to optimization in the environment, or is it coded more directly? Will we eventually see these structures emerge in artificial networks with sufficient scale and good optimization, or will we need to code it more directly?

Experienced well-being rises with income, even above $75,000 per year

Wed, 30 Mar 2022 13:34:53 -0400

Turns out that money does buy happiness. You may have heard that people’s average happiness stops improving once you make more than $75,000/year? Researchers did a better survey with more data and found that that was not the case.

The researchers cited 5 methodological improvements over the old research that suggested that it didn’t matter after $75,000:

They measured people’s happiness in real time, instead of having people try to remember past happiness levels.
They measured on a continuous scale instead of discrete. (The survey had a slider between “very bad” and “very good”, instead of discrete options to choose from.)
They measured on “dozens of separate occasions per person” using an app, instead of just a single questionnaire.
They used a comparable scale for experienced well-being (people’s evaluations of their lives) and evaluative well-being (how people feel during the day-to-day moments of their lives).
They included a large number of high-earning participants and measured higher incomes in more granular increments.

The data were collected from 33,391 employed adults (ages 18-65) in the United States. There were a total of 1,725,994 reports collected, so that’s ~52 reports per person.

Neural message passing for quantum chemistry

Fri, 25 Mar 2022 14:46:11 -0400

This post was created as an assignment in Bang Liu’s IFT6289 course in winter 2022. The structure of the post follows the structure of the assignment: summarization followed by my own comments.

To summarize, the authors create a unifying framework for describing message-passing neural networks, which they apply to the problem of predicting the structural properties of chemical compounds in the QM9 dataset.

paper summarization

The authors first demonstrate that many of the recent works applying neural nets to this problem can fit into a message-passing neural network (MPNN) framework. Under the MPNN framework, at each time step $t$ a message is computed for each vertex by summing the output of a learned function $M_t$ over the vertex and all edges and vertices connected to it. Then the next state for each vertex is a learned function $U_t$ of the previous state and the message. Finally, the “readout” function $R$ is applied to all the vertices to compute the result.

keep your tasks in the heap

Tue, 22 Mar 2022 11:23:20 -0400

Often when someone (usually a professor) is sharing their screen I see that their browser has so many tabs open that the descriptions are lost:

That was my best impersonation as a Firefox user. Chrome will let you go a lot further (like ~113 tabs) before starting to provide a dropdown to show you the list of open tabs:

Besides the obvious fact that this makes it hard to find a tab you’re looking for, you also waste computer memory and add to your cognitive load while you’re working.

The effect of model size on worst-group generalization

Thu, 17 Mar 2022 14:34:33 -0400

This was a paper we presented about in Irina Rish’s neural scaling laws course (IFT6167) in winter 2022. You can view the slides we used here, and the recording here.

quotes from a Lex Fridman interview with Philip Goff

Wed, 09 Mar 2022 09:39:16 -0500

Here are some snippets from a Lex Fridman interview with Philip Goff, a panpsychist.

The Enlightenment ideal is to follow the evidence and the arguments where they lead, but it’s very hard for human beings to do that. I think we get stuck in some conception of how we think science ought to look. People talk about religion as a crutch, but I think a certain kind of scientism, a certain conception of how science is supposed to be, gets into people’s identity and their sense of themselves and their security.

hosting a Tor onion service

Thu, 24 Feb 2022 18:28:49 -0500

Tor onions are a way to host secure services that protect the anonymity of you and your clients. It also removes load from Tor exit nodes. If you open this page in the Tor browser it will redirect you to the following address:

http://kylrthjj7mpvktolz7u6fnudt3hpdvjw4hzquanjpepgsf5vcq5divad.onion/post/tor-onion-service/

which can only be opened from inside the Tor network.

getting started

To host an onion service, we’ll have a Docker container running Tor that decodes requests and forwards them to another container hosting the service. I’ve got a GitHub repo that will help you build the Tor image and shows you an example docker-compose.yml:

Scaling laws for the few-shot adaptation of pre-trained image classifiers

Tue, 22 Feb 2022 13:19:12 -0500

The unsurprising result here is that few-shot performance scales predictably with pre-training dataset size under traditional fine-tuning, matching network, and prototypical network approaches.

The interesting result is that the exponents of these three approaches were substantially different (see Table 1 in the paper), which says to me that the few-shot inference approach matters a lot.

The surprising result was that while more training on the “non-natural” Omniglot dataset did not improve few-shot accuracy on other datasets, training on “natural” datasets did improve accuracy on few-shot Omniglot.

Learning explanations that are hard to vary

Tue, 22 Feb 2022 12:29:17 -0500

The big idea here is to use the geometric mean instead of the arithmetic mean across samples in the batch when computing the gradient for SGD. This overcomes the situation where averaging produces optima that are not actually optimal for any individual samples, as demonstrated in their toy example below:

In practice, the method the authors test is not exactly the geometric mean for numerical and performance reasons, but effectively accomplishes the same thing by avoiding optima that are “inconsistent” (meaning that gradients from relatively few samples actually point in that direction).

In search of robust measures of generalization

Mon, 21 Feb 2022 15:33:22 -0500

These authors define robust error as the least upper bound on the expected loss over a family of environmental settings (including dataset, model architecture, learning algorithm, etc.):

\[\sup_{e\in\mathcal F}\mathbb E_{\omega\in P^e}\left[\ell(\phi,\omega)\right]\]

The fact that this is an upper bound and not an average is very important and is what makes this work unique from previous work in this direction. Indeed, what we should be concerned about is not how poorly a model performs on the average sample but on the worst-case sample.

It's not just size that matters: small language models are also few-shot learners

Fri, 18 Feb 2022 13:13:54 -0500

We presented this paper as a mini-lecture in Bang Liu’s IFT6289 course in winter 2022. You can view the slides we used here.

Educated

Thu, 17 Feb 2022 09:51:11 -0500

thinking for yourself

I recognized myself a little in this book, not in the events, severity, or locations but in the path to being “educated” in the sense that Westover intends. I’ll try to convey what that sense is with some quotes from the book.

The first moment is after she takes a class on American history at BYU. She returns home and gets her face dirty while working, and her brother calls her a N—r, a joke he had made many times before.

Scaling laws for transfer

Wed, 16 Feb 2022 14:12:26 -0500

This post was created as an assignment in Irina Rish’s neural scaling laws course (IFT6167) in winter 2022. The post contains no summarization, only questions and thoughts.

Sometimes these scaling laws can feel like pseudoscience because they’re a post hoc attempt to place a trend line on data. How can we be confident that the trends we observe actually reflect the scaling laws that we’re after? In the limitations section they mention that they didn’t tune hyperparameters for fine-tuning or for the code data distribution. How can we know that a confounding hyperparameter is not responsible for the trend we see? I wonder if we aren’t really being statistically rigorous until we can predict generalization error on an unseen training setup, rather than just an unseen model size/dataset size.

Deep learning scaling is predictable, empirically

Mon, 14 Feb 2022 10:38:11 -0500

This was a paper we presented about in Irina Rish’s neural scaling laws course (IFT6167) in winter 2022. You can view the slides we used here.

It’s important to note that in the results for NMT (Figure 1) we would expect the lines in the graph on the left to curve as the capacity of the individual models is exhausted. That’s why the authors fit the curves with an extra constant added. Meanwhile, the results in the graph on the right are curved because as the data size grows, the optimal model size also grows and it becomes increasingly difficult to find the right hyperparameters to train the model down to the optimal generalization error. (See the last paragraph in Section 4.1.)

Masked autoencoders are scalable vision learners

Fri, 11 Feb 2022 14:18:30 -0500

This post was created as an assignment in Irina Rish’s neural scaling laws course (IFT6167) in winter 2022. The post contains no summarization, only questions and thoughts.

In this paper they mention that the mask vector is learned, and it sounds like the positional embeddings are also learned. I remember in Attention is all you need they found that cosine positional embeddings worked better than learned ones, especially for sequences of longer length. But now it seems like most papers are doing learned embeddings. If anyone knows why, send me an email.

Data scaling laws in NMT: the effect of noise and architecture

Wed, 09 Feb 2022 20:47:59 -0500

This paper is all about trying a bunch of different changes to the training setup to see what affects the power law exponent over dataset size. Here are some of the answers:

encoder-decoder size asymmetry: exponent not affected, but effective model capacity affected
architecture (LSTM vs. Transformer): exponent not affected, but effective model capacity affected
dataset quality (filtered vs. not): exponent and effective model capacity not effected, losses on smaller datasets affected
dataset source (ParaCrawl vs. in-house dataset): exponent not affected
adding independent noise: exponent not affected, but effective model capacity affected

Here are some other things to test that I thought of while I read this:

Parallel training of deep networks with local updates

Wed, 09 Feb 2022 10:50:21 -0500

This post was created as an assignment in Irina Rish’s neural scaling laws course (IFT6167) in winter 2022. The post contains no summarization, only questions and thoughts.

Once I learned how the loss functions worked for each chunk, my first question was whether the earlier chunks were going to be able to learn the low-level features that later chunks would need. Figure 7 seems to show that they do, although their quality apparently decreases with increasingly local updates.

A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification

Wed, 02 Feb 2022 15:35:00 -0500

This post was created as an assignment in Bang Liu’s IFT6289 course in winter 2022. The structure of the post follows the structure of the assignment: summarization followed by my own comments.

paper summarization

Word embeddings have gotten so good that state-of-the-art sentence classification can often be achieved with just a one-layer convolutional network on top of those embeddings. This paper dials in on the specifics of training that convolutional layer for this downstream sentence classification task.

Learning transferable visual models from natural language supervision (CLIP)

Wed, 02 Feb 2022 12:35:03 -0500

This post was created as an assignment in Irina Rish’s neural scaling laws course (IFT6167) in winter 2022. The post contains no summarization, only questions and thoughts.

This concept of wide vs. narrow supervision (rather than binary “supervised” and “unsupervised”) is an interesting and flexible way to think about the way these training schemes leverage data.

The zero-shot CLIP matches the performance of 4-shot CLIP, which is a surprising result. What do the authors mean when they make this guess about zero-shot’s advantage:

Distributed representations of words and phrases and their compositionality

Tue, 01 Feb 2022 16:09:19 -0500

This post was created as an assignment in Bang Liu’s IFT6289 course in winter 2022. The structure of the post follows the structure of the assignment: summarization followed by my own comments.

paper summarization

This paper describes multiple improvements that are made to the original Skip-gram model:

Decreasing the rate of exposure to common words improves the training speed and increases the model’s accuracy on infrequent words.
A new training target they call “negative sampling” improves the training speed and the model’s accuracy on frequent words.
Allowing the model to use phrase vectors improves the expressivity of the model.

negative sampling

The original Skip-gram model computed probabilities using a hierarchical softmax, which allowed the model to compute only $O(\log_2(|V|))$ probabilities when estimating the probability of a particular word, rather than $O(|V|)$. Negative sampling, on the other hand, deals directly with the generated vector representations. The negative sampling loss function basically tries to maximize cosine similarity between the input representation of the input word with the output representation of the neighboring word, while decreasing cosine similarity between the input word and a few random vectors. They find that the required number of negative examples decreases as the dataset size increases.

meaning-making in the post-modern world

Tue, 01 Feb 2022 14:35:37 -0500

Here are some snippets from a Lex Fridman interview with Peter Wang, co-founder and CEO of Anaconda:

For a lot of human history, there wasn’t so much a meaning crisis as just a food and not getting eaten by bears crisis. Once you get to a point where you can make food there was a not getting killed by other humans crisis. Sitting around wondering what it’s all about is a relatively recent luxury.

Deep learning

Thu, 20 Jan 2022 15:11:00 -0500

This post was created as an assignment in Bang Liu’s IFT6289 course in winter 2022. The structure of the post follows the structure of the assignment: summarization followed by my own comments.

paper summarization

The authors use the example of distinguishing between a Samoyed and a white wolf to talk about the importance of learning to rely on very small variations while ignoring others. While shallow classifiers must rely on human-crafted features which are expensive to build and always imperfect, deep classifiers are expected to learn their own features by applying a “general-purpose learning procedure” to learn the features and the classification layer from the data simultaneously.

Cratylus

Sun, 16 Jan 2022 15:11:14 -0500

In this dialog Hermogenes comes to Socrates to discuss Cratylus’ view of the nature of names, whether they are true to the objects they represent or are just conventional. Hermogenes believes that names are purely conventional, while Cratylus believes the opposite. Socrates falls somewhere in the middle:

I quite agree with you that words should as far as possible resemble things; but I fear that this dragging in of resemblance, as Hermogenes says, is a shabby thing, which has to be supplemented by the mechanical aid of convention with a view to correctness; for I believe that if we could always, or almost always, use likenesses, which are perfectly appropriate, this would be the most perfect state of language; as the opposite is the most imperfect.

Crito

Mon, 27 Dec 2021 14:05:02 -0700

In this dialogue Crito comes to Socrates who is in prison waiting to be executed by the state. Crito has come to convince Socrates to come and escape with him. Crito’s escape plan will not cause great inconvenience for any of Socrates’ friends, and he would be able to live well in Thessaly. Socrates ends up convincing Crito that it would be wrong for him to escape.

“the opinion of the many”

CRITO: But you see, Socrates, that the opinion of the many must be regarded, for what is now happening shows that they can do the greatest evil to anyone who has lost their good opinion.

SOCRATES: I only wish it were so, Crito; and that the many could do the greatest evil; for then they would also be able to do the greatest good—and what a fine thing this would be! But in reality they can do neither; for they cannot make a man either wise or foolish; and whatever they do is the result of chance.

Apology of Socrates

Sun, 26 Dec 2021 13:40:45 -0700

I’m starting a course of foundational texts in philosophy with a friend of mine, and this is the first one we’ve read. Socrates is often considered a founder of Western philosophy, and it was easy for me to see in the text some common philosophical themes I’ve been exposed to growing up in the West.

the fear of death is irrational

Socrates argues that the fear of death is irrational from two perspectives: one, that what happens after death cannot be bad; and two, that a righteous person needs to be more concerned with whether he is doing right or wrong than whether death occurs or not. You can see the first one here:

avatarify

Wed, 24 Nov 2021 11:58:34 -0500

Avatarify is a cool project that lets you create a relatively realistic avatar that you can use during video meetings. It works by creating a fake video input device and passing your video input through a neural network in PyTorch. My laptop doesn’t have a GPU, so I used the server/client setup.

setting up the server

Be sure you’ve installed the Nvidia Docker runtime so that the Docker container can use the GPU. You can see how I did that here. Run the following on the server:

hosting my own web services

Tue, 23 Nov 2021 07:57:00 -0500

I host several services on an Alienware gaming computer I keep at my apartment. (We call it the spaceship.) I originally got the computer so I could have a computer with a GPU for machine learning projects, but I’ve since started using this computer to host a bunch of different services. Here I’ve documented how I set up the server.

operating system

To keep things simple I use Ubuntu 20.04 LTS. Here’s a speedy way I’ve found to write installer ISOs to USB drives for installation:

moving to Québec

Fri, 29 Oct 2021 11:55:00 -0400

We just moved our family from Utah, USA, to Montréal, Québec, Canada. I entered Canada on August 18, 2021 by car, and my wife and daughter entered a few days later by air. The process actually began on April 27 when I got my acceptance letter to the Université de Montréal as a master’s student in the Département d’informatique et de recherche opérationelle. After a few days of scrambling to find out if I would be able to study there without knowing French (turns out you can as a grad student at DIRO!), I started the process of applying to enter Canada and live in Québec as a student.

Jupyter Lab Hub in Docker with Nvidia GPU support

Tue, 19 Oct 2021 20:28:25 -0400

This is how I set up my headless home server with a Jupyter Lab Docker container with an Nvidia GPU runtime. Login is handled by a GitHub OAuth application.

Nvidia drivers and the container runtime

First, check here (replacing the CUDA version in the URL with your own) to see which Nvidia drivers you need for the CUDA toolkit version you want. I’m using CUDA 11.4.2, which means I need at least driver version 470.You can use sudo apt purge nvidia-* to cleanly remove older drivers (or broken installs) before installing the desired version.

Minecraft in Docker

Mon, 02 Aug 2021 13:25:28 -0600

This guide shows how to host multiple Minecraft servers on a single machine with docker-compose.

mkdir minecraft_server
cd minecraft_server
mkdir data/
wget /post/minecraft/docker-compose.yml
 \
 -O docker-compose.yml

This docker-compose setup uses itzg’s Docker image, which you see further documentation for here.

If you’re moving from a vanilla Minecraft world, do the following to get the different world directories in the right position:

cp -r ${OLD}/world data/server/world
mkdir data/server/world_{nether,the_end}
mv data/server/world/DIM-1 data/server/world_nether/DIM-1
mv data/server/world/DIM1 data/server/world_the_end/DIM1

Here’s the map from vanilla Minecraft directories to Spigot directories (which is what itzg’s container uses):

Matrix setup with Synapse, Postgres, Maubot, and matrix-registration

Mon, 02 Aug 2021 10:30:00 -0600

This is how I set up my own Matrix server with Docker.These instructions were originally for ARM, back when I ran this server on a Raspberry Pi. Unfortunately, the Matrix community stopped releasing ARM images, so the latest version that will work on ARM without QEMU is v1.26.0, which is very old now. These instructions have been updated to use amd64 images, but I’ll preserve the references to ARM images as comments. If you’re going to work from a Pi, be sure to switch it to run in 64-bit mode for optimal performance: echo 'arm_64bit=1' | sudo tee -a /boot/config.txt && sudo systemctl reboot. There is an Ansible playbook that’s quite popular, but I host a lot of other services with Docker on the same server and I wanted to continue managing all of them together, just with docker-compose.

I really just want to edit and compile my LaTeX files in VS Code

Mon, 12 Jul 2021 21:43:47 -0600

LaTeX has a ton of different flavors, releases, and installations: MacTeX, MiKTeX, TeXworks, XeTeX, pdfTeX, LuaTeX… If you’re using Linux and just want to edit LaTeX files in Visual Studio Code and have them automatically rendered as PDFs, follow these instructions:

On Arch-based distros, install the packages listed here. On Debian-based systems, sudo apt install texlive.

Install some Perl dependencies:

sudo cpan Log::Log4perl Log::LogDispatch Log::Dispatch::File YAML::Tiny File::HomeDir

If you want to use FontAwesome on Arch-based systems, install the oft-font-awesome package and then do the following (source):

sudo ln -s /usr/share/fontconfig/conf.avail/09-texlive-fonts.conf /etc/fonts/conf.d/09-texlive-fonts.conf
fc-cache && mkfontscale && mkfontdir

On Debian-based systems, install the font package with apt:

sudo apt install texlive-fonts-recommended texlive-fonts-extra

Install the VS Code extension called “LaTeX Workshop”.
If you want to use the fontspec package and get an error saying that “the fontspec package requires either XeTeX or LuaTex”, add the magic command % !TEX program = xelatex to the top of your .tex file.
If your installation needed to add some paths to your PATH but VS Code isn’t seeing them, try opening VS Code from the shell (with code path/to/workspace) so that it inherits $PATH from the interactive shell. (There might be a way to change the bin path in the settings for LaTeX Workshop, but I haven’t needed to find that setting yet.)

The last speakers: the quest to save the world's most endangered languages

Mon, 31 May 2021 08:29:00 -0600

This book argues that language loss is always bad, but that we can do something to save it. While the stories in the book leave me feeling like every language lost is a terrible cost, I think it’s inevitable as our species merges into a global society due to technology. I think we ought to prioritize the proper treatment and respect of marginalized and alternative cultures, including their languages and how these cultures want to maintain them. But there is a cost to stopping all language loss that is just not worth it once a language has been documented for research purposes.

The seven principles for making marriage work: a practical guide from the country's foremost relationship expert

Wed, 26 May 2021 06:52:44 -0600

Better communication doesn’t really solve marriage problems. It has a low success rate, and that makes sense because there are plenty of marriages that yell and dispute. Disputation is not a sign of an unhealthy marriage. You’d have to be really magnanimous to take criticism about you, even if presented as softly as possible.

Personality does not make a marriage incompatible. People can be friends but have very distinct personalities. Handle each other’s strange side with caring and respect, as you would a friend.

Harry Potter and the methods of rationality

Thu, 20 May 2021 07:25:00 -0600

Spoiler warning: no plot held back in this review.

science is at least as beautiful as magic

In chapter 7 Harry introduces Draco to the beauty of scientific advancement, and it actually moved me to tears. You should read the whole thing, but here are some of the best quotes:

“Anyway,” Harry said, “I’m saying that you don’t seem to have been paying much attention to what goes on in the Muggle world.” Probably because the whole wizarding world seemed to regard the rest of Earth as a slum, deserving around as much news coverage as the Financial Times awarded to the routine agonies of Burundi. “All right. Quick check. Have wizards ever been to the Moon? You know, that thing?” Harry pointed up to that huge and distant globe.

“What? " Draco said. It was pretty clear the thought had never occured to the boy. “Go to the - it’s just a -” His finger pointed at the little pale thingy in the sky. “You can’t Apparate to somewhere you’ve never been and how would anyone get to the Moon in the first place?”

“Hold on,” Harry said to Draco, “I’d like to show you a book I brought with me, I think I remember what box it’s in.” And Harry stood up and kneeled down and yanked out the stairs to the cavern level of his trunk, then tore down the stairs and heaved a box off another box, coming perilously close to treating his books with disrespect, and snatched off the box cover and quickly but carefully pried out a stack of books -

… And Harry raced back up the stairs and shoved the staircase back into the trunk with his heel, and, panting, turned the pages of the book until he found the picture he wanted to show to Draco.

The one with the white, dry, cratered land, and the suited people, and the blue-white globe hanging over it all.

That picture.

The picture, if only one picture in all the world were to survive.

“That,” Harry said, his voice trembling because he couldn’t quite keep the pride out, “is what the Earth looks like from the Moon.”

Draco slowly leaned over. There was a strange expression on his young face. “If that’s a real picture, why isn’t it moving?”

Moving? Oh. “Muggles can do moving pictures but they need a bigger box to show it, they can’t fit them onto single book pages yet.”

Draco’s finger moved to one of the suits. “What are those?” His voice starting to waver.

“Those are human beings. They are wearing suits that cover their whole bodies to give them air, because there is no air on the Moon.”

“That’s impossible,” Draco whispered. There was terror in his eyes, and utter confusion. “No Muggle could ever do that. How…”

Harry took back the book, flipped the pages until he found what he saw. “This is a rocket going up. The fire pushes it higher and higher, until it gets to the Moon.” Flipped pages again. “This is a rocket on the ground. That tiny speck next to it is a person.” Draco gasped. “Going to the Moon cost the equivalent of… probably around a thousand million Galleons.” Draco choked. “And it took the efforts of… probably more people than live in all of magical Britain.” And when they arrived, they left a plaque that said, ‘We came in peace, for all mankind.’ Though you’re not yet ready to hear those words, Draco Malfoy…

“You’re telling the truth,” Draco said slowly. “You wouldn’t fake a whole book just for this - and I can hear it in your voice. But… but…”

“How, without wands or magic? It’s a long story, Draco. Science doesn’t work by waving wands and chanting spells, it works by knowing how the universe works on such a deep level that you know exactly what to do in order to make the universe do what you want. If magic is like casting Imperio on someone to make them do what you want, then science is like knowing them so well that you can convince them it was their own idea all along. It’s a lot more difficult than waving a wand, but it works when wands fail, just like if the Imperius failed you could still try persuading a person. And Science builds from generation to generation. You have to really know what you’re doing to do science - and when you really understand something, you can explain it to someone else. The greatest scientists of one century ago, the brightest names that are still spoken with reverence, their powers are as nothing to the greatest scientists of today. There is no equivalent in science of your lost arts that raised Hogwarts. In science our powers wax by the year. And we are beginning to understand and unravel the secrets of life and inheritance. We’ll be able to look at the very blood of which you spoke, and see what makes you a wizard, and in one or two more generations, we’ll be able to persuade that blood to make all your children powerful wizards too. So you see, your problem isn’t nearly as bad as it looks, because in a few more decades, science will be able to solve it for you.”

“But…” Draco said. His voice was trembling. “If Muggles have that kind of power… then… what are we? "

“No, Draco, that’s not it, don’t you see? Science taps the power of human understanding to look at the world and figure out how it works. It can’t fail without humanity itself failing. Your magic could turn off, and you would hate that, but you would still be you. You would still be alive to regret it. But because science rests upon my human intelligence, it is the power that cannot be removed from me without removing me. Even if the laws of the universe change on me, so that all my knowledge is void, I’ll just figure out the new laws, as has been done before. It’s not a Muggle thing, it’s a human thing, it just refines and trains the power you use every time you look at something you don’t understand and ask ‘Why?’”

Planted: belief and belonging in an age of doubt

Sun, 18 Apr 2021 10:21:00 -0600

(My own thoughts appear as sidenotes or in italics, to distinguish from the author’s thoughts.)

Richard Bushman categorizes those who leave the church into two broad categories: those who feel “switched off”, and those who feel “squeezed out”. Mason summarizes the switched-off group as those who encounter troubling information about church history or doctrine, and as they discover more information they become jaded by it until they can no longer see the good the church does for them or for others. The squeezed-out group “fully embrace[s] the basic principles and ordinances of the gospel. But sometimes they feel alienated by things like the dominant political conservatism among the members … or how the church ministers to our LGBT … brothers and sisters,” (p. 3).

using GPG to prove you wrote your code

Mon, 12 Apr 2021 07:13:54 -0600

GPG is cool. You can use GPG to send encrypted messages, sign files to prove you generated them, and sign git commits to prove you committed them. You can get my key here. DigitalOcean has a neat guide to getting started with GPG. It explains asymmetric encryption, key generation and revocation, and key signing and maintenance.

Git commit authorship can be modified by anyone, as demonstrated by this tool. But by uploading your GPG public key to GitHub, you allow anyone who trusts GitHub to be sure that commits marked “verified” were actually created by you.

favorite art

Sat, 03 Apr 2021 22:16:52 -0600

Here’s some of my favorite art.

Edvard Munch, The Scream, 1893

(source)

Ben Shahn, All That Is Beautiful, 1966

(source)

Peter Doig, Architect’s Home in the Ravine, 1991

(source)

The life-changing magic of tidying up: the Japanese art of decluttering and organizing

Wed, 31 Mar 2021 21:21:04 -0700

This book was my first real exposure to minimalism, and it completely changed how I feel about the possession of objects. It was super fortunate that my wife and I listened to it together on a road trip, and became equally enthralled with the idea of dumping all of our excess clutter.

all at once

We have excess clutter because of a fundamental problem with the way we deal with possessions. This is difficult to solve with simple tricks like better organization or getting rid of one thing every day. She argues that the best way to overcome the clutter is by doing a one-time hardcore purge of the entire house. She spends a long time detailing what she calls the KonMari Method (which you can read about on her website). For us, doing this initial purge took us about two weeks. We spent several hours each evening, plus all day on both Saturdays. All told I think we got rid of like 15 garbage bags-worth of things from our two-bedroom apartment.

The smartest kids in the world

Tue, 30 Mar 2021 07:00:00 -0700

The PISA test tests common senses reasoning. The countries that did best on the test were a surprise to everyone. Finland, South Korea, and Poland were all standouts in their own ways, and Ripley compares the policies and learning environments in these countries with those of the US to determine why the US is falling behind, especially in math and science.

We talk a lot about parent involvement in the US, but the US actually has above average parental involvement. It turns out it’s not very correlated with success. Parent involvement is most useful when it’s reading to your child every day or asking them deep questions about school and about their opinions of broader issues. In the US we have lots of superintendents over small school districts, so there’s a lot of overhead without much benefit. We’re also afraid of standardization, so each school district has to sort of reinvent the wheel when it comes to curricula and standards. Common Core is an attempt by state education departments to agree to standards together so that textbooks and schools can target their products and training toward a common, well-defined set of goals.

The infinite Atonement

Tue, 30 Mar 2021 06:30:00 -0700

These notes are made while reading this with a Mormon theological background, so I skip noting some of the basic Mormon doctrines about the Atonement that he teaches.

The Atonement is the central doctrine of Christianity. All scripture should be at least partially focused on it, and we’re invited to “speak of the atonement of Christ, and attain to a perfect knowledge of him” (Jacob 4:12).

What is the significance of the Atonement?

Here are some of the ways that we come to understand the Atonement:

How not to diet: the groundbreaking science of healthy, permanent weight loss

Wed, 03 Mar 2021 07:00:37 -0700

I read this book with Irresistible and the Social Dilemma on my mind, so I have a lot of notes here about addiction and big business.

Just like everything else, capitalism has screwed over our diets by giving companies the incentive to put shareholders above customers. Food companies employ lobbyists to keep subsidies on sugar/corn syrup/meat, and keep a stranglehold on public organizations. They buy billions of dollars of ads to communicate the message that it’s laziness that has caused the obesity epidemic and to push their products that appeal to the unconscious desires of our brains to produce artificial hunger.

The gene: an intimate history

Fri, 22 Jan 2021 06:14:58 -0700

These are notes I made after finishing the book, so they’ll be more heavily weighted toward concepts discussed near the end. The first half of the book was primarily dedicated to a history of genetic research, which I think helped the reader understand the issues discussed in the latter half.

playing God

It seems like our identity derives from a complicated combination of genes and chance environmental effects. Part of our strength as a species has been our natural variation, and to begin editing the genome is to assume that we can do it better than evolution has done up until this point. To choose to remove variations is to decide that normal is best. (Some of our most beautiful productions have been created by people who under our normal social environment would be considered mentally ill.) To remove variation or introduce variation thus has a literally existential effect on our identity. What does it mean for a process to understand its own instructions?

Faith is not blind

Mon, 18 Jan 2021 08:11:23 -0700

Elder Hafen struggled as a missionary with the concept of knowing versus believing: he felt he believed it was true, but not that he knew it. On the mission he felt pressure to bear testimony with the word “know”, but he chafed at that. In this book, Elder Hafen hopes to discuss the complex boundaries between believing and knowing, Richard Bushman, a prominent LDS historian, found himself in a similar situation. He felt that he didn’t have the right words to express his belief in the nuanced way that he needed, even though looking back he thinks he did believe. This makes me look forward to reading Bushman’s “Rough Stone Rolling”, to try to understand his language of faith.

Cross-lingual alignment of contextual word embeddings, with applications to zero-shot dependency parsing

Fri, 11 Dec 2020 06:30:43 -0700

Recent contextual word embeddings (e.g. ELMo) have shown to be much better than “static” embeddings (where there’s a one-to-one mapping from token to representation). This paper is exciting because they were able to create a multi-lingual embedding space that used contextual word embeddings.

Each token will have a “point cloud” of embedding values, one point for each context containing the token. They define the embedding anchor as the average of all those points for a particular token. Here’s a figure from the paper that displays a two-dimensional PCA of the contextual representations for four Spanish words, along with their anchors:

Inductive biases for deep learning of higher-level cognition

Tue, 08 Dec 2020 06:40:48 -0700

This is a long paper, so a lot of my writing here is an attempt to condense the discussion. I’ve taken the liberty to pull exact phrases and structure from the paper without explicitly using quotes.

Our main hypothesis is that deep learning succeeded in part because of a set of inductive biases, but that additional ones should be added in order to go from good in-distribution generalization in highly supervised learning tasks (or where strong and dense rewards are available), such as object recognition in images, to strong out-of-distribution generalization and transfer learning to new tasks with low sample complexity.

SpanBERT: improving pre-training by representing and predicting spans

Sat, 05 Dec 2020 16:08:03 -0700

BERT optimizes the Masked Language Model (MLM) objective by masking word pieces uniformly at random in its training data and attempting to predict the masked values. With SpanBERT, spans of tokens are masked and the model is expected to predict the text in the spans from the representations of the words on the boundary. Span lengths follow a geometric distribution, and span start points are uniformly random.

To predict each individual masked token, a two-layer feedforward network was provided with the boundary token representations plus the position embedding of the target token, and the output vector representation was used to predict the masked token and compute cross-entropy loss exactly as in standard MLM.

Tools and weapons: the promise and peril of the digital age

Thu, 03 Dec 2020 20:58:02 -0700

I started taking notes later in the book. There were lots of good insights in the first half. Sorry!

broadband access

Getting the internet to rural communities is a big deal for the rural economy. Just like electricity, it’s something that needs government support because there isn’t the economic incentive for ISPs to reach some of these locations.

ethical AI

The focus on AI now is not just a fad, but a convergence of several trends that have made AI the next logical step: the increased computational resources, flexible access to compute through the cloud, etc.

Deep contextualized word representations

Thu, 03 Dec 2020 12:01:43 -0700

This is the original paper introducing Embeddings from Language Models (ELMo).

Unlike most widely used word embeddings, ELMo word representations are functions of the entire input sentence.

That’s what makes ELMo great: they’re contextualized word representations, meaning that they can express multiple possible senses of the same word.

Specifically, ELMo representations are a learned linear combination of all layers of an LSTM encoding. The LSTM undergoes general semi-supervised pretraining, but the linear combination is learned specific to the task. It’s been shown that initial layers in LSTM encoders are more representative of syntax, while later layers tend to represent semantics, so this linear combination is a key advantage that allows ELMo to improve accuracy on tasks ranging from POS tagging to question answering.

Removing a keyword from git history

Wed, 02 Dec 2020 11:34:25 -0700

I recently had to remove a keyword from the git history of a project I was working on. This meant not just removing a file but modifying commits where the keyword was added, commits where the keyword was removed, and even commits with the keyword in the commit message. I eventually came to the right solution through a mix of blog posts and the documentation for git rebase.

For this example, assume the keyword is “matrix”. The example output shown is from the git repo for this website.

Blink: the power of thinking without thinking

Tue, 17 Nov 2020 20:44:48 -0700

Our subconscious not only manages bodily systems but also performs processing of features in our experience that our conscious does not have time to process. This has been proven in lots of experiments where people have been given subconscious cues to help them solve problems, but the people are unaware of this and make up answers when asked to explain how they came to conclusions. It’s important to trust these judgments that seem to come out of nowhere, but if we try to explain them we’ll start trying to provide rational answers, which can be totally false or misleading. (Think about when one of the jurors in the OJ Simpson trial said that race had absolutely nothing to do with their judgment.)

A short history of nearly everything

Wed, 07 Oct 2020 11:19:03 -0600

We are extremely lucky to be here, and even more lucky to be able to appreciate it. Let’s not waste it.

Overcoming catastrophic forgetting in neural networks

Thu, 01 Oct 2020 10:47:28 -0600

In the paper they use Bayes’ rule to show that the contribution of the first of two tasks is contained in the posterior distribution of model parameters over the first dataset. This is important because it means we can estimate that posterior to try to get a sense for which model parameters were most important for that first task.

In this paper, they perform that estimation using a multivariate Gaussian distribution. The means are the values of the model parameters after training on the first dataset, and the precision (inverse of variance) is the values of the diagonals along the Fisher information matrix.

Learning neural causal models from unknown interventions

Tue, 22 Sep 2020 10:39:54 -0600

This is a follow-on to A meta-transfer objective for learning to disentangle causal mechanisms

Here we describe an algorithm for predicting the causal graph structure of a set of visible random variables, each possibly causally dependent on any of the other variables.

the algorithm

There are two sets of parameters, the structural parameters and the functional parameters. The structural parameters compose a matrix where $\sigma(\gamma_{ij})$ represents the belief that variable $X_j$ is a direct cause of $X_i$. The functional parameters are the parameters of the neural networks that model the conditional probability distribution of each random variable given its parent set.

A meta-transfer objective for learning to disentangle causal mechanisms

Mon, 21 Sep 2020 08:46:30 -0600

Theoretically, models should be able to predict on out-of-distribution data if their understanding of causal relationships is correct. The toy problem they use in this paper is that of predicting temperature from altitude. If a model is trained on data from Switzerland, the model should ideally be able to correctly predict on data from the Netherlands, even though it hasn’t seen elevations that low before.

The main contribution of this paper is that they’ve found that models tend to transfer faster to a new distribution when they learn the correct causal relationships, and when those relationships are sparsely represented, meaning they are represented by relatively few nodes in the network. This allowed them to create a meta-learning objective that trains the model to represent the correct causal dependencies, allowing for improved generalization.

Deep learning generalizes because the parameter-function map is biased towards simple functions

Tue, 08 Sep 2020 07:29:09 -0600

The theoretical value in talking about the parameter-function map is that this map lets us talk about sets of parameters that produce the same function. In this paper they used some recently proven stuff from algorithmic information theory (AIT) to show that for neural networks the parameter-function map is biased toward functions with low Komolgorov complexity, meaning that simple functions are more likely to appear given random choice of parameters. Since real world problems are also biased toward simple functions, this could explain the generalization/memorization results found by Zhang et al.

The moment of lift: how empowering women changes the world

Tue, 01 Sep 2020 05:25:38 -0600

This book is about empowering women by giving them the freedom to make their own choices and speak for themselves. She said some important things about stigma in society. She talked specifically about the stigma of not talking about birth control, but she made general statements too. It’s each person’s responsibility to work against stigma and stop the human tendency to cast out others. I need to spend more time thinking about my own stigmas and biases, so that I can help those who are marginalized.

A closer look at memorization in deep networks

Mon, 31 Aug 2020 11:52:35 -0600

This paper builds on what we learned in “Understanding deep learning requires rethinking generalization”. In that paper they showed that DNNs are able to fit pure noise in the same amount of time as it can fit real data, which means that our optimization algorithm (SGD, Adam, etc.) is not what’s keeping DNNs from overfitting.

experiments for detecting easy/hard samples

It looks like there are qualitative differences between a DNN that has memorized some data and a DNN that has seen real data. In experiments they found that real datasets contain “easy examples” that are more quickly learned than the hard examples. This is not the case for random data.

Naked economics: undressing the dismal science

Sun, 30 Aug 2020 06:46:49 -0600

An important question is how much we need to fight income inequality. Is it fair to have 35% growth in the upper class and 3% growth in the lower class? Where is a good balance?
We have grown a lot richer since the Industrial Revolution, because we’ve become more productive.
Wealth is not a zero-sum game. Globalization is good because it allows us to buy cheaper, better products. We can offset short-run job loss by paying or giving human capital to those who lose their jobs to globalization
Policies often don’t do what we intend them to do, because they change people’s decisions for the involved choice. Often, the best way to do policy is to incentivize the appropriate decision. E.g. Tax the purchase of inefficient cars on a scale, so car makers want to produce cheaper efficient cars to compete. Do that instead of simply banning cars with 18 mpg or less.
The Fed’s job is to control the availability of credit through setting their interest rates, so banks who borrow from them can adjust their interest rates accordingly. It’s really hard with so many factors, but Bernanke et. al. did an outstanding job.
Things that help an economy or nation develop: effective government institutionsproperty rightsno excessive regulationhuman capitalgeography (countries between the tropics have more diseases and less productive crops)openness to traderesponsible fiscal and monetary policy(natural resources don’t matter)democracypeaceinclusion of women in economic production
We can help underdeveloped countries, but the question of how is complicated. Certainly, we should open our borders to their products.Jeffrey Sachs – all these countries need is capital, so developed countries should invest in them (e.g. through comprehensively fighting AIDS)William Easterly – We can’t measure effectiveness based on inputs. That’s like rating Hollywood movies by budget size. “Instead we should do small, context-sensitive projects with measurable benefits.“But most of the developing world’s problems come from bad government policy.

The faith of a scientist

Sun, 30 Aug 2020 06:46:49 -0600

Scientific thinking and religion go hand in hand, and help refine and give purpose to each other.

Descartes’ approach wasn’t as good as Newton’s. Descartes relied on the soundness of his own reasoning.

“The erroneous conception that revelation ended with the apostles promotes the misconception among sectarian religions that the Gospel is complete and that with a liberal admixture of human wisdom, all will be crystal clear.”

God places messages in everything. We study the scriptures, but we also study each scientific field looking for the truth He offers us there.

Weapons of math destruction: how big data increases inequality and threatens democracy

Sun, 30 Aug 2020 06:46:49 -0600

In fact, I saw all kinds of parallels between finance and Big Data. Both industries gobble up the same pool of talent, much of it from elite universities like MIT, Princeton, or Stanford. These new hires are ravenous for success and have been focused on external metrics–like SAT scores and college admissions–their entire lives. Whether in finance or tech, the message they’ve received is that they will be rich, that they will run the world. Their productivity indicates that they’re on the right track, and it translates into dollars. This leads to the fallacious conclusion that whatever they’re doing to bring in more money is good. It “adds value.” Otherwise, why would the market reward it?

A disciplined approach to neural network hyperparameters: part 1

Fri, 28 Aug 2020 14:16:29 -0600

The goal of hyperparameter tuning is to reach the point where test loss is horizontal on the graph over model complexity.

Underfitting can be observed with a small learning rate, simple architecture, or complex data distribution. You can observe underfitting decrease by seeing more drastic results at the outset, followed by a more horizontal line further into training. You can use the LR range test to find a good learning rate range, and then use a cyclical learning rate to move up and down within that range.

Forward and reverse gradient-based hyperparameter optimization

Fri, 28 Aug 2020 14:16:29 -0600

In the area of hyperparameter optimization (HO), the goal is to optimize a response function of the hyperparameters. The response function is usually the average loss on a validation set. Gradient-based HO refers to iteratively finding the optimal hyperparameters using gradient updates, just as we do with neural network training itself. The gradient of the response function with respect to the hyperparameters is called the hypergradient.

One of the great things about this work is that their framework allows for all kinds of hyperparameters. The response function can be based on evaluation over the training set, the validation set, or both. The hyperparameters can be part of the loss function, part of regularization, or part of the model architecture.

Understanding deep learning requires rethinking generalization

Wed, 26 Aug 2020 08:42:58 -0600

It turns out that neural networks can reach training loss of 0 even on randomly labeled data, even when the data itself is random. It was previously thought that some implicit bias in the model architecture prevented (or regularized the model away from) overfitting to specific training examples, but that’s obviously not true. They showed this empirically as just described, and also theoretically constructed a two-layer ReLU network with $p=2n+d$ parameters to express any labeling of any sample of size $n$ in $d$ dimensions. The proof was actually relatively easy to follow.

Why does unsupervised pre-training help deep learning?

Mon, 24 Aug 2020 11:40:00 -0600

They’re pretty sure that it performs regularization by starting off the supervised training in a good spot, instead of by somehow improving the optimization path.

Essentialism

Sun, 23 Aug 2020 07:47:12 -0600

The main character of the first story slowly changed his attitude toward demands on his resources.

“Can I actually fulfill this request, given the time and resources I have?”
“Is this the very most important thing I should be doing with my time and resources right now?”
“Just because I was invited didn’t seem a good enough reason to attend.”

It’s important to pursue “less but better” in a disciplined way. Simply doing fewer things is not meaningful. Most things in the world are trivial noise, and my job is to sift through noise and find things that are truly valuable. This involves active consideration of responsibilities and trade-offs involved in making a choice. I think I often try to “force execution at the last moment”, which is a mark of a non-essentialist.

The consciousness prior

Fri, 14 Aug 2020 09:05:56 -0700

System 1 cognitive abilities are about low-level perception and intuitive knowledge. System 2 cognitive abilities can be described verbally, and include things like reasoning, planning, and imagination. In cognitive neuroscience, the “Global Workspace Theory” says that at each moment specific pieces of information become a part of working memory and become globally available to other unconscious computational processes. Relative to the unconscious state, the conscious state is low-dimensional, focusing on a few things. The paper proposes we use an attention mechanism (in the sense of the Bahdanau 2015 paper) to produce the conscious state, and then a VAE or conditional GAN to produce the output from the conscious state.

Troubling trends in machine learning scholarship

Thu, 13 Aug 2020 10:36:05 -0700

The authors discuss four trends in AI research that have negative consequences for the community.

problems

explanation vs. speculation

It’s important to allow researchers to include speculation, because speculation is what allows ideas to form. But the paper has to carefully couch speculation inside a “Motivations” section or other verbage to ensure the reader understands its place.

It’s extremely important to define concepts before using them. Terms like internal covariate shift or coverage sound like definitions without actually being such.

Attention is all you need

Wed, 05 Aug 2020 12:37:42 -0700

I also referred to this implementation to understand some of the details.

This is the paper describing the Transformer, a sequence-to-sequence model based entirely on attention. I think it’s best described with pictures.

model overview

From this picture, I think the following things need explaining:

embeddings these are learned embeddings that convert the input and output tokens to vectors of the model dimension. In this paper, they actually used the same weight matrix for input embedding, output embedding, and the final linear layer before the final softmax.
positional encoding: since there’s no concept of a hidden state or convolution that encodes the order of the inputs, we have to add some information about the position of the tokens. They used a sinusoidal positional encoding that was a function of the position and the dimension. The wavelength for each dimension forms a geometric progression from $2\pi$ to 10000 times that.
the outputs are “shifted right”
multi-head attention: see below for a description of multi-head attention. In the encoder-decoder attention layers, $Q$ comes from the previous masked attention layer and $K$ and $V$ come from the output of the encoder. Everywhere else uses self-attention, meaning that $Q$, $K$, and $V$ are all the same.
masked multi-head attention: in the self-attention layers in the decoder, we can’t allow positions to attend to positions ahead of themselves, so we set all right-connecting values in the input of the softmax (right after scaling; see the image below) to negative infinity.
feed-forward blocks these are two linear transformation with ReLU in between. The transformations are the same across each position, but they are different transformations from layer to layer, as you might expect.
add & norm: these are residual connections followed by layer normalization.

multi-head attention

The “Mask (opt.)” can be ignored because that’s for masked attention, described above.

BERT: pre-training of deep bidirectional transformers for language understanding

Tue, 04 Aug 2020 08:57:44 -0700

The B is for bidirectional, and that’s a big deal. It makes it possible to do well on sentence-level (NLI, question answering) and token-level tasks (NER, POS tagging). In a unidirectional model, the word “bank” in a sentence like “I made a bank deposit.” has only “I made a” as its context, keeping useful information from the model.

Another cool thing is masked language model training (MLM). They train the model by blanking certain words in the sentence and asking the model to guess the missing word.

Compositional generalization by factorizing alignment and translation

Mon, 27 Jul 2020 09:11:16 -0700

They had a biRNN with attention for alignment encoding, and then a single linear function of each one-hot encoded word for encoding that single word. Their reasoning was that by separating the alignment from the meaning of individual words the model could more easily generalize to unseen words.

Semi-supervised training for automatic speech recognition

Tue, 14 Jul 2020 08:06:00 -0600

This was Manohar’s PhD dissertation at JHU.

Chapter 2 provides a relatively clear overview of how chain and non-chain models work in Kaldi.

In chapter 3 he tried using negative conditional entropy as the loss function for the unsupervised data, and it helped a bit.

In chapter 4 Manohar uses [CTC loss]/paper/ctc/.

In chapter 5, he discusses ways to do semi-supervised model training. It’s nice when you have parallel data in different domains, because then you can do a student-teacher model. When there’s no parallel data, the best you can do is decode the unsupervised data with the seed model and use that to train the LF-MMI model (see section 5.2.1).

Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks

Fri, 10 Jul 2020 09:14:59 -0600

RNNs generally require pre-segmented training data, but this avoids that need.

Basically, you have the RNN output probabilities for each label (or a blank) for every frame, and then you find the most likely path across that lattice of probabilities.

The section explaining the loss function was kind of complicated. They used their forward-backward algorithm (sort of like Viterbi) to get the probability of all paths corresponding to the output that go through each symbol at each time, and then they differentiated that to get the derivatives with respect to the outputs. Then it was backpropagation as normal from that point.

Google's neural machine translation system: bridging the gap between human and machine translation

Tue, 30 Jun 2020 08:22:30 -0600

This model was superseded by this one.

They did some careful things with residual connections to make sure it was very parallelizable. They put each LSTM layer on a separate GPU. They quantized the models such that they could train using full floating-point computations with a couple restrictions and then convert the models to quantized versions.

Google's multilingual neural machine translation system

Fri, 26 Jun 2020 08:02:12 -0600

They use the word-piece model from “Japanese and Korean Voice Search”, with 32,000 word pieces. (This is a lot less than the 200,000 used in that paper.) They state in the paper that the shared word-piece model is very similar to Byte-Pair-Encoding, which was used for NMT in this paper by researchers at U of Edinburgh.

The model and training process are exactly as in Google’s earlier paper. It takes 3 weeks on 100 GPUs to train, even after increasing batch size and learning rate.

Japanese and Korean voice search

Wed, 24 Jun 2020 14:44:02 -0600

This was mentioned in the paper on Google’s Multilingual Neural Machine Translation System. It’s regarded as the original paper to use the word-piece model, which is the focus of my notes here.

the WordPieceModel

Here’s the WordPieceModel algorithm:

func WordPieceModel(D, chars, n, threshold) -> inventory:
 # D: training data
 # n: user-specified number of word units (often 200k)
 # chars: unicode characters used in the language (e.g. Kanji, Hiragana, Katakana, ASCII for Japanese)
 # threshold: stopping criterion for likelihood increase
 # inventory: the set of word units created by the model

 inventory := chars
 likelihood := +INF
 while len(inventory) < n && likelihood >= threshold:
 lm := LM(inventory, D)
 inventory += argmax_{combined word unit}(lm.likelihood_{inventory + combined word unit}(D))
 likelihood = lm.likelihood_{inventory}(D)
 return inventory

The algorithm can be optimized by

Towards a multi-view language representation

Tue, 23 Jun 2020 08:40:04 -0600

They used a technique called CCA to combine hand-made features with NN representations. It didn’t do great on typological feature prediction, but it did do well with predicting a phylogenetic tree for Indo-European languages.

Universal phone recognition with a multilingual allophone system

Tue, 23 Jun 2020 08:33:48 -0600

These guys made sure to model allophones. They had an encoder that produced a universal phone set, and then language-specific decoders. This meant they could use data from various languages to train the system. The decoder has an allophone layer, followed by other dense trainable layers. The allophone layer is a single trainable dense layer, but was initialized by a bunch of linguists who sat down and described the phone sets belonging to each phoneme in each language present in the training set. They added an L2 penalty to divergence from the original linguist-created matrix.

using Matrix

Sun, 02 Feb 2020 12:34:56 -0700

Matrix is a federated, open source chat system. Federation means that people can message each other across different servers, like in the image below. In that way, it works sort of like email: even though you may use you@gmail.com and I might use me@kylrth.com, we can still write each other emails.

In our case, I host the server at matrix.kylrth.com, and you and I can connect to it with various clients. We can write each other messages, but we can also communicate with people on other Matrix servers.There are a ton of cool features to Matrix, such as bridging, voice and video calls, and encryption. You can read a more extensive beginner’s guide to Matrix here. Also check out the Matrix website.

Increase in learning: spiritual patterns for obtaining your own answers

Wed, 20 Nov 2019 10:14:43 -0700

Chapter 1

We are give the opportunity to have the Spirit as a constant companion! To take advantage, we need to sincerely desire it, invite it through action, and be worthy of it through obedience.

Chapter 2

Knowledge is the accumulation of facts.

Understanding comes when we apply our hearts to knowledge, which lets the Holy Ghost testify to us of the truthfulness of it. Understanding comes by revelation.

Kyle Roth

AI research is dead, long live AI

Nonviolent communication: a language of life

feelings over judgments permalink

Thinking in systems: a primer

Bowling alone: the collapse and revival of American community

the trends permalink

The whole-brain child: revolutionary strategies to nurture your child's developing mind

Speaking as from the Dust: ideologies of AI and digital resurrection in Mormon culture

Better, Nicer, Clearer, Fairer: a critical assessment of the movement for ethical artificial intelligence and machine learning

The right side of history: how reason and moral purpose made the West great

Killing sacred cows: overcoming the financial myths that are destroying your prosperity

Raising an emotionally intelligent child: the heart of parenting

Why we sleep: unlocking the power of sleep and dreams

Team Human interview with Dennis Yi Tenen

writing technology permalink

InstructEval: systematic evaluation of instruction selection methods

thoughts on technology ownership

"Low-resource" text classification: a parameter-free classification method with compressors

the NYT AI explainer misses the point

the inherent subjectivity of reality

Winners take all: the elite charade of changing the world

for a socially beneficial and responsible development of AI

Think again: the power of knowing what you don't know

the value of rethinking permalink

helping others rethink permalink

the AI art debate

Army of none: autonomous weapons and the future of war

Conscious: a brief guide to the fundamental mystery of the mind

Artificial intelligence, values, and alignment

Unsolved problems in ML safety

Being mortal: medicine and what matters in the end

The big three in economics: Adam Smith, Karl Marx, and John Maynard Keynes

The drunkard's walk: how randomness rules our lives

facts I've observed about how my brain stores knowledge

The birth order book: why you are the way you are

thoughts on Effective Altruism

AlphaTensor: discovering faster matrix multiplication algorithms with reinforcement learning

Whisper: robust speech recognition via large-scale weak supervision

Selective annotation makes language models better few-shot learners

selective annotation method permalink

Trivial or impossible—dichotomous data difficulty masks model differences (on ImageNet and beyond)

Beyond neural scaling laws: beating power law scaling via data pruning

LocoProp: enhancing backprop via local loss optimization

the effects of scale on worst-group performance

Continual-T0: progressively instructing 50+ tasks to language models without forgetting

learning French

Multitask prompted training enables zero-shot task generalization (T0)

WordBurner beta

PaLM

QA-GNN: reasoning with language models and knowledge graphs for question answering

ethics drift within bubbles

notes about neuroscience

Experienced well-being rises with income, even above $75,000 per year

Neural message passing for quantum chemistry

paper summarization permalink

keep your tasks in the heap

The effect of model size on worst-group generalization

quotes from a Lex Fridman interview with Philip Goff

hosting a Tor onion service

getting started permalink

Scaling laws for the few-shot adaptation of pre-trained image classifiers

Learning explanations that are hard to vary

In search of robust measures of generalization

It's not just size that matters: small language models are also few-shot learners

Educated

thinking for yourself permalink

Scaling laws for transfer

Deep learning scaling is predictable, empirically

Masked autoencoders are scalable vision learners

Data scaling laws in NMT: the effect of noise and architecture

Parallel training of deep networks with local updates

A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification

paper summarization permalink

Learning transferable visual models from natural language supervision (CLIP)

Distributed representations of words and phrases and their compositionality

paper summarization permalink

negative sampling permalink

meaning-making in the post-modern world

Deep learning

feelings over judgments

the trends

writing technology

the value of rethinking

helping others rethink

selective annotation method

paper summarization

getting started

thinking for yourself

paper summarization

paper summarization

negative sampling

paper summarization

“the opinion of the many”

the fear of death is irrational

setting up the server

operating system

Nvidia drivers and the container runtime

science is at least as beautiful as magic

Edvard Munch, The Scream, 1893

Ben Shahn, All That Is Beautiful, 1966

Peter Doig, Architect’s Home in the Ravine, 1991

all at once

What is the significance of the Atonement?

playing God

broadband access

ethical AI

the algorithm

experiments for detecting easy/hard samples

problems

explanation vs. speculation

model overview

multi-head attention