Minimum viable impact purchases

Several months ago I did some work on a trial basis for AI Impacts. It went well enough, but the process of agreeing in advance on what work needs to be done felt cumbersome. It's not uncommon that midway through a project, it turns out that it makes sense to do a different thing than what you'd originally envisioned - and because I was doing this for someone else, I had to check in at each such point. This didn't just slow down the process, but made the whole thing less motivating for me.

Later, I did my own research project. When natural pivot points came up, this didn't trigger a formal check-in - I just continued to do the thing that made the most sense. I think that I did better work this way, and steered more quickly towards the highest-value aspect of my research. Part of this is because, since I wasn't accountable to anyone else for the work, I could follow my own inner sense of what needed to be done.

I was talking with Katja about my work, and she mentioned that AI Impacts might potentially be interested in funding some of this work. I explained the motivation problem mentioned in the prior paragraphs, and wondered out loud whether AI Impacts might be interested in funding projects retrospectively, after I'd already completed them. Katja responded that in principle this sounded like a much better deal than funding projects prospectively, in large part because it would take less management effort on her part. This also felt like a much better deal to me than being funded prospectively, again because I wouldn't have to worry so much about checking in and fulfilling promises.

I've talked with friends about this consideration, and a few mentioned the fact that sometimes people are hired as researchers with a fairly vague or flexible research mandate, or prefunded to do more like their prior work, in the hope that they'll produce similarly valuable work in the future. But making promises like that, even if very abstract, also makes it difficult for me to proceed in a spirit of play, discovery, and curiosity, which is how I do some of my best work.

It also offends my sense of integrity to accept money for the promise to do one thing, or even one class of thing, when my real plan is to adopt a flexible stance - my best judgment might tell me to radically change course, and at this stage I fully intend to listen to it. For instance, I might decide that I should switch from research to writing and advocacy (what I'm doing now). I might even learn something that persuades me to make a bigger commitment to another course of action, starting or join some organization with a better-defined role.}

What doesn't offend my sense of integrity is to accept money explicitly for past work, with no promises about the future.

Then it clicked - this is the logic behind impact certificates. Continue reading

Some excerpts from Catistotle's Kittycatean Ethics

Every art and every inquiry, and likewise every action and pounce, seems to aim at some red dot, and hence it has been beautifully said that the red dot is that at which all cats aim. But a certain difference is apparent among ends, since some are ways of being at play, while others are certain kinds of works produced, over and above the being-at-play.
[...]
If, then, there is some end of the things we do that we want on account of itself, and the rest on account of this one, and we do not choose everything on account of something else (for in that way the pounces would go beyond all bounds, so that desire would be empty and pointless), it is clear that this would be the red dot, and in fact the reddest dot. Then would not an awareness of it have great weight in one's life, so that, like mousers who see a mouse, we would be more apt to hit on what is needed? But if this is so, one ought to try to get a grasp, at least in outline, of what it is and to what kind of knowledge or capacity it belongs.
[...]
And it would seem to belong to the one that is most governing and most a master art, and politics appears to be of this sort, since it prescribes which kinds of knowledge ought to be in the cities, and what sorts each cat ought to learn and to what extent; also, we see that the most honored capacities, such as mousing, household cattery, and meowing skill, are under this one. Since this capacity makes use of the rest of the kinds of knowledge, and also lays down the law about what one ought to do and from what one ought to refrain, the end of this capacity should include the ends of the other pursuits, so that this end would be the feline red dot. For even if the red dot is the same for one cat and for a city, that of the city appears to be greater, at least, and more complete both to achieve and to preserve; for even if it is achieved for only one cat that is something to be satisfied with, but for a litter or for cities it is something more beautiful and more divine. So our pursuit aims at this, and is in a certain way political.
[...]
Now taking up the thread again, since every kind of knowing and every pounce reach toward some red dot, let us say what it is that we claim politics aims at, and what, of all the dots aimed at by action, is the reddest. In name, this is pretty much agreed about by the majority of cats, for most cats, as well as those who are more refined, say it is being in a box, and assume that living well and doing well are the same thing as being in a box. But about being in a box-what it is-they are in dispute, and most cats do not give the same account of it as the wise. Some cats take it to be something visible and obvious, such as pleasure or wealth or honor, and different ones say different things, and even the same cat often says different things; when sick one thinks it is health, but when poor, that it is wealth, and when they are conscious of ignorance in themselves, cats marvel at those who say it is something grand and above them. And some cats believe that, besides these many red dots, there is some other red dot, by itself, which is also responsible for the being red of all these other dots.

Lessons learned from modeling AI takeoffs, and a call for collaboration

I am now able to think about things like AI risk and feel like the concepts are real, not just verbal. This was the point of my modeling the world project. I’ve generated a few intuitions around what’s important in AI risk, including a few considerations that I think are being neglected. There are a few directions my line of research can be extended, and I’m looking for collaborators to pick this up and run with it.

I intend to write about much of this in more depth, but since EA Global is coming up I want a simple description to point to here. These are just loose sketches, so I'm trying to describe rather than persuade.

New intuitions around AI risk

  • Better cybersecurity norms would probably reduce the chance of an accidental singleton.
  • Transparency on AI projects’ level of progress reduces the pressure towards an AI arms race.
  • Safety is convergent - widespread improvements in AI value alignment improve our chances at a benevolent singleton, even in an otherwise multipolar dynamic.

Continue reading

Yell at Mars to call swans

GLENDOWER. I can call spirits from the vasty deep.

HOTSPUR. Why, so can I, or so can any man;

But will they come when you do call for them?

- Shakespeare, Henry IV, Part 1

Calling swans

Recently a dear friend invited me to join them as they took their wedding photos, at the Palace of Fine Arts. There's a pond next to the structure, and across the pond we saw one of the swans who reside there. Someone observed that it would have been nice to take a picture with the swan. So I called out, in a loud and clear voice, "Excuse me! Would you come over here?" and beckoned. Repeatedly.

I was pretty sure that it wouldn't work. Swans don't understand spoken language. Even if they did, as far as I could tell they have no plausible motive to respond.

The swan turned towards us and swam halfway across the pond. As it slowed down, my companions thought of more ways to get its attention, ways that seemed more likely to work on a swan, like tossing things into the water. But my plan did more than nothing.

It's an important skill, to be able to come up with plans like that. Sometimes you need to notice when things are impossible, and give up. But other times, it's worth at least trying the plan "yell at the swan."

What heuristic was I using? I'm not sure, but I think it has to do with noticing that my model of the world is incomplete. Continue reading

Why have my parents gotten wiser as I have gotten older?

“When I was a boy of 14, my father was so ignorant I could hardly stand to have the old man around. But when I got to be 21, I was astonished at how much the old man had learned in seven years.” - Attributed (probably spuriously) to Mark Twain

The usual explanation for this is that teenagers are too foolish to understand the advice of their elders. But there’s another obvious explanation: their parents accumulate life experience that makes them wiser over those seven years.

Not all experience is created equal, and the rearing of a child all the way to adulthood is likely a substantial source of new wisdom and experience that are difficult to acquire in other ways beforehand.

When I was a child, I felt like my grandfather had a lot more perspective to offer than my father had. Some of this might just have been a different context for our interactions; most of my interactions with my dad were about day-to-day stuff. But some of this might have been that my grandfather actually had more experience.

As I talk with my dad now, it seems more and more clear that he has some sorts of wisdom and perspective I wasn’t aware of earlier. For instance, it seems like he’s more aware than before that when you have a child, you’re not buying into some set lifestyle, but instead you’re buying a chance at a highly uncertain set of outcomes. This makes me more relaxed about talking with him, because it feels more like if I do things he doesn’t agree with, he knew this was part of the deal in advance.

My mom has also talked about acquiring wisdom that she didn’t have before, in ways that have made conversations with her go better. For instance, I think we’ve both recently learned a lot about setting boundaries.

If this hypothesis is true, then the natural thing to do is to tell kids, not to listen to their parents more, but to listen to people of their grandparents’ generation more, to the extent that they’re available. It also seems like I should prioritize making more friends who are at least a few decades older than I am.

To the extent that this hypothesis is true, we should expect the last child in a long series to report this effect less than firstborns. So, my questions for you are:

  1. How many years between your parents’ firstborn and your birth? (0 if you were a firstborn.)
  2. How true does Twain’s observation seem for you, that parents seem to get wiser over time?

Effective Altruism is not a no-brainer

Ozy writes that Effective Altruism avoids the typical failure modes of people in developed countries intervening in developing ones, because it is evidence-based, humble, and respects the autonomy of the recipients of the intervention. The basic reasoning is that Effective Altruists pay attention to empirical evidence, focus on what's shown to work, change what they're doing when it looks like it's not working, and respect the autonomy of the people for whose benefit they're intervening.

Effective Altruism is not actually safe from the failure modes alluded to:

  • Effective Altruism is not humble. Its narrative in practice relies on claims of outsized benefits in terms of hard-to-measure things like life outcomes, which makes humility quite difficult. Outsized benefits probably require going out on a limb and doing extraordinary things.
  • Effective Altruism is less evidence based than EAs think. People talk about some EA charities as producing large improvements in life outcomes with certainty, but this is often not happening. And when the facts disagree with our hopes, we seem pretty good at ignoring the facts.
  • Effective Altruism is not about autonomy. Some EA charities are good at respecting the autonomy of beneficiaries, but this is nowhere near central to the movement, and many top charities are not about autonomy at all, and are much better fits for the stereotype of rich Westerners deciding that they know what's best for people in poor countries.
  • Standard failure modes are standard. We need a model of what causes them, and how we're different, in order to be sure we're avoiding them.

Continue reading

PSA on outing

If your friend is out about something to one group (say, their friends) and closeted to another group (e.g. their family or workplace) it's generally wrong to out them without their consent. If you want to avoid doing this, you should not tag or full-name them in public posts that refer to the thing they're closeted about, even on social media with privacy settings such as Facebook. If you do this, anyone who searches for them will be able to find your post. There is no privacy setting that can prevent this from being searchable, at least on Facebook. Nor does blocking such posts from their wall prevent them from being searchable on yours.

Examples of things people may be closeted about in some contexts include but are not limited to sexual orientation (e.g. homosexuality or bisexuality), gender identity (e.g. being transgender), atheism, polyamory, mental illness or disability (e.g. having bipolar or depression), or physical illness or disability.

They can't say this in public posts, because the fact that there is a secret is already more info than they want to disclose. But I don't have that kind of problem - so it falls on me to post this sort of PSA.

Copernican revolutions, lordship, and bondage

I got a wonderful compliment from a friend recently.

They had mentioned that I was a good host - that I got some important things right (this on its own made me feel recognized) - and expressed some worry that I might feel unappreciated. Another of their friends had expressed the sense that their contributions to the community weren’t being reciprocated. From what my friend could observe of my behavior I was acting fairly similarly, and they were worried I’d burn out.

I responded to the effect that I am already "burnt out" in the sense that I'm only doing things if they feel worth doing with no expectation of reciprocation. (My other motives are finding it intrinsically motivating to do good for others, and empowering allies to do good things more generally.) But, I continued, I was sad that I'm not setting off a success spiral where other people perceive the public goods I'm contributing to as benefiting them, and try to reciprocate by generating more public goods of a similar kind. A sort of public goods success spiral, where people do more, not just because they have more resources & like people, but because they perceive themselves to live in an environment with prosocial norms.

My friend responded that my behavior had inspired them to be a better host, and given them affordances of things to do, that wouldn’t have occurred to them otherwise. They gave a concrete example: offering people water when they come in. And this was exactly the thing I'd wanted to happen, and had sort of given up on.

I shouldn’t have given up. But I should have expected it to be hard, because it implies a level of spiritual development that you can’t just skip ahead to. I should have known this, because I've read Hegel's Phenomenology of Spirit. Continue reading

Group cognition

I don’t even see groups

I want to foreground part of the subtext in my recent post on community and my problems with it. One underlying problem appears to me to be that I simply don’t perceive groups. Slate Star Codex writes about how important group membership is for making friends:

If I had written this essay five years ago, it would be be titled “Why Tribalism Is Stupid And Needs To Be Destroyed”. Since then, I’ve changed my mind. I’ve found that I enjoy being in tribes as much as anyone else.

Part of this was resolving a major social fallacy I’d had throughout high school and college, which was that the correct way to make friends was to pick the five most interesting people I knew and try to befriend them. This almost never worked and I thought it meant I had terrible social skills. Then I looked at what everyone else was doing, and I found that instead of isolated surgical strikes of friendship, they were forming groups. The band people. The mock trial people. The football team people. The Three Popular Girls Who Went Everywhere Together. Once I tried “falling in with” a group, friendship became much easier and self-sustaining precisely because of all of the tribal development that happens when a group of similar people all know each other and have a shared interest. Since then I’ve had good luck finding tribes I like and that accept me – the rationalists being the most obvious example, but even interacting with my coworkers on the same hospital unit at work is better than trying to find and cultivate random people.

Scott’s original social strategy is exactly how I go about making friends. Where it didn’t work, I just kept upgrading my social skills for one-on-one interactions. This I think is part of why some people think I have unusually poor social skills, and others say I have unusually good ones. I developed them unevenly relative to the norm. Continue reading

Projects of philosophers

Someone once described Michael Vassar to me as paying attention not to the conclusions but to the intellectual processes of people he’s talking to, not in order to conform to them, but out of curiosity, in case they are using some valuable heuristic he should add to his toolbox. I often find that I get something different than usual from philosophers because I read them this way. I find it jarring when people casually refer to Nietzsche’s philosophy (to name one example) to reference some proposition or other he's famous for asserting. It seems natural to me to be referring to his methods, but his specific conclusions seem like almost a weird irrelevancy. I think this is one of the most valuable things I got out of my St. John’s education - the ability to read thinkers to figure out what their project was, rather than a bunch of specific propositions they were advancing. This is my attempt to share this way of reading, by working through an example. Continue reading