Category Archives: Cooperation

Exploitation as a Turing test

A friend recently told me me that the ghosts that chase Pac-Man in the eponymous arcade game don't vary their behavior based on Pac-Man's position. At first, this surprised me. If, playing Pac-Man, I'm running away from one of the ghosts chasing me, and eat one of the special “energizer” pellets that lets Pac-Man eat the ghosts instead of vice-versa, then the ghost turns and runs away.

My friend responded that the ghosts don't start running away per se when Pac-Man becomes dangerous to them. Instead, they change direction. Pac-Man's own incentives mean that most of the time, while the ghosts are dangerous to Pac-Man, Pac-Man will be running away from them, so that if a ghost is near, it's probably because it's moving towards Pac-Man.

Of course, I had never tried the opposite – eating an energizer pellet near a ghost running away, and seeing whether it changed direction to head towards me. Because it had never occurred to me that the ghosts might not be optimizing at all.

I'd have seen through this immediately if I'd tried to make my beliefs pay rent. If I'd tried to use my belief in the ghosts' intelligence to score more points, I'd have tried to hang out around them until they started chasing me, collect them all, and lead them to an energizer pellet, so that I could eat it and then turn around and eat them. If I'd tried to do this, I'd have noticed very quickly whether the ghosts' movement were affected at all by Pac-Man's position on the map.

(As it happens, the ghosts really do chase Pac-Man – I was right after all, and my friend had been thinking of adversaries in the game Q-Bert – but the point is that I wouldn’t have really known either way.)

This is how to test whether something's intelligent. Try to make use of the hypothesis that it is intelligent, by extracting some advantage from this fact. Continue reading

Canons (What are they good for?)

People in the Effective Altruist and Rationalist intellectual communities have been discussing moving discourse back into the public sphere lately. I agree with this goal and want to help. There are reasons to think that we need not only public discourse, but public fora. One reason is that there's value specifically in having a public set of canonical writing that members of an intellectual community are expected to have read. Another is that writers want to be heard, and on fora where people can easily comment, it's easier to tell whether people are listening and benefiting from your writing.

This post begins with a brief review of the case for public discourse. For reasons I hope to make clear in an upcoming post, I encourage people who want to comment on that to click through to the posts I linked to by Sarah Constantin and Anna Salamon. For another perspective you can read my prior post on this topic, Be secretly wrong. The second section explores the case for a community canon, suggesting that there are three distinct desiderata that can be optimized for separately.

This is an essay exploring and introducing a few ideas, not advancing an argument. Continue reading

GiveWell: a case study in effective altruism, part 6

This is the last of a series of blog posts examining seven arguments I laid out for limiting Good Ventures funding to the GiveWell top charities. In this post, I articulate what it might look like to apply the principles I've proposed. I then discuss my prior relationship with and personal feelings about GiveWell and the Open Philanthropy Project.

A lot of arguments about effective altruism read to me like nitpicking without specific action recommendations, and give me the impression of criticism for criticism's sake. To avoid this, I've tried to outline here what it might look like to act on the considerations laid out in this series of posts in a principled way. I haven't constructed the arguments in order to favor, or even generate, the recommendations; to the contrary, I had to rewrite this section after working through the arguments. Continue reading

Don't panic. Think.

Otto Von Bismarck is supposed to have said that there is a special providence for drunkards, fools, and the United States of America. The people of the United States of America have repudiated that providence, in order to become a normal country. That protection has now been withdrawn.

It is a normal outcome for a presidential election in the Americas, with a constitutional system modeled after that of the United States, to empower an authoritarian strongman. We are, after enjoying more than two hundred years of our special providence, finally experiencing a normal outcome. This is bad news, but it is most likely not catastrophic news.

In my pre-election post, I outlined two main bad things about Trump:

  1. He is a threat to global political stability, might lead to a military conflict between great powers, and slightly increases the chances of a nuclear exchange.
  2. He is a threat to local political stability and might lead to the breakdown of civil order.

These things were real risks, and still are. They are very, very bad in expectation. But they are still fairly improbable. This is very bad news, but to run through the streets panicking would be committing a category error. To the extent that a Trump victory carries tail risk, we have already incurred that cost. We have already lost that measure. The only thing to do is manage the mainline scenarios.

(UPDATE: I basically endorse Paul Christiano's take on managing the tail risk.)

To respond reasonably to a Trump victory, we have to think clearly about the threats posed by a Trump regime, and the opportunities we have to change that.

I'm going to start by explaining why, while both those outcomes are real risks, the system is unlikely to suddenly collapse. Then I will explore what Trump's support means, and what we should do about it.
Continue reading

Matching-donation fundraisers can be harmfully dishonest

Anna Salamon, executive director of CFAR (named with permission), recently wrote to me asking for my thoughts on fundraisers using matching donations. (Anna, together with co-writer Steve Rayhawk, has previously written on community norms that promote truth over falsehood.) My response made some general points that I wish were more widely understood:

  • Pitching matching donations as leverage (e.g. "double your impact") misrepresents the situation by overassigning credit for funds raised.
  • This sort of dishonesty isn't just bad for your soul, but can actually harm the larger world - not just by eroding trust, but by causing people to misallocate their charity budgets.
  • "Best practices" for a charity tend to promote this kind of dishonesty, because they're precisely those practices that work no matter what your charity is doing.
  • If your charity is impact-oriented - if you care about outcomes rather than institutional success - then you should be able to do substantially better than "best practices".

Continue reading

GiveWell: a case study in effective altruism, part 5

This is part of a series of blog posts examining seven arguments I laid out for limiting Good Ventures funding to the GiveWell top charities. My prior post considered the third through fifth arguments, on influence, access, and independence. In this post, I consider the sixth and seventh arguments:

Argument 6: If no one else is willing to fund a program, then this is evidence that the program should not be funded. Crowding out other donors destroys this source of independent validation.

Argument 7: If Good Ventures fully funds every high-value giving opportunity it finds, this could lead to other donors preemptively abandoning programs the Open Philanthropy Project is looking into, thus substantially reducing the amount of effective giving in the Open Philanthropy Project's perceived current and potential focus areas.

Argument 6 is sometimes an important consideration, but is a poor fit for the GiveWell top charities, to the extent that most donors are already largely moved by GiveWell's recommendations. Argument 7 points to a real problem, and one that reflects poorly on the effective altruism movement, but the problem is not mainly concentrated in the area of funding. Continue reading

Vote-trading and personal honor

The 2016 US presidential election is likely unusually important, because Trump seems unusually likely to damage global coordination in ways that increase the risk of major wars – and to damage US political norms in ways that are likely to accelerate the decline of discourse and governance.

This is also an election in which the libertarian candidate has been unusually viable because he has any experience at all as a major government figure – despite his apparent lack of interest in the sorts of things a president needs to know about, such as other countries. Many people also want to register a protest vote with Green Party candidate Jill Stein, since they find Hillary Clinton's respectable establishment liberal misconstruals of the true and the good objectionable, and prefer disrespectable anti-establishment left-wing misconstruals of the true and the good.1

Many people agree that Trump is terrible and it would be much less bad if Clinton wins, but some people prefer a third-party candidate and are unwilling to simply back the lesser of two evils. Some people end up favoring a vote for Clinton on net; others favor a third party vote. Both types are distributed over many states.

A protest vote has the same value anywhere. Federal funding also becomes available for any party that gets more than 5% of the popular vote – and it seems like Johnson's share of the vote could pass that threshold. On the other hand, due to the US electoral college system, the cost of forgoing a Clinton vote has very different effect depending on which state you're voting in. In a "safe state" overwhelmingly likely to go to one of the two major candidates, your vote has very little effect on the outcome of the election. But in a "swing state" where the outcome is more in doubt, your vote has a comparatively large effect on the outcome. Scott Aaronson points out that this distinction creates the opportunity for gains from trade, and has been promoting the idea of vote-swapping in order to reconcile these interests. The idea is that one or more Clinton supporters in safe states pledge to vote for a particular third-party candidate, in exchange for a third-party voter pledging to vote for Clinton.

In a one-to-one swap, this keeps third party national percentages the same, but increases the chance the swing state goes for the desired candidate. This is enough to yield gains from trade if both sides share a preference for one major-party candidate over the other. But even if that's not true, a many-to-one swap can still create gains from trade, by increasing both the chance that the desired major-party candidate wins, the third-party candidate's share of the vote total.

One of my friends recently suggested that we can't trust this system not to be gamed by Trump voters. I think that this is mistaken. Continue reading

References   [ + ]

1. This is not about the vaccines thing, which seems overblown – Stein seems basically correct there, that mandated vaccines are good but people have justified distrust in the medical establishment which needs to be addressed.

GiveWell: a case study in effective altruism, part 4

This is part of a series of blog posts examining seven arguments I laid out for limiting Good Ventures funding to the GiveWell top charities. My prior post considered the second argument, that even assuming symmetry between Good Ventures and other GiveWell donors, Good Ventures should not fund more than its fair share of the top charities, because it has a legitimate interest in preserving its bargaining power. In this post, I consider the third through fifth arguments:

Argument 3: The important part of GiveWell's and the Open Philanthropy Project's value proposition is not the programs they can fund with the giving they're currently influencing, but influencing much larger amounts of charitable action in the future. For this reason it would be bad to get small donors out of the habit of giving based on GiveWell recommendations.

Argument 4: The amount of money Good Ventures will eventually disburse based on the Open Philanthropy Project's recommendations gives them access to potential grantees who would be interested in talking to one of the world's very largest foundations, but would not spend time on exploratory conversations with smaller potential donors who are not already passionate about their program area.

Argument 5: GiveWell, the Open Philanthropy Project, and their grantees and top charities, cannot make independent decisions if they rely exclusively or almost exclusively on one major donor. They do not want Good Ventures to crowd out other donors, because it makes them more dependent on Good Ventures, which will reduce the integrity of their decisionmaking process, and therefore the quality of their recommendations.

If you already think that GiveWell is doing good, Argument 3 should make you more excited about it - and implies that Good Ventures should be looking for ways to give away money faster in order to build a clear track record of success sooner.

Argument 4 seems plausible at some margin - if Good Ventures gives away most of its money quickly, then it will become a small foundation and have access to fewer potential grantees. But it would be a surprising coincidence if the amount of money Good Ventures will eventually give away were very close to this access threshold. If giving money away freely now will make it difficult to change behavior later once remaining funds are close to the access threshold, this is an argument for communicating this intention in advance, which may require Good Ventures and its donors to make their funding commitments more explicit.

I deal with two components of Argument 5 separately. First, GiveWell's top charities may become less effective if dependent on a single primary donor. Second, GiveWell and the Open Philanthropy Project have a legitimate interest in preserving their own independence.

The top charities independence consideration seems unlikely to uniformly apply to all the GiveWell top charities; each has a different funding situation and donor base, so this seems like a situation worth assessing on a case-by-case basis, not with a blanket 50-50 donation split between Good Ventures and everyone else.

To the extent that Good Ventures becoming the dominant GiveWell donor threatens GiveWell's institutional independence, this problem seems built into the current institutional structure of GiveWell and the Open Philanthropy Project, in ways that aren't materially resolved by Good Ventures only partially funding the top charities. Continue reading

GiveWell: a case study in effective altruism, part 3

This is part of a series of blog posts examining seven arguments I laid out for limiting Good Ventures funding to the GiveWell top charities. My prior post considered the first argument, that the Open Philanthropy Project, and thus Good Ventures, has superior judgment to that of GiveWell donors. In this post, I consider the second argument:

Even if Good Ventures isn't special, it should expect that some of its favorite giving opportunities will be ones that others can't recognize as good ideas, due to different judgment, expertise, and values. If the Open Philanthropy Project does not expect to be able to persuade other future donors, but would be able to persuade Good Ventures, then these opportunities will only be funded in the future if Good Ventures holds onto its money for long enough. Thus, while Good Ventures may currently have a lower opportunity cost than individual GiveWell donors, this will quickly change if it commits to fully funding the GiveWell top charities.

This post is my most direct response to GiveWell's blog post explaining the reasoning behind its "splitting" recommendation.

Argument 2: Bargaining power

In its blog post on giving now vs later, GiveWell discusses potential policies it might have recommended to Good Ventures on funding the GiveWell top charities' funding gap. Good Ventures and individual GiveWell donors may have very different opinions on what else their money should be spent on, but still agree that the optimal allocation of resources should prioritize the GiveWell top charities.

Without holding the view that Good Ventures currently has a higher opportunity cost than individual GiveWell donors, GiveWell might still believe that committing to fully funding the GiveWell top charities' funding gaps would be a mistake on the part of Good Ventures. GiveWell might believe that this commitment would be bad because it cedes all of Good Ventures's bargaining power to other GiveWell donors.

GiveWell begins with a principled argument, asking whether Good Ventures should respond to each additional dollar given by other GiveWell donors by giving less ("funging"), more ("matching"), or the same amount ("splitting"). GiveWell recommends splitting, and in the first major section, I explore the principled case for this, assuming the conditions of symmetry laid out above. I argue that the principled case for splitting is only coherent under very pessimistic assumptions about effective altruists' ability to cooperate with one another. These assumptions may be justified, but as far as I can tell, haven't been seriously tested.

GiveWell goes on to make a specific recommendation that Good Ventures's "fair share" of the GiveWell top charities is 50% of the top charities' total room for more funding. In the second major section of this post, I see whether this recommendation seems intuitively fair, trying a couple of different simple back-of-the-envelope quantitative comparisons. I argue that the most intuitive relative allocation assigns substantially more of the funding burden to Good Ventures at present. Continue reading

Locker room talk

Locker room dystopias

Recently, presidential candidate Donald Trump made the news because he was caught on tape bragging about how women let him get away with sexually assaulting them because he is a star. He and his supporters have defended him on the basis of this being ordinary locker room banter. Part of what I think is important to me about this is that I perceive Trump and his supporters are making a threat-of-isolation power play. The implied threat is:

If you complain about this sort of behavior, you will be alone. There are no allies to be found. Any group of men you appeal to will back up your abuser. Every woman you know has already accepted her place in this game.

This helps explain why there has been such strong pushback against Trump's comments (e.g. professional athletes saying they've never heard anything like this in their locker rooms), and why this is important. In an interview with Trump supporter Rudy Giuliani, journalist Jake Tapper responds with incredulity at Giuliani's claim that this is a normal way for a man to talk: Continue reading