Moral Mazes as Transformative Treatment

An email I sent that I thought was worth sharing here:

Dear Professor Healy,

In many cases it seems like what you're calling Transformative Treatments amounts to coerced preference falsification.

Transformative Treatments as Preference Falsification

To start with an example you use, academic microeconomic ideological indoctrination superficially transforms students by making it clear that if they don't posture as economic agents, then they fail the test.

There's a cover story that you should prefer to have VNM-consistent preferences because otherwise you can be money-pumped / dutch-booked. That argument wouldn't persuade a rational agent; they'd already have consistent object-level preferences, and wouldn't additionally have a preference for having consistent preferences. But the threat of sneering at people with inconsistent preferences works on people who are controlled by social fear. In this case the "transformation" being wrought is from one simulacral set of preferences to another. The transformation is effected by appealing to some deeper prefences, such as a preference not to be sneered at.

Psychiatry seems like a mixed case. One friend reports that when she was institutionalized, there were improv classes in the psych ward to teach the prisoners how to pass as normal. It was quite clear and nearly explicit that passing as normal - rather than any other more objective requirement of sanity - was the requirement for release, and that if you tried to discuss the fact that normality was an act with the doctors, they'd consider you still insane.

By contrast another friend reports begging her parents not to put her back on mood-blunting antidepressants, but once back on antidepressants, feeling indifferent about the situation. This seems closer to a pure case of "transformative treatment," though it's fortunately at least somewhat reversible in her case.

Professionalism as a Transformative Treatment

In Fuck Nuance you pointed out that adding dimensions to a model can make it harder to pin things down, and that this kind of nuance actually reduces the precision of a model. In other words, nuance is opposed to using models to understand, predict, and act in the world. I'm not sure whether you meant to imply this but I think this is often motivated by a preference for obfuscation, and against clarity. And I think that this preference is the result of a transformational treatment.

I worked at Fannie Mae for six years, starting in 2008. During this time I was inducted into something I don't have a good term for, but Robert Jackall's book Moral Mazes does a good job explaining features of it (see Sarah Constantin's summary or Zvi Mowshowitz's selected quotes). I usually call the mindset I'm talking about "moral mazes," though sometimes I call it "business sadomasochism" or professionalism. The year after that I worked for a charity evaluator where I helped defraud billionaire donors with fake research in order to poison African children.

I didn't go in with the intent to commit fraud or hurt people, and it would have been better for my career to complain about this to the billionaire donors. But I was only hired because I'd been conditioned to respond to certain kinds of social cues as commands to participate in epistemic violence. In the process my mind had been broken to the point where, even after I quit, it took me five years to be able to talk about what I'd done and why. Happy to go into detail if you're curious.

The mindset I was inducted into seems to be a coalitional strategy that cooperates with itself within any given social network against anyone not playing its game. I think it's a form of CPTSD; it has the characteristic attributes of a compulsion to pass along the trauma and side with transgressors. As far as I can tell it's occupied most official institutions (Cf the idea of a damaged ruling class), which is why, among other things, the global COVID response has been so dismally bad, and also why Silicon Valley VCs mostly can only fund pyramid schemes now.

A friend of mine went to Davos to pitch his cryptocurrency startup, and found himself devoting so much of his visual processing to managing his social position that he developed facial-recognition problems, mistakenly assuming that a Swiss guy talking about Bhutan was Bhutanese. I myself developed related vision problems while participating in a conversation in which someone well-connected in nonprofit finance tried to induct an honest entrepreneur I know into the Moral Mazes pattern.

By now you may have the accurate impression that I'm describing some sort of Lovecraftian nightmare world. Often people experience being informed about a transgression as a demand to publicly side against the transgressor. I'm not asking for that, or for credulous agreement. What I and my friends need right now are more people who can discuss the problem rationally - including finite, reasonable skepticism! - and are willing to think carefully about what to do in response.

Are you interested in talking sometime?

6 thoughts on “Moral Mazes as Transformative Treatment

  1. Kenny


    I'm not ready to admit, now, that you're exactly right about this dynamic, but I am willing to admit that it's very possible; very plausible even. Sad!

    The "professionalism" essay is particularly dark!

    I feel like I _need_ to find some positive way to spin this – for my own safety!

      1. Kenny

        I was being a little dramatic, but I was thinking of this as something like an 'infohazard' that could cause depression/despair – you did write [emphasis mine]:

        > By now you may have the _accurate impression_ that I'm describing some sort of Lovecraftian nightmare world.

        Learning you're living in a Lovecraftian nightmare world (or _more_ of a nightmare world than you'd already accepted) is pretty dispiriting!

        1. Benquo Post author

          Psychedelics and high-quality bodywork can help a *lot*, as can having people to talk with.


Leave a Reply

Your email address will not be published. Required fields are marked *