Tag Archives: Experiment

Null Results

Humans tend to look for evidence that reinforces our beliefs, not evidence that contradicts them. This is called confirmation bias. A related problem is that people tend to publish results that show various treatments or interventions working, but not results that show them failing to work, because the latter is less interesting. This is called publication bias.

In the spirit of combating those things, I'm going to share some things that didn't work for me.

Rolfing

Rolfing is basically a kind of massage (though Rolfers insist there's a difference) with a slight amount of evidence that it produces feelings of relaxation over longer periods of time than other massage methods. Rolfers are certified by the Rolfing Institute, and put you through a series of ten sessions. Then you're done, permanently. They say it takes a few months for the changes to become manifest, and it's been a few months, so I'm now ready to talk about what it did for me.

Rolfing is famously uncomfortable. During sessions I was asked to give lots of feedback about how intense the pressure was on a scale from 1 to 10 (the target was something like 6 or 7), and since I am a tough guy idiot I was reluctant to actually say "hey, that's an 8," so I probably got some unnecessary pain. Some of the parts of the massage are also kind of weird-feeling, like the session that focused on my chest - I'm not used to strong pressure on my ribs or sternum, but ultimately it was bearable and I felt very good after the sessions.

During the sessions, when the pressure was particularly intense and a little painful I turned it into kind of a mindfulness practice. I would focus on the sensation, in detail, and to some extent that defused the distress. This should be familiar to many people who have meditated. And it's positive evidence for the effectiveness of meditation in teaching a certain kind of mental control.

The really nice thing about Rolfing was that I finally got a massage with a sufficient amount of pressure. Even though it was sometimes painful, it was a satisfying experience. I think this preference for massages with a lot of pressure might be hereditary. When I was a kid I'd give my mom shoulder massages so hard my hands hurt, and that was just barely enough pressure for her. I'm totally the same way - I've never gotten a shoulder massage that had too much pressure - although I have had just barely enough, during Rolfing. If you don't like lots of pressure you probably won't like Rolfing.

Some minor aspects of my posture seemed to be improved, and I got unsolicited compliments on my posture from people who didn't know I was going through a course of Rolfing, but now several months later I seem to have backslid significantly. For example, my feet, which used to point outwards but after Rolfing pointed straight ahead, point outwards again. I haven't experienced the kind of enhanced bodily awareness that I heard about anecdotally that initially interested me in Rolfing.

On the whole it wasn't a huge win, but might have just barely been worth the time and money for me. If you don't have or make much money, and don't have reasons other than general well-being to try it, I can't recommend it; it's too costly an experiment.

Anonymous Feedback Form

On Less Wrong, Gwern wrote about putting up a personal anonymous feedback form. I thought the idea was really cool, so I put one up myself January 16th. There are links on my about page on Twitter, Facebook, and here. Aside from my own tests, I have received zero responses.

Hypotheses:

1) I'm not doing anything wrong, so nobody needs to get in touch with me.

2) Nobody really cares what I do so they're not motivated to provide feedback.

3) The feedback form is insufficiently visible.

4) Somehow the link or form is broken. (I had a friend test it out and it seemed to work.)

5) I'm so much less popular than Gwern that zero results over two months is not very far statistically from the expected results

Love

"Has your kind really evolved separate information-processing mechanisms for deoxyribose nucleic acid versus electrochemical transmission of synaptic spikes?"

"I don't really understand the question's purpose," Akon said.  "Our genes are made of deoxyribose nucleic acid.  Our brains are made of neurons that transmit impulses through electrical and chemical -"

The fake man's head collapsed to his hands, and he began to bawl like a baby.

[...]

The fake man suddenly unfolded his head from his hands.  His cheeks were depicted as streaked with tears, but the face itself had stopped crying.  "To wait so long," the voice said in a tone of absolute tragedy.  "To wait so long, and come so far, only to discover that nowhere among the stars is any trace of love."

"Love?" Akon repeated.  "Caring for someone else?  Wanting to protect them, to be with them?  If that translated correctly, then 'love' is a very important thing to us."

"But!" cried the figure in agony, at a volume that made Akon jump.  "But when you have sex, you do not untranslatable 2!  A fake, a fake, these are only imitation words -"

"What is 'untranslatable 2'?" Akon said; and then, as the figure once again collapsed in inconsolable weeping, wished he hadn't.

"They asked if our neurons and DNA were separate," said the Ship's Engineer.  "So maybe they have only one system. [...]"

"They share each other's thoughts when they have sex," the Master of Fandom completed.  "Now there's an old dream.  And they would develop emotions around that, whole patterns of feeling we don't have ourselves...  Huh.  I guess we do lack their analogue of love."

-Three Worlds Collide

I used to think that traditional literary descriptions of the emotion of sexual love were just hyperbole for dramatic or comic effect, and really people just felt caring or lust or both. Recently I found out that some people say they actually have those sensations in that way, though I still can't quite alieve it yet.

Here's an interesting take on the emotion of love. Read the whole thing. It's good.

What I want to know about this is - do you recognize this emotion? Forget the author's opinions about what you should do about love, and what love means, for a moment - forget it's even called love. Does the author's description of how the emotion manifests physically ring true? Is this one specific recognizable feeling that you have actually experienced?

I don't recognize this sensation at all:

Love is a feeling. It's hot and fluttery and tingly. I get it in my guts and chest and face. The feeling is accompanied by a series of enthusiastic thoughts, such as "This person is the greatest person ever", "I wonder how I can make this person feel good", and/or "I want to climb onto this person and put my face close to their face and smoosh my body onto their body."

I know what feeling goes along with the first thought. And I know what feeling goes along with the third. (The second is ambiguous.) They're completely different feelings. I don't think my experience is vanishingly rare, either - but I'm not sure, which is why I want to hear from you, whether or not you've experienced the emotion described in the linked article.

I've had deep feelings of caring toward some people. I've experienced admiration. I've experienced lust. I've had all these feelings toward some people. But for me they are totally different feelings: they feel completely different. And I never thought of using the word "love" to describe any of them, alone.

Generally, when I say I love someone, I am talking about a more permanent disposition, over a longer period of time than a single emotion. I mean that I generally feel caring toward them, and don't expect that disposition to change in the foreseeable future.

I think there's a specific emotion or combination of feelings that some people experience, and mean when they talk about love, that other people never experience. It's hard for me to write this, because even thinking about this makes me worry that I'm defective, that I'll never love the people I care about the way they would want to be loved.

I think people who talk about love often talk past each other because of the typical mind fallacy, and the illusion of transparency. People who have that "love" emotion see other people who don't bonding romantically and talking about love, and assume that they feel the same thing inside. People who don't have that emotion see people who do making long-term commitments and talking about love, and assume that it's just a word for the behavior.

I don't like summarizing anything from Wittenstein's Philosophical Investigations because unlike some other books, an adequate summary would be nearly as long as the whole thing. But in one place Wittgenstein gives an analogy for purely personal subjective experience.

Imagine that everyone carries around a small box. And no one looks into any box aside from their own. But people say "there is a beetle in my box", and refer to the thing in their box as a beetle. Now, is it meaningful to ask whether someone else's box really has a beetle? It's not a falsifiable statement - after all, you're not going to look inside someone else's box to find out whether their "beetle" looks like yours. The subjective experience of love is very similar to this beetle in a box.

Not quite, though - the description in the linked article is specific enough that I can tell whether I've had that experience. It would be nice to have a more precise vocabulary of emotions and sensations, to avoid this type of confusion. I wonder what other confusions are caused by a large number of people missing out on some "universal human experiences."

Of course I have the alternate hypothesis that people who describe love as an emotion are the mistaken ones. That they're just giving another name to lust, or to caring, when experienced under special circumstances. But limerence is apparently a real thing, and if that's real, then the milder feelings of romantic love are less improbable, so they're likely real as well.

Doubt, Science, and Magical Creatures

Doubt

I grew up in a Jewish household, so I didn't have Santa Claus to doubt - but I did have the tooth fairy.

It was hard for me to believe that a magical being I had never seen somehow knew whenever any child lost their tooth, snuck into their house unobserved without setting off the alarms, for unknown reasons took the tooth, and for even less fathomable reasons left a dollar and a note in my mom's handwriting.

On the other hand, the alternative hypothesis was no less disturbing: my parents were lying to me.

Of course I had to know which of these terrible things was true. So one night, when my parents were out (though I was still young enough to have a babysitter), I noticed that my tooth was coming out and decided that this would be...

A Perfect Opportunity for an Experiment.

I reasoned that if my parents didn't know about the tooth, they wouldn't be able to fake a tooth fairy appearance. I would find a dollar and note under my pillow if, but only if, the tooth fairy were real.

I solemnly told the babysitter, "I lost my tooth, but don't tell Mom and Dad. It's important - it's science!" Then at the end of the night I went to my bedroom, put the tooth under the pillow, and went to sleep. The next morning, I woke up and looked under my pillow. The tooth was gone, and in place there was a dollar and a note from the "tooth fairy."

This could have been the end of the story. I could have decided that I'd performed an experiment that would come out one way if the tooth fairy were real, and a different way if the tooth fairy were not. But I was more skeptical than that. I thought, "What's more likely? That a magical creature took my tooth? Or that the babysitter told my parents?"

I was furious the possibility of such an egregious violation of experimental protocol, and never trusted that babysitter in the lab again.

An Improvement in Experimental Design

The next time, I was more careful. I understood that the flaw in the previous experiment had been failure to adequately conceal the information from my parents. So the next time I lost a tooth, I told no one. As soon as I felt it coming loose in my mouth, I ducked into the bathroom, ran it under the tap to clean it, wrapped it in a tissue, stuck it in my pocket, and went about my day as if nothing had happened. That night, when no one was around to see, I put the tooth under my pillow before I went to sleep.

In the morning, I looked under the pillow. No note. No dollar. Just that tooth. I grabbed the incriminating evidence and burst into my parents bedroom, demanding to know:

"If, as you say, there is a tooth fairy, then how do you explain THIS?!"

What can we learn from this?

The basic idea of the experiment was ideal. It was testing a binary hypothesis, and was expected to perfectly distinguish between the two possibilities. However, if I had known then what I know now about rationality, I could have done better.

As soon as my first experiment produced an unexpected positive result, just by learning that fact, I knew why it had happened, and what I needed to fix in the experiment to produce strong evidence. Prior to the first experiment would have been a perfect opportunity to apply the "Internal Simulator," as CFAR calls it - imagining in advance getting each of the two possible results, and what I think afterwards - do I think the experiment worked? Do I wish I'd done something differently? - in order to give myself the opportunity to correct those errors in advance instead of performing a costly experiment (I had a limited number of baby teeth!) to find them.

Cross-posted at Less Wrong.