My current outlook on LLMs is that they are some combination of bullshit to fool people who are looking to be fooled, and a modest but potentially very important improvement in the capacity to search large corpuses of text in response to uncontroversial natural-language queries and automatically summarize the results. Beyond this, I think they’re massively overhyped. The most aggressive hype is that they are an AGI development project - in other words, that they’re close to being conscious, generative minds on the same order as ours, which can do as wide a range of tasks as a human. This is clearly false. The more moderate hype is that they can do meaningful generative work within the domain where they were trained: written language content (which can of course be converted to and from audio language content pretty well). For instance, they might in some limited sense be able to internally represent the content of the language they're indexing and reproducing. This would necessarily entail the capacity for "regular expressions for natural language." I believe that even this much more limited characterization is false, but I am less confident in this case, and there are capacities they could demonstrate that would change my mind. Language learning software seems like a good example. It seems to me that if LLMs contain anything remotely like the capacity of regular expressions for natural language that take into account the semantic values of words, they should make it relatively easy to create a language learning app that is strictly better than the best existing automated resources for smartphone users trying to learn the basics of a new-to-them language.
Continue readingCategory Archives: Education
Happy Birthday to My Firstborn Baby Boy: A Memoir
Once I had my first couple of gout attacks, I read somewhere that people who'd experienced both said it was more unpleasant than childbirth, that supposedly indescribable suffering by which women martyr themselves for the continuation of the human race. Gout sure is painful, but not indescribably or infinitely so. It just hurts a lot in one spot, and more if there's even slight pressure on it - enough pain that at times I experienced it as patterns of light rather than an embodied sensation. There is no virtue in suffering, but if I could thereby make a new person, composed of a mixture of the core instructions for building my own body and those for somebody else I loved who would help me care for and cultivate that new person, then I would go off allopurinol for long enough to endure a few days of pain. My reproductive partner can speak for herself if she wishes, but my impression of labor was that it bore little resemblance to the acute panicked episodes depicted on television and in popular movies. Several months of deep massage by Valentin Rozlomii doubtless helped, as did some movement exercises she found on YouTube (some curb walking earlier in the day, and the Miles Circuit later at night), and half a tab of acid shortly before labor. By the time we arrived at the hospital, she was fully dilated and ready to give birth.
Labor, it turns out, is aptly named. It is not inherently torturous; it is a great deal of work, which calls for strength, flexibility, and stamina, for which one can be more or less ready for. Like many sorts of labor, birth labor is more of a distressing ordeal if one is simultaneously attempting to maintain a class persona with its attending stereotyped patterns of stiffness and selective dissociation. And like many other sorts of labor, it can be made onerous by various efforts at coercive extraction.
On the "due" date, my partner's ob/gyn did not consult with her about her preferences, her situation, or likely risks and benefits, but simply informed us that she was scheduling an induction in a week's time. The expedients mentioned above were a successful attempt to autoinduce just before the deadline, after which we had been advised that induction might not be available if we didn't accommodate the schedule. We remain skeptical that they would have refused in a true medical emergency; it was most likely a compliance scare tactic. Even so, it worked at least a little.
Once we were set up in a hospital room, the nurses issued strident instructions to my partner about how to pose, and how to push. Afterwards, my partner told me that she wished I'd advocated harder to give her space, as the instructions had served only to confuse her, contradicting her own experience of her body - especially, instructing her to experience pushing out a baby as though it felt like defecation, even though she could tell perfectly well that a different pattern of muscular activity was needed. Such instructions might perhaps be helpful for women who do not understand their own bodies well enough to distinguish between their reproductive and digestive musculature - though I suspect there is no clear, intersubjectively verifiable evidence for this like a randomized controlled trial - but were actively harmful in this case. Eventually, the nurses relented and gave her some time to rest, and my partner was able to tune in to her own body and make measurable progress on freeing our baby from her body, but she was so exhausted from following bad instructions that she agreed to a vacuum-assisted extraction, which, fortunately, not only succeeded at bringing the baby out into the world, but does not seem to have inflicted any lasting harm.
I had likewise heard and read many times that caring for a newborn is a torturous ordeal, like a forced march or sleep deprivation torture. What I have found is that caring for my baby in his first year of life was not torture or an unnatural-feeling ordeal. What it is, is a lot of work, which limits how much other work one can do at the same time without compromising one's health.
Continue readingSome casual thoughts on forming an educational library
This is a relatively low-effort post, though the result of quite a lot of thinking. It's a transcript of a conversation with ChatGPT.
Me
The book Thinking Physics has a widely acclaimed pedagogical method and is uniquely well-regarded in its field. Please recommend similarly good educational books on different topics.
Continue readingThere seems to be a market failure in cultivating children's agency.
Watching this monkey eating a banana has me thinking about the market for nondestructive education:
My son is learning to orient in space by manipulating the banana. There's a natural reward involved in figuring out how to rotate the banana correctly, distinguishing between the sides in an internal model rather than gradient-descending towards one end (which may or may not yield the sweet flesh inside), figuring out the difference between bringing the peel to his mouth and bringing the inside.
The biggest thing that distinguishes this from how I mostly see people treating babies is patience - I had to sit through him getting confused and a little frustrated multiple times, and distinguish between challenges big enough for him to process, and the point where he was about to spiral into helpless sadness, and only intervene in the latter case. And of course I had to make other active choices as well, like giving him a banana, and not "baby food."
For some particular skills or fields that a child expresses an interest in, it may make sense to employ domain experts, but - especially at the beginning - it seems to me like what's most needed is someone to arrange an enriched environment in the first place, and give the child both the stimulation and the room to investigate freely the sorts of things that would be valuable for them to investigate.
More recently, he responded to me playing a few simple songs for him on the ukulele at first by bucking his hips in a simple "dance," but soon afterwards by deciding he'd rather figure out how to pluck the strings himself.
Another example - at early ages, the "language program" that would make most sense, would be to hire native speakers of the target languages, chosen on the basis of how valuable the target language is and the availability of suitable native speakers, just like my partner and I choose his foods and toys based on suitability. These native speakers wouldn't mainly have the job "language teacher," but "playmate" - around and willing to play with the children exclusively or primarily in their native language. Depending on the scale of the overall program, children could to some extent choose how much to engage with this, just like my son chooses to play with some objects more than others.
At present, I don't know how to pay for that kind of curation and facilitation oriented child care at any scale that would free up my time. I keep hearing good things in the abstract about things like Montessori schools, but in practice, it doesn't seem like the people I know have access to this sort of thing, no matter how much money they're willing to throw at the problem, no matter how well-connected they are - to the contrary, the success rate in having one's child accepted by any school as worthy of attention seems surprisingly low. People tend to talk around the problem, using language around developmental disability or autism - but they do so in cases where their child is very obviously not autistic, just very slightly rambunctious and uncowed. The majority of the families I'd have regarded as most promising seem to only barely have access to schooling at all.
Which would suggest offering to sell it instead - but my impression is that there's no market for it either at a price that would satisfy the Law of Iron Wages, i.e. be adequate to pay for the reproduction of my skilled labor.
Related but not the same thing: https://www.takingchildrenseriously.com/
Towards optimal play as Villager in a mixed game
On Twitter, Freyja wrote:
Things capitalism is trash at:
- Valuing preferences of anything other than adults who earn money (i.e. future people, non-humans)
- Pricing non-standardisable goods (i.e. information)
- Playing nicely with non-quantifiable values + objectives (i.e. love, ritual)
Things capitalism is good at:
- Incentivising the production of novel goods and services
- Coordinating large groups of people to produce complex bundles of goods
- The obvious: making value fungible
Anyone know of work on -
a) integrating the former into existing economic systems, or
b) developing new systems to provide those things while including capitalism's existing benefits?
This intersected well enough with my current interests and those of the people I've been discoursing with most closely that I figured I'd try my hand at a quick explanation of what we're doing, which I've lightly edited into blog post form below. This is only a loose sketch, I think it does reasonably precisely outline the argument, but many readers may find that there are substantial inferential leaps. Questions in the comments are strongly encouraged.
Any serious attempt at (b) will first have to unwind the disinformation that claims that the thing we have now is capitalism, or remotely efficient.
The short version of the project: learning to talk honestly within a small group about how power works, both systemically and as it applies to us, without trying to hold onto information asymmetries. (There's pervasive temptation to withhold political information as part of a zero-sum privilege game, like Plato's philosopher-kings.) Continue reading
Zetetic explanation
There is a kind of explanation that I think ought to be a cornerstone of good pedagogy, and I don't have a good word for it. My first impulse is to call it a historical explanation, after the original, investigative sense of the term "history." But in the interests of avoiding nomenclature collision, I'm inclined to call it "zetetic explanation," after the Greek word for seeking, an explanation that embeds in itself an inquiry into the thing.
Often in "explaining" a thing, we simply tell people what words they ought to say about it, or how they ought to interface with it right now, or give them technical language for it without any connection to the ordinary means by which they navigate their lives. We can call these sorts of explanations nominal, functional, and formal.
In my high school chemistry courses, for instance, there was lots of "add X to Y and get Z" plus some formulas, and I learned how to manipulate the symbols in the formulas, but this bore no relation whatsoever to the sorts of skills used in time-travel or Robinson Crusoe stories. Overall I got the sense that chemicals were a sort of magical thing produced by a mysterious Scientific-Industrial priesthood in special temples called laboratories or factories, not things one might find outdoors. Continue reading