Monthly Archives: May 2024

LLMs for language learning

My current outlook on LLMs is that they are some combination of bullshit to fool people who are looking to be fooled, and a modest but potentially very important improvement in the capacity to search large corpuses of text in response to uncontroversial natural-language queries and automatically summarize the results. Beyond this, I think they’re massively overhyped. The most aggressive hype is that they are an AGI development project - in other words, that they’re close to being conscious, generative minds on the same order as ours, which can do as wide a range of tasks as a human. This is clearly false. The more moderate hype is that they can do meaningful generative work within the domain where they were trained: written language content (which can of course be converted to and from audio language content pretty well). For instance, they might in some limited sense be able to internally represent the content of the language they're indexing and reproducing. This would necessarily entail the capacity for "regular expressions for natural language." I believe that even this much more limited characterization is false, but I am less confident in this case, and there are capacities they could demonstrate that would change my mind. Language learning software seems like a good example. It seems to me that if LLMs contain anything remotely like the capacity of regular expressions for natural language that take into account the semantic values of words, they should make it relatively easy to create a language learning app that is strictly better than the best existing automated resources for smartphone users trying to learn the basics of a new-to-them language.

Continue reading

Gentrification and Nationalism

In the Feudal system that succeeded the old Roman empire, owning land predominantly meant holding the right to tax the people who lived on that land. One could in principle do other things to those people (if you can't credibly threaten to destroy or expel them, it's hard to collect taxes), but for the most part, taxes and labor levies were the best use feudal lords knew how to or cared to make of their lands. There were some limitations and exceptions, determined by a combination of custom, law, and explicit contracts. Peasants farmed to survive, and often improved the land they lived on because of a customary expectation that they'd get to benefit from the improvements.

The bourgeois revolution of the 17th century, pioneered by the Dutch Calvinists, replaced the old feudal property system with one oriented around owner-operators, whose clear title to the land they interacted with meant that they could more profitably improve it, and borrow against their assets to finance such improvements. This led to a productivity advantage for areas that operated on the new rules.

Theodore Herzl is widely regarded as the founder of modern Zionism. His substantive proposal was for Jews to buy cheap land in Palestine from people living under Ottoman rule, improve it, and live in the newly valuable land. This was expected to be a sustainable trade in part because wealthy Ottoman landholders were in practice operating according to older, feudal customs, collecting rent from peasant tenants. Herzl's Ashkenazi Jews, well positioned to convert this land to a bourgeois system, could capture more economic value than they paid the land's prior owners.

These trades were mutually beneficial among the explicit parties to the transaction, but since peasants who were thereby forced off the land frequently had no legal claim to it under Ottoman law, they were generally not compensated for their loss. This sort of change in property regime is similar to Britain's earlier experience with the Enclosure Acts, and created similar sorts of social dysfunction. It is also similar to smaller-scale processes of displacement called "gentrification," in which people with access to new higher-wage jobs in an area - or more generally, people with a relation to the state that allows them to capture more of the value created by activities they are involved in - rent or buy homes that used to be occupied by people with lower incomes, thus driving up home rents and forcing the old tenants in an area to move elsewhere. The situation is also similar to cases where leveraged buyouts allowed outside investors to purchase companies, and increase shareholder profits by breaking promises made by management to employees.

In all these cases, part of the profit of the trade comes from exploiting the difference between the older customary mode of cooperation, and the explicit rights recognized by the central enforcing authority.

Whether the people thus disenfranchised were defenseless because they had been operating according to higher-trust assumptions, or whether they had simply accepted a bad deal because they didn't have the power to negotiate a better one with their oppressors, it is reasonable for such people to interpret their displacement in terms of conflict rather than economics. If the trade is genuinely one that increases total value, it ought to be possible to compensate the losers adequately for their loss, and not to do so constitutes a sort of aggression, even if lawful. And if the trade cannot be structured in a way that leaves everyone better off, then it is simply a transfer of wealth from some people to others, and thus zero-sum.

The gentrification story is incomplete; it cannot explain why Europe's Jews were the ones executing this trade - and as an explicitly collective enterprise. This happened in part because bourgeois capitalist revolution coexisted with zero-sum territorial competition among emerging European states.

Continue reading