Notes

more examples appreciated . . .

• Close open loops - Oliver Habryka
• Advice for prospective startup founders - Max Hodak
• Abstract advice to researchers tackling the difficult core problems of AGI alignment - Tsvi Benson-Tilsen
• Advice (10-20yos) - Patrick Collison
• What Should You Do With Your Life? Directions and Advice - Alexey Guzey
• How To Do What You Love - Paul Graham
• How To Do Great Work - Paul Graham
• How I've run major projects by Ben Kuhn
• To listen well, get curious by Ben Kuhn
• Attention is your scarcest resource by Ben Kuhn
• Staring into the abyss as a core life skill by Ben Kuhn
• My favorite essays of life advice by Ben Kuhn
• How To Be Successful by Sam Altman
• Salary Negotiation by Patrick McKenzie
• For potential Ph.D. students by Ravi Vakil
• The "Three Things" Exercise for getting things out of talks by Ravi Vakil
• On writing by Terence Tao
• Advice for the Young Scientist by John Baez
• How to write mathematics badly by Jean-Pierre Serre
• Life is Short by Paul Graham
• You and Your Research by Richard Hamming
• An Opinionated Guide to ML Research by John Schulman
• Strong Inference by John R. Platt
• Cargo Cult Science by Richard Feynman
• Principles of Effective Research by Michael Nielsen
• Cormac McCarthy on Scientific Writing

We'd like to understand a mind's effects on the world. To do this, we'd like to understand the structures / elements which determine / designate mind trajectories (preferably over long horizons).

Descriptively, minds can often be said to have goals. The minds we currently observe most often have goals, and goal-oriented frames provide short, accurate compressions of mind outputs.

How does a mind come to have a goal? All observed minds, artificial or otherwise, do not exist absent amental structure. They are grown through a deterministic (enough) process, on silicon or biological substrate, such that all goal-drivers have a "physical" representation in the corresponding mind-artifact. "Physical" explanations for goal development include:

• tracing the development of circuits in a neural network over the course of training;
• endocrinal perspectives on the upregulation / downregulation of various hormones leading to increased sex drive / etc.;
• neuroscientific perspectives on "the steering system" and reward functions lying therein;
• evolutionary perspectives on "why we seek out sweet food" and, perhaps contentiously, on social behaviors and psychology writ large.

Each is a valid explanation in context. Yet I'd argue none properly engage with "mentalistic" aspects of mind: the structures / elements of mind-systems unified by their predominance in minds and lack elsewhere. Insofar as we expect goals to be common properties of minds, we'd expect the structures / elements of minds to contribute to their development.

Tsvi hypothesizes that a dominant driver of human behavior is a reflective process in which a human mind finds itself ascribing goals in accordance with what goals it would make sense for the human mind to have conditioning on the past history of its actions.¹ He calls this "the Fictitious Imputed Adopted Telos hypothesis." FIAT provides plausible, "mentalistic" explanations for goal development, in that a mind's drive for coherence (a mentalistic property) lets it amplify more "hard-coded" drives into goals that are more general than the initial activation state of the drive, by generalizing from commonalities between past experience.

Differentiating flavors of "fictitious imputed adopted telos":

• the underlying reflectory dynamics are mind-like, so perhaps FIAT is a dynamism that applies across mind-like structures, in a way that wouldn't be expected for "evolutionary" drive amplification arguments (in particular Mateusz Bagiński suggests an instiantiation during the development of Jewish monotheism). little bit tricky because many systems undergo evolutionary optimization
• imitative learning in humans is a well-supported hypothesis, and imitative dynamics possess a lot of explanatory power. What differentiates FIAT? FIAT is ambiguous with many imitative perspectives
• nomenologically, how does FIAT generalize? I expect FIAT to embody something like an abstraction hierarchy

With regards to the latter: attractive and repulsive dynamics are not symmetric in a relevant way; the specification of "do(x)" is crisper than the specification of "not do(x)." Even taking a predictive perspective, space of trajectories ending in "x" is much smaller than space of trajectories not ending in "x." Expanding x-space seems easier? in the do(x) vs not do(x) case, because credit attribution for the causal factors at ending up in not do(x) trajectories is harder because the space is larger. So you'd expect FIAT to "do worse" here.

(this is what I'm trying to call "negatory goal development")

Is this negatory goal development particularly worse for FIAT than otherwise? Well, I expect non-FIAT / "physical" alternatives to generalize less well in "not do(x)" cases, so total negatory goal development via e.g. imitative methods is probably smaller.

There's a story you could tell here where a lot of the pathological aspects of minds come from FIAT-induced negatory goal development. Bateson's schismogenesis hypotheses are, as far as I can tell, disjoint or at the very least operating not on the same hierarchical level (it's FIAT-accepting-agnostic). I also think these "negatory" patterns are better explicated in Braitenberg or Braitenberg-peers.

Ideally we'd study goal-development in purely epistemic settings? Not sure if our bounded reasoning models are high-fidelity enough, or whether they'll give us any interesting insights.

[taken from Thom's Structural Stability and Morphogenesis]

Meiosis is the process by which germ cells divide into gametes in preparation for organismal reproduction. It differs from mitosis in numerous ways, but in particular that the ultimate daughter cells are haploid (only retaining one copy of each chromosome as opposed to two).

Thom proposes a geometric model of meiosis based on the following principle:

at the pachytene stage, the local genetic state of the chromatic tetrad is the result of a competition between the genetic regimes defined by the parent chromosomes joined in the synaptic complex.

The pachytene stage of meiosis is a substage of prophase I where homologous chromosomes fully pair and genetic recombination occurs; it determines the haplotype of each chromosome in the resulting gametes. This haplotype determination is synonymous with "the local genetic state of the chromatic tetrad:" the tetrad consisting of the paired homologs and "the local genetic state" referring to the specific allele arrangements. Complexities of the these mechanisms (e.g. chiasma) are hidden under "competition." The synaptic complex is the protein structure binding the homologous chromosomes (maternal-paternal pairs).

Represent the synaptic complex by the cylinder $x^2+y^2=1$ in ${\mathbb{R}}^3,$ where the half $x<0$ represents (say) the maternal chromosome and the half $x>0$ represents the paternal one. Insofar as pachytene "determines" recombination between non-sister chromatids, this motivates the addition of a "shockwave" which cuts each plane of the cylinder (e.g. $z$ constant) as some convex curve $c(z)$ through the origin, such that when associating each gene $g_i$ with a plane $z=g_i$ the lines $x=0$ and $c(z)$ separate the cylindrical cross-section into four chromatids.

The case where each $c(z)$ is a straight line corresponds to the case where every recombination is symmetric. Generally, the chromatids correspond to continuous regions cut out by $c(z)$ as one travels up the $z$-axis, so this is proposed to be a satisfactory, geometric model for recombination: small local changes, the large geometric ones satisfying the models correspond to known behavior.

For instance, chiasma:¹

Hence our model of meiotic crossing over will give the following situation associated with chiasma: a narrow zone of gene conversion bounded by two zones of multiple fractures of total odd order, each involving a pair of distinct chromatids.

I think I take the structural fidelity of frames more seriously than many around me.

If I pay too much attention to the words I'm speaking or writing, I become nauseous.¹ If I pay too much attention to the words others are speaking or writing, I become nauseous and also relatively misanthropic. This is minimized when I'm operating inside idiolects; my current favorite is Finnegans Wake.

Writing tires me because engaging in conceptual wheat-and-chaff separation, as necessary to dictate words to page, generates nearly irremovable, painful, mental detritus. I don't know how to "garbage collect" while preserving the reproducibility of the generative thought-patterns. Some information is in the artifact, and some is in my head. Their relationship is complicated and not easily compressible!

Relatedly, it's very difficult for me to ingest someone else's "vocabulary."² Locally operating within it is fine. It requires some suspension of disbelief but I've had a lot of practice. Retaining those insights, however, is much harder.

I only feel clean when frames are correct, coherent, and minimal. No one has frames that are correct, coherent, and minimal. The burden of the boundedly-rational agent is to manage this tradeoff. My frames are certainly nowhere near optimal. But translation costs are real.

I'd really like to develop a healthier relationship with confusion. (I wish it wasn't physically aggravating). My best guess at how to do this is to find deep frames I'm satisfied with. Tips appreciated.

The "[t]echnologist" remains . . . valorized? (at least, in certain San Franciscan subcultures). Tech people who have {passion, taste, discernment}, intrepid changemakers seeking to leverage innovation as a force for good. Perhaps it's Lindy: the word was coined in Bentham's Chrestomathia to describe an expert in "Technology," the "connected view" of arts and manufactures, exactly two centuries ago.

Certainly they play an integral role in a modern society. I do, however, take some issue with:

A “technologist” is anyone who thinks critically about the opportunity for technology to improve society.

(1) Too broad. "Technologists" are builders. The democratization of internet tech came a unification of thought and practice. (The Collisons when building Stripe)

(2) Too narrow. Frontend gardeners and Linus Torvalds provide taste as a public good; curated creation raises the quality waterline; artists shouldn't be excluded.

(3) I worry the predominance of tech, the centralization of progress in tech, takes too much mindshare of the thoughtful and ambitious. Startups are vehicles of a certain type. Venture provides stimulus of a certain type. There are other types.

What will it say about our time that the greatest discoveries were made in industry? Hermits and the military made the atom bomb.¹ Billionaires and not-quite-hermits are making superintelligences, mind uploads, aging cures.

Maybe the not-quite-hermits are not-hermits for lack of places to hermit. Technologisms cannibalize. Memetic centralization may be as dangerous as economic centralization.

Valor is worth its weight in gold. To me, society lacks hermit heroes. There's a place to start.