All essays & notes
Essay

Physics from the Inside: Could Randomness Be a Shadow of State Geometry?

On limited observation, deeper structure, and the search for simpler rules

An intricate teal and gold curved-line sculpture above a simpler two-dimensional projected shadow
A complex structure and its reduced projection: a visual metaphor for a complete state and what an observer can see. AI-generated illustration.

We usually imagine scientific investigation as something done from the outside. We build a machine, inspect its parts, vary its inputs, and infer the rules connecting cause and effect.

Fundamental physics has no such comfortable vantage point.

We are inside the system we are trying to understand. Our instruments, our clocks, and the physical processes that make observation possible all belong to the same world whose architecture we hope to reconstruct.

This makes physics a remarkable kind of reverse engineering. We observe spectra, collisions, correlations, gravitational effects, and the traces of cosmic history. From these outputs, we try to infer the smallest set of rules capable of generating them.

Perhaps some of the difficulty comes from complexity. But another possibility interests me: the variables available to us may describe only part of the complete state.

Some of the difficulties we are encountering in The Emergent Frame have brought me back to this question:

Could apparently probabilistic behavior be the observable shadow of a deeper, deterministic state geometry?

Knowing a rule is not knowing a state

A deterministic law fixes what happens next once the complete current state is specified. It does not guarantee that an observer knows that state.

Let x denote the complete state, and let a deterministic rule advance it:

xt+Δt​=F(xt​).

Suppose an observer can access only a reduced description,

y=π(x).

The map π might retain a few observables while discarding internal orientation, phase, correlations, or other information. Several complete states could then give the same observed state y.

Imagine two such states, x1​ and x2​. They look identical through the observational map:

π(x1​)=π(x2​).

Yet after the same deterministic update, their observable outcomes could differ:

π(F(x1​))=π(F(x2​)).

An observer who knows only y cannot select a unique next outcome. The effective description may require probabilities even though the complete evolution is deterministic.

There is a useful converse. If every complete state with the same observed value always evolves to the same next observed value, then a closed deterministic rule on the observed variables is possible. Projection alone does not force randomness. What matters is whether the discarded information affects later observations.

Probability needs more than missing information

This distinction explains why a probability description might be useful. It does not yet tell us which probabilities to assign.

For that, we need a distribution over the possible underlying states: a preparation procedure, a justified measure, or some other rule specifying how states are sampled.

If μ is a normalized probability measure on the complete state space X, and A is an observable event in Y, its projected probability is

P(A)=μ(π−1(A)).

The measure is doing real work here. Counting unspecified hidden possibilities is not a derivation of a probability law. Geometry and dynamics would have to justify the relevant measure, including its relation to preparation and observation.

The reduced description may also retain memory. Earlier observations can constrain the hidden state, so the present value yt​ need not contain everything useful for predicting the next outcome.

The research question is therefore richer than “Can deterministic rules look random?” We need to ask when restricted observations produce a stable, reproducible statistical description, and why that description takes the particular form seen in experiments.

A lesson from deterministic programs

Computing offers an accessible illustration.

A pseudorandom generator can produce a long sequence from a fixed seed using entirely deterministic updates. Someone who knows the algorithm and seed can reproduce it. Someone who sees only the output may find its next value difficult to predict.

Cryptography sharpens the distinction between possessing information and being able to use it. A relation can be mathematically definite while recovering its hidden input from the public output is computationally difficult. In that case, the obstacle may be computation rather than information lost through a many-to-one projection.

These are different mechanisms, but they share a lesson: knowing the law does not necessarily give an observer practical access to the state.

This is why I find the computing analogy useful: what looks unpredictable from the output can still have a definite structure behind it.

What I mean by state geometry

The phrase “deeper state space” can sound like a proposal for an extra spatial dimension. That is not required.

A state space contains the variables needed to describe a system. Even an ordinary mechanical system can require position and momentum; its state dimension exceeds its spatial dimension without adding another direction in which an object can travel.

The distinction is already useful in TEF’s work on helical line families. There, a selected three-dimensional effective position description is accompanied by an independent internal phase. The resulting local state space is four-dimensional, while the chosen effective position space is three-dimensional.

For me, this geometric distinction makes the question tangible: how much of a complete state does an assigned position actually reveal?

For the present question, “state geometry” is deliberately broad. A candidate might involve internal phases, relational data, constraints, or other degrees of freedom. Its content must eventually be specified through equations and observables.

Why the Hodge conjecture caught my attention

The Hodge conjecture enters this reflection through a different question: how can an abstract mathematical invariant correspond to a concrete geometric structure?

For a smooth projective complex algebraic variety V, Hodge theory organizes cohomology into components,

Hk(V,C)=p+q=k⨁​Hp,q(V).

The conjecture concerns rational classes of type (p,p). It asks whether every such class can be expressed as a rational linear combination of classes represented by algebraic cycles of complex codimension p:

α∈H2p(V,Q)∩Hp,p(V),α=i∑​qi​[Zi​],qi​∈Q.

Here the rational cohomology is understood through its image in complex cohomology. The general conjecture remains open; the Clay Mathematics Institute’s overview explains its relationship between abstract classes and algebraic cycles.

What attracts me is that relationship. If the conjecture holds, such a class lies in the rational span of classes arising from algebraic cycles.

This suggests a question for physical model building: could some stable observable regularities eventually be traced to distinguished structures in a complete state geometry?

What interests me here is the possibility of giving an abstract regularity a geometric home. I would like to understand whether a similar relationship could connect physical observations to structures in the underlying state space.

Quantum physics sets the harder test

It is easy to construct classical examples in which incomplete information produces probabilities. Reproducing quantum statistics is much more demanding.

Bell’s theorem constrains hidden-variable explanations of quantum correlations. In a locally causal model with a hidden-state distribution independent of the measurement settings, correlations obey Bell inequalities. Determinism alone does not evade that constraint. Bell’s original paper establishes the central obstruction.

For me, the interesting question is how a state geometry would connect measurement, the Born probabilities, and quantum correlations while preserving the absence of controllable faster-than-light signalling. The difficulty lies in that connection, not simply in imagining more hidden variables.

Reverse engineering with fewer rules

The broader ambition remains familiar. Newtonian mechanics brought terrestrial and celestial motion into one framework. Maxwell connected electricity, magnetism, and light. General relativity made spacetime geometry central to gravitation.

What draws me to these examples is their explanatory economy: phenomena that seemed separate became consequences of a shared structure.

Imagine a creature whose entire observable world is one-dimensional. It can measure positions, distances, motion, and correlations with extraordinary precision. Yet if the line it inhabits belongs to a larger structure, its native variables may be unable to express that structure directly. It could spend centuries refining an increasingly accurate one-dimensional theory without finding the variables in which a simpler explanation becomes natural.

The difficulty may not be that the underlying rule is complicated. It may be that the observer does not possess the variables in which the rule is simple.

This is a question of representation, not merely of missing information. The variables available to us might describe observations beautifully while making their common origin difficult to express. An effective theory can be extraordinarily accurate and still obscure the architecture that generates it. That possibility interests me as much as the question of randomness itself: an accurate description need not be a natural description.

Where TEF is currently stuck

TEF itself illustrates why this question matters. We have obtained several partial constructions: geometric, kinematic, and effective-field descriptions. But they do not yet arise from one common microscopic dynamics. We can describe particular structures and explore their consequences; what remains missing is a shared account of how those structures arise and connect.

The current bottleneck is therefore not another isolated calculation. It is whether a simpler state structure can make several presently separate sectors emerge from the same rules. Can geometry, matter, and interactions become consequences of a common architecture rather than constructions assembled one at a time? This is what brings me back to complete states and observational maps: perhaps we have been trying to connect descriptions that are already partial.

If every phenomenon requires another independent assumption, we may be reconstructing the complexity of known physics under different names rather than explaining it. I want to know whether there is a simpler level at which those connections become natural.

Keeping the question open

There is a productive tension here. We hope to explain more of nature through fewer principles, while remaining inside the system whose principles we seek. The difficulty of reconstructing a world from within is also what makes the problem so compelling to me.

For me, this raises a larger question:

Can an observer embedded within a physical system reconstruct the architecture that generates both the observable world and the observer itself?

The question about probability sits inside that larger problem:

Could the probabilities we observe reflect invariant structure that our observable variables do not fully resolve?

What I hope to find is a connection between geometry, evolution, and observation that makes the laws we see feel less like separate facts and more like aspects of one underlying structure.