Dynamics and uncertainty

Differential equations provide a universal language that is shared across many disciplines of scientific modeling and computing. This framework allows for understanding and predicting complex phenomena via rigorous analysis and simulation. Many laws describe deterministic variations of the state of a system with one free variable (usually time) or with multiple (usually (time and) space). Such deterministic relationships are then formulated as ordinary (ODEs) and partial differential equations (PDEs). However, there are cases—and we will get to know a specific one—in which it makes sense to deliberately model parts of a system as “random”. From my experience, stochasticity is usually incorporated into a model if …

• … parts of the system are understood but explicitly modeling them would introduce a degree of complexity that is consciously avoided.
• … parts of the system are explained by different plausible models and choosing one of them is consciously avoided.
• … parts of the system are not (properly, completely) understood and random noise is the best we got to model this component at all.

Introducing a stochastic component into the model allows for explicitly encoding uncertainty about the model or individual components. “Random noise”, after all, does not mean that you know nothing. It just implies that multiple—even infinitely many—solutions of the same system are plausible. (And, yes, perhaps one also knows nothing). Communicating uncertainty is crucial in many applications—in particular critical ones, such as weather- and climate modeling, finance, disease spread, and many more.

The formal language that allows us to introduce stochasticity into a dynamical system are, you guessed it, stochastic differential equations (SDEs). Unlike ODEs and PDEs, solutions of SDEs are not functions, but probability distributions over functions. These solutions are known as stochastic processes. Drawing samples from a stochastic process yields trajectories that are plausible realizations of the dynamics system under the chosen noise model. But more on that in a second. Before we dive into the technical details, let me briefly introduce the practical example that allows us to connect technical terms and symbols to interpretable concepts and quantities.

A simple economy

We have a closer look at SDEs in the context of a specific dynamical system: an economy in which multiple (actually: infinitely many) agents or individuals earn income, accumulate wealth, and consume goods. Here’s a rough overview.

In this massively simplified model of an economy there exists one single good, say apples. People in our economy, whom I at times call “individuals” or “agents”, receive a certain amount of this good as income. We decide against a realistic, complicated model for income dynamics. Instead, the income each agent receives follows the trajectory of a stochastic process and the agents have no way to influence what they earn. They cannot work harder to earn more or tend their apple tree in hope of a better harvest. This is what economists would call an exogenous force; it enters the model “from outside” without the possibility for anyone to control it.

Agents can do exactly two things with each apple they receive: consume it immediately or give it to another individual in the economy. Consumption, of course, brings joy—or utility—for the individual. This utility increases further and further—though at a diminishing rate—the more an individual consumes. But the economy would be quite boring if everybody just received a random amount of apples, which they immediately eat and gain a varying degree of joy from it. So there is a second aspect to the individuals’ dynamics, other than income.

Whenever an individual chooses not to consume everything they earn—and we will see in Part 2 why and how they decide that—their only other option is to hand a portion of their apples to other individuals in the economy. In other words, individuals can lend apples to other individuals, who in turn borrow them. In this exchange, the borrower will hand the lender the corresponding amount of IOUs; claims on apples of theirs that they receive at some point in the future. By lending out goods and compiling claims on goods produced in the future, you accumulate wealth. If an unlucky individual’s income situation has been dire for some time, they had to borrow and give out claims on their future goods, which pushed their wealth into the negative. An apple-loan, if you will. If they are really unlucky they reach a point at which they are not allowed anymore to borrow more. They hit a limit, which we will call the borrowing constraint and have to live from what they receive as income. A time limit at which you have to have paid back your debt does not exist. But there is an interest rate that borrowers have to pay to the wealthy people who lent them their apples.

import {stepFigure} from "./figures/steps.js"
viewof stepSize = Inputs.range([0.005, 1], {value: 0.1, step: 0.005, label: "step size h"})
stepFigure({d3, h: stepSize, width})

import {agentsFigure} from "./figures/agents.js"
agentsFigure({width, invalidation})

The generator of an Itô process: how do functions of the state evolve?

We are interested in the rate at which any (scalar) function changes, in expectation, with time. This rate is obtained by applying an operator , called the (infinitesimal) generator of the process. Instead of pasting in the textbook definition, I will try and provide the intuitive idea.

Let be some (scalar) function of the state. An example: could be defined as the total income (“labor” + capital income) that an agent gains. We will think a similar (and slightly more interesting) example through in more detail, later. Since the state evolves dynamically over time and the function reacts to changes in its arguments, the state dynamics—both deterministic and stochastic—should somehow transfer to the expected drift of . But how?

The deterministic part

The (deterministic) drift of the wealth-income process transfers to as a product of state velocity sensitivity of , for each state dimension. For example, in the wealth dimension, the state changes at rate . That’s the velocity part. And how varies with changes in wealth, the sensitivity part, is simply determined by the wealth-component of the gradient of with respect to wealth, i.e. .¹ The same holds for the income dimension. This is nothing but the chain rule of differentiation. Writing for simplicity, we know that sensitivity velocity, or: how varies with the state how the state varies with time. Plugging in the respective derivatives, evaluated at the current state , we get and taking the inner product (right-hand side of eq. (6)) gives Both of these terms will appear in the generator. But before we get to the actual definition, something is missing.

The stochastic part

So a part of the generator—how the drift of the state transfers to the drift of —just follow from basic calculus. Which leaves the stochastic part of the state: the income shocks from the Wiener process , scaled by the dispersion coefficient —see eqs. (3) and (5). As we saw in eq. (4), these shocks are centered at zero, i.e., they have zero mean—income is as likely to be pushed up by as down by . So…on average, the shocks cancel out. Does that mean that they do not affect the expected rate of change of ?

Well, not necessarily. If is curved in the dimension driven by noise, then the shocks do not cancel. Why that’s the case is best explained visually:

import {generatorFigure} from "./figures/generator.js"
viewof step = Inputs.form({
  theta: Inputs.range([0, 1.5], {value: 0.55, step: 0.01, label: "drift strength"}),
  sigma: Inputs.range([0, 1.2], {value: 0.7, step: 0.01, label: "dispersion σ"}),
})
generatorFigure({Plot, ...step, width})

The figure shows what happens to a bunch of agents, each starting from the same income. Each agent is mapped through a function and advanced one time step. Crucially, in this example, the function is curved upward—convex—in . Left: Without the noise, the drift moves each agent by the same amount, in the same direction (upward, as is increasing). Middle: Without the drift, adding noise from the Wiener process spreads the agents out across . On average, the value of increases, as the convex curvature in increases upward shocks more than it decreases downward shocks. Right: A full step of the process—drift + noise—increases the function value of and the state (income) itself.

So, in expectation, for convex functions , shocks will lead to larger values of . Analogously, if is curved downward—concave—in , then the income volatility will decrease on average. To account for this effect, the generator of our process contains a third term, , referred to as the Itô correction. The math tells us that …

• …wherever the function is (strictly) convex in , . Therefore, the term is strictly positive, which implies that income volatility increases the rate at which changes.
• …wherever the function is (strictly) concave in , . Therefore, the term is strictly negative, which implies that income volatility decreases the rate at which changes.

To summarize: even though the noise that drives the income process is symmetric around zero, not influencing to first order, the curvature introduces an asymmetric effect with respect to the spread of the income process.

The generator in full

Let us collect what I described above in a formal statement: For any (scalar) function of the state of the process from eq. (5), the rate at which changes, in expectation, over an infinitesimal time increment is given by an operator called the generator of the process. The generator of the wealth-income process is defined as follows:

Let’s make this more clear by looking at an example.

Conclusion & what’s next

This post introduced a simple macroeconomics model for how heterogenous agents drift and diffuse through a landscape of wealth and income. These dynamics are governed by an SDE, in which income follows a diffusion process and wealth accumulates according to a simple budget identity: what an agent saves at each time instant equals their total income (“labour + capital”) minus their consumption. Assuming a fixed and known interest rate and consumption policy allowed us to begin studying properties of the resulting process.

The generator gives us the expected rate of change of any (scalar) function of the stochastic state. It fixes two things that standard calculus does not account for: (1) that the function argument (the state) is nowhere differentiable, and (2) that the stochasticity in the state has a nontrivial effect on how a curved function changes over time in expectation. The meaning of the correction became more clear by thinking this through in the context of a practical example.

What’s left?

From the economics perspective, fixing the interest rate and the consumption is precisely removing the interesting bits from the model. So, it would be a shame to not have a look at how to determine these quantities in a sliiiightly more realistic setting. Luckily, determining both of these quantities connects beautifully to two more general insights about stochastic processes: the Feynman–Kac formula and the Fokker–Planck equation. The next part of this post will introduce both, which will also teach us how the agents choose their individual consumption, repeatedly, over time, and how the interest rate is determined by the aggregate state of the economy. So, if you’re interested, stay tuned :)