Welcome to the Stochastic Control Theory Reading Seminar, organized by Georgy Gaitsgori and Richard Groenewald.
This Spring we will continue studying Stochastic Control Theory by reading Professor Daniel Ocone's lecture notes on Stochastic Control. Our talks will be held inperson in Columbia University (Math 528) on Mondays from 5p.m. to 6p.m. EDT.
This seminar is the logical continuation of the seminar held in Spring 2022.
If you would like to come or to be added on the mailing list, please email gg2793@columbia.edu.
Date and time  Speaker  Title and abstract 

Monday, January 30, 5:00p.m. EDT  Georgy Gaitsgori  Organizational meeting and reviewing stochastic calculus
We will discuss some organizational details and our plans. Afterward, we will cover the first pages of lecture notes. We will recall definitions of random fields, stochastic integral, and SDEs. We will also mention the main definitions and theorems about SDEs which will be used later. 
Monday, February 6, 5:00p.m. EDT  Georgy Gaitsgori  Reviewing stochastic differential equations
We will try to cover the first 16 pages of Ocone's lecture notes. We will recall definitions of Brownian motion, random field, stochastic integrals, and stochastic differential equations (SDEs). Then we will state the main results about SDE, such as the existence and uniqueness of solutions, comparison principle, Markovianity, etc. We will prove some of the theorems, and if time permits, we will discuss SDEs with random coefficients. 
Monday, February 13, 5:00p.m. EDT  Richard Groenewald  Introduction to Controlled SDEs
We will cover theorem 4 and all of sections 5, 6.1 and 6.2 of Ocone's notes. We will look at existence/uniqueness results for SDEs with a control parameter (a more general setting than in the previous lecture) and explore connections with PDEs of a particular form via a FeynmanKacesque approach. 
Monday, February 20, 5:00p.m. EDT  Georgy Gaitsgori  Abstract Dynamic Programming Equation, Markov semigroups and their generators
We will cover sections 6.26.4 of Ocone's notes. We will introduce the notions of an abstract twoparameter family of operators and its generator. We will consider a special case of Markov processes and their generators and prove an abstract dynamic programming equation. After that, we will find a generator of a diffusion process (SDE) and show how it is connected to a value function of a (control) problem we discussed last time. 
Monday, February 27, 5:00p.m. EDT  Richard Groenewald  More on Elliptic PDEs, and the notion of a "derivative" of a stochastic process
We will cover sections 6.5 and 6.6 of Ocone's notes. In particular, we will discuss an existence and uniqueness result for classical solutions to elliptic PDEs of the form discussed in the last few weeks. We will introduce the concept of a derivative of a stochastic process and use this to further analyze said PDEs. 
Monday, March 6, 5:00p.m. EDT  Georgy Gaitsgori  Introducing notions of Stochastic Control Problems
We will recall the formal notion of a controlled SDE and the results about the existence and uniqueness of its solutions. We will then start discussing finite horizon stochastic control problems. In particular, we will introduce notions of admissible controls and value functions and prove some properties (boundness and growth rate) of the latter. 
Monday, March 13, 5:00p.m. EDT  No seminar (Spring break) 

Monday, March 20, 5:00p.m. EDT  Richard Groenewald  Heuristics on the Dynamic Programming Principle and HJB Equation
We will discuss the dynamic programming principle for our control problem along with the associated HamiltonJacobiBellman equation. We will formulate the definition of viscosity solutions to general PDEs. This session will follow pages 17 to 31 of part 2 of Ocone's notes. 
Monday, March 27, 5:00p.m. EDT  Georgy Gaitsgori  Recalling HJB equation and solving two particular stochastic control problems
We will first recall HJB equation from the last week and then see how to use it to solve stochastic control problems. We will then consider two problems from Benes, Shepp, and Witsenhausen's celebrated paper: the Bounded velocity follower problem and the Monotone follower problem with a finite horizon. We will find the corresponding HJB equations, solve them, and try to deduce the actual value functions of the problems and optimal controls from the obtained solutions. 
Monday, April 3, 5:00p.m. EDT  Georgy Gaitsgori  Solving Monotone follower problem and discussing DPP rigorously
We will continue our discussion from the last seminar. We will consider another stochastic control problem, namely Monotone follower, and use the HJB equation to find its solution. Then we will come back to dynamic programming principle and discuss how to prove it rigorously. 
Monday, April 10, 5:00p.m. EDT  Richard Groenewald  More on Viscosity Solutions and the Dynamic Programming Principle
We will review the concept of viscosity solutions and prove several useful properties. If time permits, we will discuss the ideas behind a rigorous proof of the dynamic programming principle in the setting of section 2 of Ocone's notes. 
Monday, April 17, 5:00p.m. EDT  Ioannis Karatzas  Sequential estimation with control and discretionary stopping
We show that "fullbang" control is optimal in a problem that combines features of (i) sequential leastsquares estimation with Bayesian updating, for a random quantity observed in white noise; (ii) bounded control of the rate at which observations are received, with a superquadratic cost per unit time; and (iii) "fast" discretionary stopping. We describe also the optimal filtering and stopping rules in this context. [Joint work with Erik Ekstrom and Juozas Vaicenavicius, Uppsala.] 
Monday, April 24, 5:00p.m. EDT  No seminar 

Monday, May 1, 5:00p.m. EDT  Georgy Gaitsgori  On the Kulldorff's Goal problem
We will discuss an explicitly solvable stochastic control problem of Martin Kulldorff. This is a problem of a goal type. Namely, one considers a controlled diffusion and tries to maximize the probability of getting to an endpoint of an interval in finite time. If time permits, we will mention some other goal problems and related results. 