Incremental value of information for discrete-time partially observed stochastic systems

A discrete-time stochastic control problem for general (nonlinear in state, control, observation and noise) models is considered. The same noise can enter into the state and into the observation equations, and the state/observation does not need to be affine with respect to the noise. Under mild assumptions the joint distribution function of the state/observation processes is obtained and used for computing the Gateaux and Frechet derivatives of the cost function. Under partial observation the control actions are restricted by the measurability requirement and we compute the Lagrange multiplier associated with this "information constraint". The multiplier is called a "dual", or "shadow" price, and in the literature of the subject is interpreted as an incremental value of information . The present and the future are two factors appearing in the multiplier and we study how they are balanced as time goes on. An algorithm for computing extremal controls in the spirit of R. Rishel (1985) is also obtained.