Dynamic portfolio optimization with transaction costs and state-dependent drift

This thesis consists of three essays in financial economics, more precisely in the field of asset pricing and optimal portfolio choice. The first essay studies the asset pricing implications of heterogeneity in preferences coupled with long run risk in a dynamic general equilibrium pure exchange economy. It is concerned with the understanding of how risk sharing among different types of agents affects the assets returns moments in equilibrium and moreover, how these agents reacts dynamically to the long run risk that they perceive. The second and third essay approach the question of optimal portfolio with stochastic opportunity sets. The second essay proposes an approximation methodology for deriving approximate closed form solutions of dynamic optimal portfolio allocation problems. The third essay examines the optimal consumption, portfolio rule and life insurance purchase problem of a wage earner having uncertain lifetime when the market price of risk follows a mean-reverting stochastic process.

The paper studies connections between arbitrage and utility maximization in a discrete-time financial market. The market is incomplete. Thus one has several choices of equivalent martingale measures to price contingent claims. Davis determines a unique price for a contingent claim which is based on an optimal dynamic portfolio by use of a `marginal rate of substitution' argument. Here conditions will be given such that this price is determined by a martingale measure and thus by a consistent price system. The underlying utility function U is defined on the positive half-line. Then dynamic portfolios are admissible if the terminal wealth is positive. In case of the logarithmic utility function, the optimal dynamic portfolio is the numeraire portfolio.

The paper studies connections between risk aversion and martingale measures in a discrete‐time incomplete financial market. An investor is considered whose attitude toward risk is specified in terms of the index b of constant proportional risk aversion. Then dynamic portfolios are admissible if the terminal wealth is positive. It is assumed that the return (risk) processes are bounded. Sufficient (and nearly necessary) conditions are given for the existence of an optimal dynamic portfolio which chooses portfolios from the interior of the set of admissible portfolios. This property leads to an equivalent martingale measure defined through the optimal dynamic portfolio and the index 0 < b≤ 1. Moreover, the option pricing formula of Davis is given by this martingale measure. In the case of b= 1; that is, in the case of the log‐utility, the optimal dynamic portfolio defines the numéraire portfolio.

A model for optimal consumption and investment is posed whose solution is provided by the classical Merton analysis when there is zero transaction cost. A probabilistic argument is developed to identify the loss in value when a proportional transaction cost is introduced. There are two sources of this loss. The first is a loss due to “displacement” that arises because one cannot maintain the optimal portfolio of the zero-transaction-cost problem. The second loss is due to “transaction,” a loss in capital that occurs when one adjusts the portfolio. The first of these increases with increasing tolerance for departure from the optimal portfolio in the zero-transaction-cost problem, while the second decreases with increases in this tolerance. This paper balances the marginal costs of these two effects. The probabilistic analysis provided here complements earlier work on a related model that proceeded from a viscosity solution analysis of the associated Hamilton–Jacobi–Bellman equation.

Deep reinforcement learning (DRL) has been extensively used to address portfolio optimization problems. DRL agents acquire knowledge and make decisions through unsupervised interactions with their environment without requiring explicit knowledge of the joint dynamics of portfolio assets. Among these DRL algorithms, the combination of actor-critic algorithms and deep function approximators is the most widely used DRL algorithm. Here, we find that training the DRL agent using the actor-critic algorithm and deep function approximators may lead to scenarios where the improvement in the DRL agent's risk-adjusted profitability is insignificant. We argue that such situations primarily arise from the following two problems: sparsity in positive reward and the curse of dimensionality. These limitations prevent DRL agents from comprehensively learning asset price change patterns in the training environment. As a result, the DRL agents cannot effectively explore the dynamic portfolio optimization policy to improve the risk-adjusted profitability in the training process. To address these problems, we propose a novel multi-agent learning system based on the hierarchical deep reinforcement learning (HDRL) algorithmic framework in this research. Under this framework, the agents work together as a learning system for portfolio optimization. Specifically, by designing an auxiliary agent that works together with the executive agent for optimal policy exploration, the learning system can focus on exploring the policy with higher risk-adjusted return in the action space with positive return and low variance. The performance of the proposed learning system is evaluated using a portfolio of 29 stocks from the Dow Jones index in four different experiments. In the training process, the objective functions of the actor and critic both ultimately achieve stable convergence in the training process. The risk-adjusted profitability of our learning system in the training environment is significantly improved. Hence, we prove that the policies executed by our learning system in out-sample experiments originate from the DRL agents' comprehensive learning of asset price change patterns in the training environment. Furthermore, we find that adopting the auxiliary agent and HDRL training algorithm can efficiently overcome the issue of the curse of dimensionality and improve the training efficiency in the positive reward sparse environment. In each back-test experiment, the proposed learning system is compared to sixteen traditional strategies and ten strategies based on machine learning algorithms in the performance of profitability and risk control ability. The empirical results in the four evaluation experiments demonstrate the efficacy of our learning system, which outperforms all other strategies by at least 8.2% in terms of Sharpe ratio, Sorino ratio, and Calmar ratio. This indicates that the policies learned in the training environment can exhibit excellent generalization ability in the back-testing experiments.

For portfolio choice problems with proportional transaction costs, we discuss whether or not there exists a shadow price, i.e., a least favorable frictionless market extension leading to the same optimal strategy and utility. By means of an explicit counterexample, we show that shadow prices may fail to exist even in seemingly perfectly benign situations, i.e., for a log-investor trading in an arbitrage-free market with bounded prices and arbitrarily small transaction costs. We also clarify the connection between shadow prices and duality theory. Whereas dual minimizers need not lead to shadow prices in above “global” sense, we show that they always correspond to a “local” version.

Importance sampling in stochastic optimization: An application to intertemporal portfolio choice

In this paper we tackle the problem of dynamic portfolio optimization, i.e., determining the optimal trading trajectory for an investment portfolio of assets over a period of time, taking into account transaction costs and other possible constraints. This problem is central to quantitative finance. After a detailed introduction to the problem, we implement a number of quantum and quantum-inspired algorithms on different hardware platforms to solve its discrete formulation using real data from daily prices over 8 years of 52 assets, and do a detailed comparison of the obtained Sharpe ratios, profits, and computing times. In particular, we implement classical solvers (Gekko, exhaustive), D-wave hybrid quantum annealing, two different approaches based on variational quantum eigensolvers on IBM-Q (one of them brand-new and tailored to the problem), and for the first time in this context also a quantum-inspired optimizer based on tensor networks. In order to fit the data into each specific hardware platform, we also consider doing a preprocessing based on clustering of assets. From our comparison, we conclude that D-wave hybrid and tensor networks are able to handle the largest systems, where we do calculations up to 1272 fully-connected qubits for demonstrative purposes. Finally, we also discuss how to mathematically implement other possible real-life constraints, as well as several ideas to further improve the performance of the studied methods.

This paper proposes a solution method for the discrete-time long-term dynamic portfolio optimization problem with state and asset allocation constraints. We use the ideas of Model Predictive Control (MPC) to solve the constrained stochastic control problem. MPC is a solution technique which was developed to solve constrained optimal control problems for deterministic control applications. MPC solves the optimal control problem with a receding horizon where a series of consecutive open-loop optimal control problems is solved. The aim of this paper is to develop an MPC approach to the problem of long-term portfolio optimization when the expected returns of the risky assets are modeled using a factor model based on stochastic Gaussian processes. We prove that MPC is a suboptimal control strategy for stochastic systems which uses the new information advantageously and thus is better than the pure optimal open-loop control. For the open-loop optimal control optimization, we derive the conditional portfolio distribution and the corresponding conditional portfolio mean and variance. The mean and the variance depend on future decision about the asset allocation. For the dynamic portfolio optimization problem, we consider constraints on the asset allocation as well as probabilistic constraints on the attainable values of the portfolio wealth. We discuss two different objectives, a classical mean–variance objective and the objective to maximize the probability of exceeding a predetermined value of the portfolio. The dynamic portfolio optimization problem is stated, and the solution via MPC is explained in detail. The results are then illustrated in a case study.

Realized Covariance Estimation in Dynamic Portfolio Optimization

Portfolio Choice with State-Dependent Adjustments: Analyzing Leveraged Positions Without Parametric Assumptions

European Options under Proportional Transaction Costs: An Algorithmic Approach to Pricing and Hedging

Recently, there has been a growing interest in optimization problems associated with the arbitrage pricing of derivative securities in imperfect markets (in particular, in models with transaction costs). In this paper, we examine the valuation and hedging of European claims in the multiplicative binomial model proposed by Cox, Ross and Rubinstein [5] (the CRR model), in the presence of proportional transaction costs. We focus on the optimality of replication; in particular, we provide sufficient conditions for the optimality of the replicating strategy in the case of long and short positions in European options. This work can be seen as a continuation of studies by Bensaid et al. [2] and Edirisinghe et al. [13]. We put, however, more emphasis on the martingale approach to the claims valuation in the presence of transaction costs, focusing on call and put options. The problem of optimality of replication in the CRRmodel under proportional transaction costs was recently solved in all generality by Stettner [30].

This paper examines how proportional transaction costs, short-sale constraints, and margin requirements affect inferences based on asset return data about intertemporal marginal rates of substitution (IMRSs). It is shown that small transaction costs can greatly reduce the required variability of IMRSs. This suggests that the low variability of many parametric, aggregate consumption based IMRSs need not be inconsistent with asset return data. Euler inequalities for a transaction cost economy with power utility are tested using aggregate consumption data and returns on stocks and short maturity U.S. Treasury bills. In the majority of cases there is little evidence against power utility specifications with low risk-aversion parameters. The results are obtained with transaction costs on stocks as small as .5% of price, and are in sharp contrast to the strong rejection of the analogous Euler equalities for a frictionless economy.

Algorithmic trading for online portfolio selection under limited market liquidity