Model-aware reinforcement learning for high-performance Bayesian experimental design in quantum metrology

The recent advances in Machine Learning hold great promises for the field of quantum sensing and metrology. With the help of reinforcement learning, we can tame the complexity of quantum systems and solve the problem of optimal experimental design. Reinforcement learning is a powerful model-free technique that allows an agent, which is typically a neural network, to learn the best strategy to reach a certain goal in a completely a priori unknown environment. However, in general, we know something about the quantum system the agent is interacting with, at least that it follows the rules of quantum mechanics. In quantum metrology, we typically have a model for the system and only some parameters of the evolution or of the initial state are unknown. We present here a general Machine Learning technique that can optimize the precision of quantum sensors, and in doing so it exploits the knowledge we have on the system. We have developed a Python package to automate a broad class of optimizations that can be found in the tasks of quantum parameter estimation, quantum metrology and quantum hypothesis testing. What the agent is learning here is an optimal adaptive strategy, that, on the basis of the previous outcomes, decides the next measurements to perform. It works both for Bayesian estimation and for frequentist estimation. The user is required to implement the physics of the system to be studied and state which parameters in the experiment are controllable and which are unknowns. The functions of the library allow then to easily train a neural network agent for optimizing the precision of the sensor, by simulating the experiment. We have explored some applications of this technique to magnetometry on NV centers (both DC and AC), to state discrimination in quantum optics and to phase estimation. So far, we were able to certify better results than the current state-of-the-art controls for many examples. The Machine Learning technique developed here can be applied in all those scenarios where the quantum system is well characterized and relatively simple and small. In these cases, we can squeeze every last bit of information from a quantum sensor by controlling it with a neural network appropriately trained.

Data assimilation is the process of estimating the state of dynamic systems (linear or nonlinear, Gaussian or non-Gaussian) as accurately as possible from noisy observational data. Although the Three Dimensional Variational (3D-VAR) methods, Four Dimensional Variational (4D-VAR) methods and Ensemble Kalman filter (EnKF) methods are widely used and effective for linear and Gaussian dynamics, new methods of data assimilation are required for the general situation, that is, nonlinear non-Gaussian dynamics. General Bayesian recursive estimation theory is reviewed in this thesis. The Bayesian estimation approach provides a rather general and powerful framework for handling nonlinear, non-Gaussian, as well as linear, Gaussian estimation problems. Despite a general solution to the nonlinear estimation problem, there is no closed-form solution in the general case. Therefore, approximate techniques have to be employed. In this thesis, the sequential Monte Carlo (SMC) methods, commonly referred to as the particle filter, is presented to tackle non-linear, non-Gaussian estimation problems. In this thesis, we use the SMC methods only for the nonlinear state estimation problem, however, it can also be used for the nonlinear parameter estimation problem. In order to demonstrate the new methods in the general nonlinear non-Gaussian case, we compare Sequential Monte Carlo (SMC) methods with the Ensemble Kalman Filter (EnKF) by performing data assimilation in nonlinear and non-Gaussian dynamic systems. The models used in this study are referred to as state-space models. The Lorenz 1963 and 1966 models serve as test beds for examining the properties of these assimilation methods when used in highly nonlinear dynamics. The application of Sequential Monte Carlo methods to different fixed parameters in dynamic models is considered. Four different scenarios in the Lorenz 1063 [sic] model and three different scenarios in the Lorenz 1996 model are designed in this study for both the SMC methods and EnKF method with different filter siz

Spin squeezing has received much attention due to the interesting physics and important applications such as quantum metrology and quantum information processing. We here present a scheme to engineer stable spin squeezing in an array of nitrogen vacancy centers (NVCs) coupled to a rectangular hollow metallic waveguide. The remarkable feature of the waveguide as the common environment media is that one can switch on/off either the waveguide induced dipole-dipole interactions or correlated spontaneous emissions among the NVCs by designing their spatial separation. It permits us to achieve a dissipative Dicke model after the dipole-dipole interactions vanish due to destructive interference. With the external driving lasers on each NVC, a second-order phase transition is triggered, separating the steady state into two phases with and without collective spin squeezing. Supplying a physical realization of the dissipative Dicke model, our study gives a bridge between the generation of the stable spin squeezing and the phase transition physics.

Surface wave inversion is a crucial technique in geophysics for subsurface imaging. However, traditional methods can be computationally intensive, especially for complex models. This study introduces automatic differentiation (AD) as an efficient alternative to finite difference (FD) methods for gradient calculation in surface wave inversion. We compare AD and FD methods using three synthetic examples of varying complexity. Our results demonstrate that AD is significantly faster, with speed improvements of 3 to 12 times over FD, depending on model complexity. Moreover, AD requires up to 3 times less memory than FD. In terms of accuracy, AD provides gradient calculations that are exact up to machine precision, while FD is subject to truncation errors. This improved accuracy translates to more reliable inversion results, particularly for complex models. The efficiency and accuracy gains of AD are especially beneficial for gradient-based inversion methods in geophysics, where computational resources often limit the scale and complexity of problems that can be addressed. Our findings suggest that integrating AD into gradient-based inversion methods could significantly enhance subsurface imaging techniques across various geophysical applications.

Relative phase distribution and the precision of optical phase sensing in quantum metrology

We propose an efficient scheme for transferring quantum states and generating\nentangled states between two qubits of different nature. The hybrid system\nconsists a single nitrogen vacancy (NV) center and a superconducting (SC)\nqubit, which couple to an optical cavity and a microwave resonator,\nrespectively. Meanwhile, the optical cavity and the microwave resonator are\ncoupled via the electro-optic effect. By adjusting the relative parameters, we\ncan achieve high fidelity quantum state transfer as well as highly entangled\nstates between the NV center and the SC qubit. This protocol is within the\nreach of currently available techniques, and may provide interesting\napplications in quantum communication and computation with single NV centers\nand SC qubits.\n

Computerized adaptive testing (CAT) aims to present items that statistically optimize the assessment process by considering the examinee's responses and estimated trait levels. Recent developments in reinforcement learning and deep neural networks provide CAT with the potential to select items that utilize more information across all the items on the remaining tests, rather than just focusing on the next several items to be selected. In this study, we reformulate CAT under the reinforcement learning framework and propose a new item selection strategy based on the deep Q-network (DQN) method. Through simulated and empirical studies, we demonstrate how to monitor the training process to obtain the optimal Q-networks, and we compare the accuracy of the DQN-based item selection strategy with that of five traditional strategies-maximum Fisher information, Fisher information weighted by likelihood, Kullback‒Leibler information weighted by likelihood, maximum posterior weighted information, and maximum expected information-on both simulated and real item banks and responses. We further investigate how sample size and the distribution of the trait levels of the examinees used in training affect DQN performance. The results show that DQN achieves lower RMSE and MAE values than traditional strategies under simulated and real banks and responses in most conditions. Suggestions for the use of DQN-based strategies are provided, as well as their code.

In this paper, our prime objective is to apply the techniques of parameter estimation theory and the concept of Quantum Metrology in the form of Fisher Information to investigate the role of certain physical quantities in the open quantum dynamics of a two entangled qubit system under the Markovian approximation. There exist various physical parameters which characterize such system, but can not be treated as any quantum mechanical observable. It becomes imperative to do a detailed parameter estimation analysis to determine the physically consistent parameter space of such quantities. We apply both Classical Fisher Information (CFI) and Quantum Fisher Information (QFI) to correctly estimate these parameters, which play significant role to describe the out-of-equilibrium and the long range quantum entanglement phenomena of open quantum system. Quantum Metrology, compared to classical parameter estimation theory, plays a two-fold superior role, improving the precision and accuracy of parameter estimation. Additionally, in this paper we present a new avenue in terms of Quantum Metrology, which beats the classical parameter estimation. We also present an interesting result of revival of out-of-equilibrium feature at the late time scales, arising due to the long range quantum entanglement at early time scale and provide a physical interpretation for the same in terms of Bell's Inequality Violation in early time scale giving rise to non-locality.

We define two ways of quantifying the quantum correlations based on quantum Fisher information (QFI) in order to study the quantum correlations as a resource in quantum metrology. By investigating the hierarchy of measurement-induced Fisher information introduced in Lu et al. [X. M. Lu, S. Luo, and C. H. Oh, Phys Rev. A 86, 022342 (2012)], we show that the presence of quantum correlation can be confirmed by the difference of the Fisher information induced by the measurements of two hierarchies. In particular, the quantitative quantum correlations based on QFI coincide with the geometric discord for pure quantum states.

Geosteering, a key component of drilling operations, traditionally involves manual interpretation of various data sources such as well-log data. This introduces subjective biases and inconsistent procedures. Academic attempts to solve geosteering decision optimization with greedy optimization and approximate dynamic programming (ADP) showed promise but lacked adaptivity to realistic diverse scenarios. Reinforcement learning (RL) offers a solution to these challenges, facilitating optimal decision-making through reward-based iterative learning. State estimation methods, e.g., particle filter (PF), provide a complementary strategy for geosteering decision-making based on online information. We introduce RL-Estimation, a method that integrates an RL-based geosteering framework with PF to address realistic geosteering scenarios. RL-Estimation deploys PF to process real-time well-log data to estimate the location of the well relative to the stratigraphic layers, which then informs the RL-based decision-making process. Our findings indicate that RL-Estimation achieves at least 20% higher reservoir contact compared to using RL or PF alone. Additionally, RL-Estimation performs within 2% of the theoretically optimal benchmark, which assumes access to the true state instead of relying on estimates from PF. These results demonstrate the synergy between RL and PF, highlighting the method’s effectiveness and near-optimal performance in realistic geosteering scenarios.

Sensor control for multi-object state-space estimation using random finite sets

With their ability to process and transfer quantum information, large-scale entanglement-based quantum networks could be at the heart of a new age of quantum information, enabling fundamentally new applications such as distributed quantum computation, quantum communication, and quantum enhanced sensing. Due to their long spin coherence, controllable local qubit registers and optically active spins, color centers in diamond are prime candidates for nodes of such a network, and have enabled some of the most advanced quantum network demonstrations to date. These demonstrations include the distribution of a 3-node GHZ state across a quantum network, entanglement swapping, entanglement distillation, and memory-enhanced quantum communication. To move beyond current proof-of-principle networks, a further increase of entanglement generation rates is crucial. This thesis presents theoretical and experimental work on enhancing the spin-photon interface of color centers in diamond to achieve this goal, making use of the Purcell effect. The discussed work follows two main directions: embedding of color centers in open, tuneable Fabry-Perot micro-cavities, and in all-diamond photonic crystal cavities. First, we describe theoretical and experimental progress towards cavity-enhanced quantum networks based on nitrogen-vacancy (NV) centers in diamond. Due to their first order sensitivity to electric fields, we choose to embed NV centers in microns-thin diamond membranes that can be integrated into open fiber-based micro-cavities. We develop analytical methods to optimize the design of such open cavity systems for maximum Purcell enhancement of embedded color centers. We demonstrate a method to fabricate optically coherent NV centers in microns-thin diamond membranes, and use such structures to demonstrate the resonant excitation and detection of coherent, Purcell enhanced NV emission. A theoretical model in excellent agreement with our results suggests our system can improve entanglement rates between distant NV centers by two orders of magnitude with near-term improvements to the setup. Second, we describe progress towards an efficient spin-photon interface of group-IV color centers in diamond by coupling them to photonic nanostructures, allowing for large-scale integration. Due to their first-order insensitivity to electric fields, group-IV color centers can be brought in close proximity to surfaces (~ 100 nm), allowing for sub-wavelength mode volumes and thus very high Purcell factors. We numerically optimize photonic crystal cavity designs to maximize the Purcell enhancement of embedded emitters, test the robustness of our designs to real-world fabrication imperfections, and devise a method to efficiently interface nanophotonic structures. We then proceed to fabricate all-diamond photonic crystal cavities, making use of a dry etching technique that is selective to different crystallographic directions, and characterize the resulting optical quality factors. Finally, we marry the developed fabrication methods to fabricate microns-sized diamond platelets that can be transferred from a holding frame in a controlled fashion. This capability could be crucial for the realization of hybrid photonic circuits that we expect to be at the heart of future large-scale quantum networks.

An oscillator with position-dependent mass exposed to a thermal bosonic bath

The quantum Fisher information, the quantum analog of the classical Fisher information, is a central quantity in quantum metrology and quantum sensing due to its connection to parameter estimation and fidelity susceptibility. Using numerically exact methods applied to a paradigmatic open quantum system, the spin-boson model, we calculate both static and dynamical quantum Fisher information matrix elements with respect to spin-bath couplings and magnetic field strengths. As the spin-bath interaction increases, we first show that the coupling-coupling matrix elements relative to the ground state of the Hamiltonian are linked to the entanglement growth and signal the Berezinskii-Kosterlitz-Thouless quantum phase transition through their nonmonotonic behavior. We also point out that the static quantum Fisher information exhibits a nonperturbative behavior in the zero-coupling limit, which we justify with an analytic argument. Furthermore, we demonstrate that the time-dependent matrix elements can reveal non-Markovian effects as well as the transition from the coherent to incoherent regime at the Toulouse point, remaining robust under pure dephasing noise. Nonmonotonic signatures of the quantum Fisher information matrix reflect changes in quantum resources such as entanglement and coherence, quantify non-Markovian behavior, and enable criticality-enhanced quantum sensing, thereby shedding light on key features of open quantum systems.

Phase estimation has a wide range of applications. Over the years, several strategies have been studied to improve precision in phase estimation. These strategies include using exotic quantum states to quantum detection schemes. This dissertation summarizes my effort in improving the precision of phase estimation with a linear and nonlinear interferometer. Chapter 1 introduces quantum optics and quantum metrology. I introduce all relevant quantum states of light used. We also look into tools and terminologies of quantum metrology such as Fisher information, shot-noise limit, Heisenberg limit, etc., along with examples of phase estimation with a Mach-Zehnder interferometer. In Chapter 2, I discuss multiple phase estimation using a multimode interferometer. Building upon previous work, our scheme consists of a multimode interferometer with single-photon inputs. By using a quantum Fisher information analysis, we show that our scheme gives a constant improvement over other schemes. We also show that our scheme with photon-number-resolving detection approaches the quantum Cram\'er-Rao bound. Moreover, we also consider the probabilistic nature of photon emission at the input, and we study its effect on phase sensitivity. I discuss phase estimation with SU(1,1) interferometer in Chapter 3. We look at phase sensitivity in this interferometer with different input states. Namely, we consider two different phase estimation scheme, one using thermal and squeezed states, and others using coherent and displaced squeezed states with parity and on-off as a detection scheme. We also look into the effect of photon loss inside the interferometer. In Chapter 4, we revisit phase estimation in SU(1,1) interferometer from the perspective of quantum Fisher information. I discuss in detail a longstanding confusion regarding the use of quantum Fisher information in SU(1,1) interferometer. We show that phase averaging or quantum Fisher information matrix method is needed in general for calculating the phase sensitivity which resolves inconsistencies reported in previously published articles.