A Minimal Continuous-Time Markov Pharmacometric Model

The measured power spectrum density PWSEX of the fluctuating membrane current of neurons often differs from the well known Lorentz curve. Representing the frequency dependence in the high frequency (f) above the corner frequency as f−n, n is less than 2, which indicates that the slope is smaller than in the Lorentz curve f−2. The reason for the rise of the high-frequency side of PWSEX is the aliasing effect produced by application of the FFT to discrete data over a finite interval. In this study, the theoretical expression PWSTH for the power spectrum density obtained from a discrete-line Markov model is used in the analysis of the experimental data. In the practical analysis of experimental data, we assume that the number of states of a single kind of channel on the membrane of the neuron is two or three. Using the discrete-time Markov model, an expression for PWSTH is derived. When the sampling time approaches zero, this expression approximates the known expression for the power spectrum density under the continuous-time Markov model. The plot of PWSTH gave a good fit to the experimental data. The open time that we obtained agreed well with existing values. These results confirm the usefulness of this form of PWSTH when it is applied to processing discrete data sampled at finite intervals.

SummarySleep is a complex biological process regulated by networks of neurons and environmental factors. As one falls asleep, neurotransmitters from sleep–wake regulating neurones work in synergy to control the switching of different sleep states throughout the night. As sleep disorders or underlying neuropathology can manifest as irregular switching, analysing these patterns is crucial in sleep medicine and neuroscience. While hypnograms represent the switching of sleep states well, current analyses of hypnograms often rely on oversimplified temporal descriptive statistics (TDS, e.g., total time spent in a sleep state), which miss the opportunity to study the sleep state switching by overlooking the complex structures of hypnograms. In this paper, we propose analysing sleep hypnograms using a seven‐state continuous‐time Markov model (CTMM). This proposed model leverages the CTMM to depict the time‐varying sleep‐state transitions, and probes three types of insomnia by distinguishing three types of wake states. Fitting the proposed model to data from 2056 ageing adults in the Multi‐Ethnic Study of Atherosclerosis (MESA) Sleep study, we profiled sleep architectures in this population and identified the various associations between the sleep state transitions and demographic factors and subjective sleep questions. Ageing, sex, and race all show distinctive patterns of sleep state transitions. Furthermore, we also found that the perception of insomnia and restless sleep are significantly associated with critical transitions in the sleep architecture. By incorporating three wake states in a continuous‐time Markov model, our proposed method reveals interesting insights into the relationships between objective hypnogram data and subjective sleep quality assessments.

Computation of bounds for transient measures of large rewarded Markov models using regenerative randomization

Event Abstract Back to Event Hidden Markov models for the complex stimulus-response relationships of multi-state neurons Sean Escola1* and Liam Paninski1 1 Columbia University, United States Recent experimental results suggest that neurons are associated with multiple firing regimes, or states. We develop a general framework for estimating state-dependent neural receptive fields from paired spike-train and stimulus data assuming that neurons transition between several discrete hidden states. We modify both the discrete-time and continuous-time traditional hidden Markov model (HMM) frameworks to permit point-process observables such as spike-trains. For maximal flexibility in our model, we allow the external, time-varying stimulus and the neuron's own spike-history to drive both the spiking behavior in each state and the transitioning behavior between states. We employ an appropriately modified expectation-maximization algorithm to learn the model parameters. The expectation step is solved by the standard forward-backward algorithm for HMMs. The maximization step, though it is not analytically tractable, reduces to a simple concave optimization problem if the model is restricted slightly. We test our algorithm on two stimulated data sets and are able both to fully recover the parameters used to generate the data and to accurately recapitulate the sequence of hidden states. As a demonstration of the flexibility of our model, we apply our framework to a recently-published data set of purportedly multi-state neurons and show that we can implement a hybrid half multi-state/half histogram model which captures more of the neuronal variability than either a simple HMM or a simple histogram-based model alone Conference: Bernstein Symposium 2008, Munich, Germany, 8 Oct - 10 Oct, 2008. Presentation Type: Poster Presentation Topic: All Abstracts Citation: Escola S and Paninski L (2008). Hidden Markov models for the complex stimulus-response relationships of multi-state neurons. Front. Comput. Neurosci. Conference Abstract: Bernstein Symposium 2008. doi: 10.3389/conf.neuro.10.2008.01.043 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 13 Nov 2008; Published Online: 13 Nov 2008. * Correspondence: Sean Escola, Columbia University, New York, United States, gse3@columbia.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Sean Escola Liam Paninski Google Sean Escola Liam Paninski Google Scholar Sean Escola Liam Paninski PubMed Sean Escola Liam Paninski Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

MARKOV1 is an interactive software tool used to formulate and solve discrete space, continuous time Markov and semi-Markov models. The software is menu-driven and allows easy model formulation in a spreadsheet-like environment. MARKOV1 comes with example applications for reliability, availability, maintenance, project planning and inventory control. MARKOV1 displays many statistics associated with Markov models, including steady-state probabilities, entrance rates, absorption and first passage probabilities and times, annual discounted costs, cost of first transitions, and transient behavior. Markov reward process with linear and exponential cost curves, continuous time discounting, multiple alternatives, costs grouped into categories. Computation times depend on the number of states, transitions, and costs in the model.

The progression of a disease may be affected by many risk factors, such as gender, age, and current disease state. Such information is collected and made publically available by published clinical studies, yet combining this information into a disease model remains a challenge. This paper extends the previously published maximum likelihood estimation technique to estimate model parameters from indirect secondary data. Such information is available in the scientific literature so the modeler can access more data when estimating model parameters. The extension to the estimation procedure allows model transitions that depend on different sets of covariates for which secondary data are available. This extension uses a Markov model with transition probabilities stored in multi-dimensional tables accessed by covariate values. The paper uses a set of cases, including a case of cardiovascular disease in diabetes. The cases demonstrate the proposed method with various model variations. To help cope with model multiplicity, a selection method is demonstrated for picking a preferred model according to likelihood and structure criteria.

We used continuous-time Markov models based on CD4 cell counts and anti-CD3 reactivity (i.e., measure for T-cell quality) to study the progression of HIV infection in a cohort study of homosexual men in Amsterdam. We also compared the effectiveness of anti-CD3 reactivity as a marker for disease progression with that of CD4 cell counts. We used data from 467 men (6905 visits) with visits at 3-month intervals between October 1984 and March 1993. To account for measurement error and short time-scale variability, the immunological stage at each visit was determined using a kernel smoother on log-transformed data from each individual. The Markov model had six marker-defined stages and a seventh stage for clinical AIDS. The initial stage-occupation probabilities for seroconverters were used to estimate the incubation time from infection to AIDS. Confidence intervals were calculated using the bootstrap method to account for the effect of smoothing on the variability of our estimates. The CD4 staging scheme estimated the median time from seroconversion to AIDS at 8.3 years [95% confidence interval (CI), 8.1-8.6], and a similar estimate was obtained with the anti-CD3 staging model. The CD4 model predicts that 10.2% (95% CI, 9.9-13.1) will remain AIDS-free 15 years after seroconversion. The mean number of stages visited before AIDS is lower with the CD4 model (7.4; 95% CI, 7.2-7.7) than with the anti-CD3 model (11.3; 95% CI, 10.8-12.0), implying that anti-CD3 predicts progression less well than CD4 cell count. CD4 lymphocyte counts and anti-CD3 reactivity are each associated with an increased hazard for progression to AIDS. Therefore, men in different CD4-stages (anti-CD3 stages) follow different incubation period distributions to AIDS. However, anti-CD3 predicts progression less well than CD4 cell count. Staged time-continuous Markov models are useful to study immunological markers for HIV disease progression.

The throughput at the medium-access control (MAC) layer in a wireless network, that uses the carrier-sense multiple-access (CSMA) protocol, is degraded by collisions caused by failures of the carrier-sensing mechanism. Two sources of failure in the carrier-sensing mechanism are delays in the carrier sensing mechanism and hidden terminals, in which an ongoing transmission cannot be detected at a terminal that wishes to transmit because the path loss from the active transmitter is large. In this chapter, the effect of these carrier-sensing failures is modeled using a continuous-time Markov model. The throughput of the network is determined using the stationary distribution of the Markov model. The throughput is maximized by finding optimal mean transmission rates for the terminals in the network subject to constraints on successfully transmitting packets at a rate that is at least as great as the packet arrival rate.

Short-term load prediction is the foundation of dispatch and decision for urgent demand response of power grid. However, traditional load prediction approaches feature not so good adaption due to high random behaviors of individual household loads. In this paper, user home appliance data within extremely short time interval are acquired with help of non-intrusive load identification technique. By in-depth analysis of household home appliance operation characteristics, differentiate parameters and relevant edge values of various states are determined, resulting in state space set of specific home appliance. Time-domain probability distribution function and density function of specific home appliance are derived based on user behavior analysis model construction using continuous-time Markov model. State transition probability matrix and rate matrix are obtained, too. User short-term load prediction procedures are constructed by applying solution of Kolmogorov forward equations. Finally, 100 typical households electricity consumption data of Jiangsu Province in China from July to September are sampled for analysis. The accuracy and effectiveness of proposed method is verified by analysis and comparison results with traditional load prediction method.

Functions for fitting continuous-time Markov and hidden Markov multi-state models to longitudinal data. Designed for processes observed at arbitrary times in continuous time (panel data) but some other observation schemes are supported. Both Markov transition rates and the hidden Markov output process can be modelled in terms of covariates, which may be constant or piecewise-constant in time.

The aim was to analyze the relationship between peficitinib exposure and efficacy response according to American College of Rheumatology (ACR) 20 criteria and 28‐joint disease activity score based on C‐reactive protein (DAS28‐CRP) in rheumatoid arthritis (RA) patients, and to identify relevant covariates by developing exposure–response models. The analysis incorporated results from three multicenter, placebo‐controlled, double‐blind studies. As an exposure parameter, individual post hoc pharmacokinetic (PK) parameters were obtained from a previously constructed population PK model. Longitudinal ACR20 response rate and individual longitudinal DAS28‐CRP measurements were modeled by a non‐linear mixed effect model. Influential covariates were explored, and their effects on efficacy were quantitatively assessed and compared. The exposure–response models of effect of peficitinib on duration‐dependent increase in ACR20 response rate and decrease in DAS28‐CRP were adequately described by a continuous time Markov model and an indirect response model, respectively, with a sigmoidal Emax saturable of drug exposure in RA patients. The significant covariates were DAS28‐CRP and total bilirubin at baseline for the ACR20 response model, and CRP at baseline and concomitant methotrexate treatment for the DAS28–CRP model. The covariate effects were highly consistent between the two models. Our exposure–response models of peficitinib in RA patients satisfactorily described duration‐dependent improvements in ACR20 response rates and DAS28‐CRP measurements, and provided consistent covariate effects. Only the ACR20 model incorporated a patient's subjective high expectations just after the start of the treatment. Therefore, due to their similarities and differences, both models may have relevant applications in the development of RA treatment.Clinical trial registrationNCT01649999 (RAJ1), NCT02308163 (RAJ3), NCT02305849 (RAJ4).

This paper sort to establish some risk factors of stroke and to estimate the effect of the covariates at different levels of disease states. A review of the literature on stroke risk factors did not reveal any article that estimate possible covariate effect of transition. To fill this gap, we incorporate the covariates in a Continuous Time Markov Model in multi-state models to observe the transition rates of the patients at two-monthly intervals for two years. Patient variables are age, sex, location of the patient, local treatment, smoking, alcohol intake, and hemiparesis. Secondary data from stroke patients under rehabilitation at the Tamale Teaching Hospital from 2014 to 2019 was used. It is observed that males recover earlier in all states compared to females. Old and Older patient groups have some probability of transiting to less severe states; and they have similar probabilities of transiting from mild to a more severe states. The youth are better off in the severe state than the two older groups. In severe state, a patient without local treatment lives less than two months before death, whiles patients who seek local treatment may remain with severe stroke for at least two months before transiting to a less severe state. Thus left hemiparesis patients are about twice less likely (0.06738) to transit to the severe state than right hemiparesis patients (0.1207).

In order to meet the demand for large-scale device access in Narrowband Internet of Things (NB-IoT) and to overcome the problem that some resources are wasted due to the use of single decode-and-forward (DF) or amplify-and-forward (AF) collaboration in traditional collaborative communication, this paper introduces a non-orthogonal multiple access (NOMA) and orthogonal multiple access (OMA) hybrid access method into the NB-IoT and proposes a new Hybrid NOMA transmission protocol with relay adaptively selectable collaboration, which is represented as an algorithm. Based on this protocol, we classify the whole transmission process into five states after deriving the outage probability of each link. We then consider the system as a discrete-time Markov model, the closed expressions of the system outage probability and throughput are derived based on the system steady-state probability. In order to improve the system’s reliability, we further optimize the above protocol by allowing the source node to retransmit the unsuccessful received superimposed signals a limited number of times. Numerical results and simulations show that the outage probability is lower when multiple retransmissions are possible. The proposed relay adaptive collaborative hybrid NOMA transmission protocol has advantages over the pure OMA transmission mode.

Achieving self-sustainability has been one of the key challenges in designing smart grid connected residential systems. Solar enabled and power grid connected, dual-powered residential systems is an attractive solution, but it is not carbon free and cost optimal for the end user. This paper proposes temporal energy cooperation among the dual-powered residences as a potential cost and energy efficient solution. Through this paper, we present a microgrid based, multi-residence cooperative energy transfer mechanism to offset the power grid dependency. The developed analytical framework characterizes the green energy storage as a discrete time Markov model and aims to exploit the temporal residential load variations, towards designing self-sustainable systems at a much lower capital expenditure (CAPEX). Our simulation results capture the variation of optimum residence cluster size as a function of energy sharing price and load skewness to become cost profitable. The results also demonstrate a significant reduction in CAPEX, achieved through the proposed energy cooperative framework, over a non-cooperative residential system.

BackgroundDespite many decades of declining mortality rates in the Western world, cardiovascular disease remains the leading cause of death worldwide. In this research we evaluate the optimal mix of lifestyle, pharmaceutical and population-wide interventions for primary prevention of cardiovascular disease.Methods and FindingsIn a discrete time Markov model we simulate the ischaemic heart disease and stroke outcomes and cost impacts of intervention over the lifetime of all Australian men and women, aged 35 to 84 years, who have never experienced a heart disease or stroke event. Best value for money is achieved by mandating moderate limits on salt in the manufacture of bread, margarine and cereal. A combination of diuretic, calcium channel blocker, ACE inhibitor and low-cost statin, for everyone with at least 5% five-year risk of cardiovascular disease, is also cost-effective, but lifestyle interventions aiming to change risky dietary and exercise behaviours are extremely poor value for money and have little population health benefit.ConclusionsThere is huge potential for improving efficiency in cardiovascular disease prevention in Australia. A tougher approach from Government to mandating limits on salt in processed foods and reducing excessive statin prices, and a shift away from lifestyle counselling to more efficient absolute risk-based prescription of preventive drugs, could cut health care costs while improving population health.