Bandits with switching costs

Immersive technologies, such as virtual and augmented reality, demand high framerate, low latency, and precise synchronization between real and virtual environments. To meet these requirements, an edge server typically needs to perform high-quality rendering, and must predict user head motion and transmit a portion of the rendered panoramic scene that is large enough to cover the user's viewport, yet small enough to satisfy bandwidth constraints. Each portion yields two feedback signals: prediction feedback, indicating whether the selected portion covers the actual viewport, and transmission feedback, indicating whether all data packets are successfully delivered. While prior work models this setting as a multi-armed bandit with two-level bandit feedback, it overlooks that prediction feedback can be retrospectively computed for all possible portions, thus providing full-information feedback. In this work, we introduce a new two-level feedback model that combines full-information feedback with bandit feedback, and we formulate the portion selection problem as an online learning task under this hybrid setting. We derive an instance-dependent regret lower bound for this new hybrid feedback setting, and we propose AdaPort, a hybrid learning algorithm that leverages both the full-information feedback and bandit feedback to improve learning efficiency. We then show that the instance-dependent regret upper bound for AdaPort matches the lower bound asymptotically, proving its asymptotic optimality. Simulations using synthetic data and real-world traces demonstrate that AdaPort consistently outperforms state-of-the-art baselines, validating the benefits of exploiting the hybrid feedback structure.

We consider the problem of online learning in a Markov decision process MDP with finite states but continuous actions. This generalizes both the traditional problem of learning an MDP with finite actions and states, as well as the so-called continuum-armed bandit problem which has continuous actions but with no state involved. Based on previous works for these two problems, we propose a new algorithm for our problem, which dynamically discretizes the action spaces and learns to play strategies over these discretized actions that evolve over time. Our algorithm is able to achieve a T-step regret of about the order of $$T^{\frac{d+1}{d+2}}$$ with high probability, where d is a newly defined near-optimality dimension we introduce to capture the hardness of learning the MDP.

Indonesia's Government has made policy in the education sector due to the COVID-19 pandemic situation. Previously teaching and learning process especially in the higher institution is held face to face, and it switches to fully online learning. This situation forces the lecturer and students to adopt online learning and effected to their learning experiences. This current study is aimed at analyzing the students’ difficulties during online learning in speaking for professional context subject during COVID-19 situation. This research was held using a descriptive qualitative method approach. The data was obtained from an online questionnaire and interview conducted to 45 students of the second semester of the English Department who joining speaking for professional context class at Universitas Pendidikan Mandalika in Mataram, West Nusa Tenggara, Indonesia. The findings of the research revealed various problems for students as indicated in the current research. A total of 5 problems arose during online learning in the Speaking for Professional class. The arisen problem was: The first problem was related to Youtube video as the source of learning materials. The second problem was dealing with the students’ difficulties in elaborating the materials given by the lecturer during an online class. The third problem was related to the students’ procedure in doing the project or assignment given by the lecturer. The fourth problem was dealing with Students’ limitations in accessing ZOOM Meetings. The Students’ learning equipment and supporting facilities also contributed as the fifth arisen problems of the students while having online learning during COVID-19 pandemic situations. The findings provide information for further development and improvement in online teaching and learning procedures. Further research on the strategy of how the students face the challenge and adapt are encouraged.

Since the appearance of 5G technology, it has provided convenience for all aspects of society and life. 5G technology has also made great contribution to education, and the help effect for education is very significant, especially for online learning. In March 2020, the epidemic situation of covid-19 in China is severe, and the concept of "classes suspended but learning continues" has become the trend of school teaching. In particular, colleges and universities, involving a wide range of population migration, basically adopt online learning methods to carry out teaching activities. Therefore, the traditional teaching mode with teachers in the classrooms no longer meets the current teaching needs, and online learning has become the mainstream. Compared with the traditional face-to-face teaching, there are many problems in online learning due to network speed and latency.Based on the above considerations, this paper studies the online learning system based on the 5G mobile communication technology network, as well as the traditional face-to-face teaching and online learning under other different networks, and compares the students' mastery of knowledge under different teaching methods through examination. In the current situation of complex network, this paper proposes the number of knowledge points and the number of effective interaction between teachers and students, and puts forward the prospect of online learning in the future.

We study the problem of batch learning from bandit feedback in the setting of extremely large action spaces. Learning from extreme bandit feedback is ubiquitous in recommendation systems, in which billions of decisions are made over sets consisting of millions of choices in a single day, yielding massive observational data. In these large-scale real-world applications, supervised learning frameworks such as eXtreme Multi-label Classification (XMC) are widely used despite the fact that they incur significant biases due to the mismatch between bandit feedback and supervised labels. Such biases can be mitigated by importance sampling techniques, but these techniques suffer from impractical variance when dealing with a large number of actions. In this paper, we introduce a selective importance sampling estimator (sIS) that operates in a significantly more favorable bias-variance regime. The sIS estimator is obtained by performing importance sampling on the conditional expectation of the reward with respect to a small subset of actions for each instance (a form of Rao-Blackwellization). We employ this estimator in a novel algorithmic procedure---named Policy Optimization for eXtreme Models (POXM)---for learning from bandit feedback on XMC tasks. In POXM, the selected actions for the sIS estimator are the top-p actions of the logging policy, where p is adjusted from the data and is significantly smaller than the size of the action space. We use a supervised-to-bandit conversion on three XMC datasets to benchmark our POXM method against three competing methods: BanditNet, a previously applied partial matching pruning strategy, and a supervised learning baseline. Whereas BanditNet sometimes improves marginally over the logging policy, our experiments show that POXM systematically and significantly improves over all baselines.

The purpose of the study is to describe perception of students on online learning. The research is a qualitative that employs a case study as the method of the research. It focuses on how they face online learning. The researcher took the data from the students in the first semester. The first semester students are regular students and workers (have part-time job while studying). To collect the data, each student was interviewed. The interview applied was semi-structured; a type of interview in the category of in-depth interviews, for instance, interviews that are more freely conducted when compared to structured interviews. The analyses from the students’ perception show that online learning is ineffective and most of students feel uninterested. They like teaching and learning in the classroom. Furthermore, some troubles in online learning emerged such as losing connection and adjusting with the online class (zoom / google meeting). The unclear and less interaction also become the problem of online learning. It needs of course several meetings offline at least in one semester. However, online learning should be applied for the students who cannot attend to classroom. Based on the interview, tools for online learning to use are WhatsApp, google classroom, google meeting, and Zoom meeting. In addition, the result of the research shows that the most interesting tool that the student use is zoom meeting since the students are able to experience face to face interaction either to the lecturers or the other students. Although there is sometimes a problem from the signal, they can communicate and ask directly to the lecturer at least. For the future, Hybrid learning can be employed as the technology grows.

Online learning has not been fully implemented by college Japanese teach-ers, due to the lack of supports from intelligent technology and the poor de-cision-making of college management. This paper attempts to develop a teaching model that effectively integrates online learning to enhance the learning ability of Japanese among college students. To begin with, the au-thors summarized the status quo, impacts, and problems of online learning among Japanese learners in colleges. Through expert survey and literature review, an index system was established to measure the online learning abil-ity of Japanese among college students. The weight of each index was de-termined by analytic hierarchy process (AHP). Based on index system, a grey relational analysis (GRA) model was set up to evaluate the online learn-ing ability of Japanese among college students. The research results provide a good reference for the online learning of other language majors.

What is the most statistically efficient way to do off-policy optimization with batch data from bandit feedback? For log data generated by contextual bandit algorithms, we consider offline estimators for the expected reward from a counterfactual policy. Our estimators are shown to have lowest variance in a wide class of estimators, achieving variance reduction relative to standard estimators. We then apply our estimators to improve advertisement design by a major advertisement company. Consistent with the theoretical result, our estimators allow us to improve on the existing bandit algorithm with more statistical confidence compared to a state-of-theart benchmark.

We consider multi-armed bandit problems with switching cost, and present asymptotically optimal adaptive allocation schemes. We illustrate an optimal allocation scheme for a large class of distributions which includes members of the exponential family.

We consider the problem of best arm identification in the fixed confidence setting for a variant of the multi-arm bandit problem. In our problem, each arm is associated with two attributes, a known deterministic cost, and an unknown stochastic reward. In addition, it is known that arms with higher costs have higher rewards across every sample path. The net utility of each arm is defined as the difference between its expected reward and cost. We consider two information models, namely, the full information feedback and sequential bandit feedback. We derive a fundamental lower bound on the sample complexity of any policy and also propose policies with provable performance guarantees that exploit the structure of our problem. We supplement our analytical results by comparing the performance of various candidate policies via synthetic and data-driven simulations.

This study aims to determine the mathematical problem solving ability of students in solving story problems with online learning. The method in this study used a qualitative descriptive research method. The results showed that there were differences in the ability to identify the elements that were known and asked, there were differences in the ability to choose and apply strategies or problem solving procedures and there were differences in the ability to examine and explain the results or answers according to problems between students with high, medium mathematical problem solving abilities. and low. There are 43% of students who have been able to follow and adapt online learning while 57% of other students have difficulty understanding the material being taught. There are 60.86% of students who have a high enough enthusiasm in online learning (online) but 39.13% of students are not happy if learning mathematics is done online due to factors that influence it such as technical constraints. There are 73.91% of students with a positive response to the problem of mathematical problem solving ability even though the learning is carried out online and 26.08% of students with a negative response.

This study used a systematic review, namely the research process by identifying previous research. Furthermore, this research applies a systematic analysis method that identifies the literature available on Google Scholar. This method has been used to produce original research as well as help researchers to critically analyze, evaluate and synthesize complex ideas. The five phases of systematic literature review are: 1) Framing questions for review 2) Relevant work 3) Assessing study quality 4) Summarizing the evidence 5) Interpret findings. From the results of the review a description it can be understood that there are student problems; they first have difficulty in terms of grammar, vocabulary, pronunciation. However, they also have problems in their psychology in learning speaking. Regarding online learning reviews, online learning is one of the alternative solutions for the learning process, including for students and teachers, during the COVID-19 pandemic. Although it raises some problems, for teachers and students, all parties must be ready to implement it. So that online learning has positive and negative impacts.

Low learning engagement and high dropout rates have been common problems in online learning. From the perspective of learners’ emotional experience, this paper investigated the self-regulation and flow experience of online learners through questionnaires to explore the relationship between them. The data were collected from the students in three countries of China, Tunisia and Serbia. They have participated in a three-week online course for Global Competition on Design for Future Education (GCD4FE). The results showed that the goal setting and the task strategies of Chinese students were significantly higher than that of students in Tunisia and there was a positive impact of self-regulation on flow of three countries. Consequently, some strategies can be taken to improve students’ self-regulation in online learning, so as to promote positive emotional experience to increase online learning engagement and reduce dropout rate.

During the COVID-19 pandemic, numerous schools halted face-to-face teaching and instead resorted to online classes. The impact of online learning on students’ academic worries, social relationships, and psychological well-being has received growing attention. Based on a sample of 1095 students aged 14–16 in 12 secondary schools in Hong Kong, structural equation modeling was used to investigate the association between online learning problems, academic worries and performance, social interaction, and psychosocial well-being during the pandemic. The path relationships among variables were compared by gender groups and socio-economic status (SES). Online learning problems were found to be negatively associated with psychosocial well-being through academic worries and social interaction as mediators. The effect of mediation among female students was larger than that of males, whereas the impact of academic worries on both psychosocial well-being and social interaction was stronger and significant only in the low SES group. As the COVID-19 pandemic significantly impacted the learning and well-being of students, attention should be given to gender and SES difference among students.

Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Share Icon Share Twitter Facebook Reddit LinkedIn Tools Icon Tools Reprints and Permissions Cite Icon Cite Search Site Citation Derbi Wulan Fitri, Toha Saifudin, Nur Chamidah; Classification of online school problems from tweets on Twitter using support vector algorithm. AIP Conference Proceedings 25 January 2023; 2554 (1): 030016. https://doi.org/10.1063/5.0104576 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentAIP Publishing PortfolioAIP Conference Proceedings Search Advanced Search |Citation Search