Efficient on-the-fly Web bot detection

A significant problem nowadays is detection of Web traffic generated by automatic software agents (Web bots). Some studies have dealt with this task by proposing various approaches to Web traffic classification in order to distinguish the traffic stemming from human users' visits from that generated by bots. Most of previous works addressed the problem of offline bot recognition, based on available information on user sessions completed on a Web server. Very few approaches, however, have been proposed to recognize bots online, before the session completes. This paper proposes a novel approach to binary classification of a multivariate data stream incoming on a Web server, in order to recognize ongoing user sessions as generated by bots or humans. The present approach uses deep neural networks combined with Wald's Sequential Probability Ratio Test to express the relationship between subsequent HTTP requests in an ongoing session and to assess the likelihood of each session being generated by a bot or human before it ends. Experimental results showed the ability of the proposed approach to detect Web bots online with high performance scores and a small number of false negatives, as evidenced by the Recall index, minimizing the impact on human visitors. Another valuable indicator is the speed of decision: the present method allows very quick classification of nearly all sessions, leaving only very few of them undecided.

Web servers provide most internet services, such as information sharing, financial, health, entertainment, and education. In this context, the web has become the principal place for attackers. Unfortunately, most defensive techniques for web servers cannot deal with the complexity and evolution of cyber attacks on HTTP requests. However, machine learning approaches can help detect some attacks. This work presents the RequestBERT-BiLSTM, a new model to detect possible HTTP request attacks without using Log Parser. We evaluated the model on public datasets such as CSIC 2010, ECML/PKDD 2007, and BGL. We also developed a new dataset from a real environment to evaluate the method. In addition, we illustrate that the traditional log analysis step can degrade the model’s performance due to parser errors. Furthermore, we compared the performance of the proposed approach with literature models, and we obtained a detection rate above 95%.KeywordsHTTP requestsAttack detectionLog parser

Until the development of HTTP request smuggling in 2005, individual HTTP requests were considered as independent entities and could not be split or merged. This is a security problem caused by inconsistent content length interpretation approach between web servers, or the web server is not fully implemented in accordance with the RFC standard. It is especially dangerous for web services with complex web architectures. It can route the victims to receive malicious responses, amplify the impact of certain low-threat vulnerabilities, steal user credentials, or bypass network devices’ defenses. However, since its concept and implementation are quite difficult to overcome, it is often ignored by many network administrators, making users who browse such websites vulnerable to the HTTP request smuggling attacks. This paper proposes a general solution to deal with various HTTP request smuggling attacks. A reverse proxy implemented by Flask validates and cleans dubious HTTP requests from the client side and ensures that the original requests comply with RFC standards. Therefore, the website administrators no longer need to configure complicated network settings or customize some open-source project codes to resist or minimize the risk of the HTTP request smuggling attacks. A series of experiments demonstrate that this method is effective and practical.

The trustworthiness between anonymous web service client and provider influences service stability and collaboration. Trustworthiness includes some aspects such as service's security, controllability and survivability. So, a definition of trustworthiness for web service is given in the paper, and then a web service trustworthiness evaluation and management model is brought forth inspired by human small-world network. The model consists of three web service federations: WSRRC, APAEAS and AWSORT. WSRRC is a Web Service Resource Register Center, which is established by UDDI protocol. APAEAS is an Area Proxy Authentication Evaluating Autonomy System, which collects some authentication information of web service clients, accepts clients' special requirement and feedbacks service's trustworthiness values to AWSORT. AWSORT is an Area WS Resource Organizing Tree, which organizes and manages web service resources; records web service trustworthiness values, keeps web service state, assigning web service. The model establishes a trustworthy environment for anonymous web service clients and providers. Furthermore, some detailed evaluating parameters about service trustworthiness and quality is discussed and some service management algorithms are proposed in the paper. The simulation results show that model is feasible for semantic grid integration and establishment for virtual organization.

Sophisticated technology in the world has been developing very rapidly, especially in terms of accessing all the information that is on the web. Website is one of technological sophistication that presents various kinds of information needed, from children to adults easily accessing websites on the internet. Unwittingly the number of accesses made by users around the world causes a web server or a place that provides a website will be weak and can occur down. When a web server becomes weak, hackers will use it to attack the web server, retrieve data, important information, and even more fatal is when user data is stolen and misused by hackers. The web developers only focus on a website appearance but do not focus on access and security of the website. Therefore, we need an optimal web server that can accommodate the many accesses caused by users and the security of the web server to safeguard data and information stored. This research applies a package namely Reverse Proxy which is used to optimize a web server and Web Application Firewall which is used to maintain the security of a web server.

Overload control in QoS-aware web servers

HTTP low and slow DoS is a kind of attack wherein an attacker sends a stream of very slow HTTP requests to a web server. These slow HTTP requests are aimed at specific applications or server resources. By sending HTTP requests very slowly, these attacks waste the server’s resources as the server has to wait for the slow HTTP requests to be completed. HTTP low and slow DoS attacks are primarily used against thread-based web servers like Apache and IIS. By sending very slow requests, these attacks tie up the server threads. This results in a Denial of service for other legitimate users. Since low and slow DoS attacks do not require flooding or sending a large number of HTTP requests, they are not easily detected. Traditional network layer tools cannot detect low and slow DoS attacks and other mitigation and detection strategies are required for the detection of low and slow DoS attacks. In this paper, we propose an LSTM deep learning-based approach for detecting HTTP DoS attacks. The proposed study was conducted on the CIC DoS dataset and a synthetically generated dataset and achieved an impressive accuracy of 0.99.

A new approach to web server attacks detection based on the statistical analysis of HTTP requests and principles of immunocomputing is proposed in the paper. We use a set of legitimate HTTP requests as training data. Each request is represented as its byte frequency distribution. Immunocomputing is used to calculate the binding energy between the training data and sampled HTTP requests. If the binding energy is less than some threshold, an alarm will be triggered. Our approach has been tested with the DARPA data set and the data set collected from the vulnerable web server. We have shown that our approach detects various attacks with a high degree of accuracy.

The original C10K problem [1] studies how to provide reasonable service to 10,000 simultaneous clients or HTTP requests using a normal web server. We call the following problem the reverse C10K problem, or RC10K -- how to support 10,000 simultaneous outbound HTTP requests running on a web server. The RC10K problem can be found in scenarios like service orchestrations and server-side mashups. A server-side mashup needs to send several simultaneous HTTP requests to partner services for each inbound request. Many approaches to improving the performance and scalability of HTTP servers can be applied to tackle the original C10K problem. However, whether these approaches can tackle the reverse C10K problem needs to be verified. In this paper, we discuss the RC10K problem for server-side mashups, and propose a design that takes advantage of advanced I/O, multithreading, and event-driven programming. The results of analysis and experiments show that our design can reduce the resource requirements by almost one order of magnitude with the same performance provided, and it is promising to tackle the RC10K problem.

The popularity of web services has largely influenced the way in which enterprise business is conducted. Since web services enable easy accessibility of data, dynamic connections, and relatively less human interventions, ensuring confidentiality and integrity of data that is transmitted via web services protocols becomes more significant. If a single service does not fulfill the service consumer requirements, it is necessary to compose several web services, which together satisfy the user requirements. Security attacks occur on SOAP messages that are communicated among web services while accessing a service or during service composition. Most of the existing works on web services security have provided solutions only for ensuring client authentication, confidentiality, and integrity of information in network layer and not in application layer. WS-Security and XML based web service security also provides message layer security in network layer and not in application layer. Hence, a novel approach that prevents the message alteration attack on SOAP messages and a security solution that detects and overcomes XML injection attack have been proposed in this paper. Our approach uses pluggable APIs in the service provider side and security services in the middleware side. The attacks were simulated and non-vulnerability of the proposed solutions to these attacks have been verified.

Online services are no longer isolated. The release of public APIs and technologies such as web hooks are allowing users and developers to access their information easily. Intelligent agents could use this information to provide a better user experience across services, connecting services with smart automatic behaviours or actions. However, agent platforms are not prepared to easily add external sources such as web services, which hinders the usage of agents in the so-called Evented or Live Web. As a solution, this paper introduces an event-based architecture for agent systems, in accordance with the new tendencies in web programming. In particular, it is focused on personal agents that interact with several web services. With this architecture, called MAIA, connecting to new web services does not involve any modification in the platform.

Web services technologies change the software industry drastically by developing and integrating enterprise web services and applications in order to enable the users to access them. There are several research initiatives in the field of Web services for enterprise applications. However, none of the initiatives have attempted for web applications pertaining to academic institution using Web services technology. Web service composition is the combination of web services from different service providers to create a more sophisticated, value added web service to the web users. Researchers put on efforts to bring out the state-of-the art in web services technologies and security technologies for web services. The composition of web services is another promising area for research and development. A study reveals that there are number of approaches providing secure web services. However, there is no attempt made so far to secure the services at multi-level. This paper proposes a Multi-level Secured Framework (MLSF) for the students, faculty and employees of the academic institutions which provides secured composite web services such as admission service, fees payment service, library service, personalized service and so on. The MLSF enhances the security at various levels such as user level, device level, web services level and institution level.

This paper present possible further development of intelligent life-event based portals. Introduction of ‘deep’ knowledge (semantics) in the shape of ontologies, distributed components in the shape of web services and intelligent agents are three main topics of probable future architecture of e-gov. This paper relates to the developed prototype of life-event based portal and its possible future architectural refinement.

A Web-Based Perioperative Dashboard as a Platform for Anesthesia Informatics Innovation.

Web application security has become a major requirement for any business, especially with the wide web attacks spreading despite the defensive measures and the continuous development of software frameworks and servers. In this study, we present a proposed model for a web application firewall that used machine learning and features engineering to detect common web attacks. Our proposed model analyses incoming requests to the webserver, parses these requests to extract four features that describe completely HTTP request parts (URL, payload, and headers), and classifies whether a request is normal or an anomaly. We took into consideration the limitation of previous works that use URL and payload only in classification and provided five features that describe and summarize all parts of the HTTP request using features engineering and previous experience in the field of the software security domain. Extracted features are length of request, percentage of characters allowed, percentage of special characters, and attack weight. These features were calculated for four different datasets CSIC 2010, HTTPParams 2015, Hybrid dataset (CSIC 2010 and HTTPParams), and real logs for the compromised web server. We evaluated our proposed model by using these updated datasets with four classification algorithms (Naive Bayes, logistic regression, decision tree, and support vector machine) with two methods (train test split and cross-validation) to negate the probability of overfitting and ensure that features are effective. Features values for a normal request are usually short request length, large allowed character ratio, small special character ratio, and zero attack weight or close to zero. Features values for anomaly requests are large request length, small allowed character percentage, large special character percentage, and very large numerically attack weight. Our proposed model achieved a classification accuracy of 99.6% with datasets used in research studies in this field and 98.8% with datasets of real web servers.