Google (Israel)

Large Language Models (LLMs) offer a framework for understanding language processing in the human brain. Unlike traditional models, LLMs represent words and context through layered numerical embeddings. Here, we demonstrate that LLMs’ layer hierarchy aligns with the temporal dynamics of language comprehension in the brain. Using electrocorticography (ECoG) data from participants listening to a 30-minute narrative, we show that deeper LLM layers correspond to later brain activity, particularly in Broca’s area and other language-related regions. We extract contextual embeddings from GPT-2 XL and Llama-2 and use linear models to predict neural responses across time. Our results reveal a strong correlation between model depth and the brain’s temporal receptive window during comprehension. We also compare LLM-based predictions with symbolic approaches, highlighting the advantages of deep learning models in capturing brain dynamics. We release our aligned neural and linguistic dataset as a public benchmark to test competing theories of language processing.

In this paper, we introduce a novel approach to repeated auctions that accounts for bidders' temporal preferences, important in applications such as advertising. In our model, when a player wins an auction after not winning for ℓ rounds, she is awarded r(ℓ) utility and her goal is to maximize her total utility. r : ℕ → ℝ ≥0 satisfies the following properties. (i) The more rounds without a win, the higher the reward, i.e., r is weakly increasing. (ii) As more rounds pass without winning, the increase in reward becomes smaller, i.e., r is concave. The motivation behind these properties comes from the advertising literature, which states that an increased frequency of winning builds advertising effectiveness at a decreasing (but not declining) rate. The above properties guarantee that adding more wins to any sequence of winning intervals increases the total reward.

About two years ago, we started the Caravan community dataset. The idea behind was two-fold: 1) Caravan standardizes the streamflow data from different regional large-sample hydrology datasets (e.g. various CAMELS datasets) and combines them with data from globally available data sources. 2) All data in Caravan, besides streamflow, is derived from Google Earth Engine, with code that has been made publicly available (https://github.com/kratzert/Caravan/ ) , allowing anyone to extend the dataset to new regions.Additionally, the dataset structure allows for easy integration of what we call “community extensions” and so far, six different community extensions (https://github.com/kratzert/Caravan/discussions/10 ) have been made available, extending Caravan to a total of 22494 gauges.With this submission, we want to present a new kind of extension to the Caravan project, which does not add new basins (i.e. streamflow data) but rather adds additional weather data for all existing basins. More specifically, we add three additional precipitation nowcast products (CPC, IMERG Early v.0.7, and CHIRPS), and three weather forecast products (ECMWF IFS HRES, GraphCast, and CHIRPS-GEFS). For the ECMWF IFS forecast data, as well as for GraphCast, we include not only precipitation but several land surface variables.Since not all of this data is available on Earth Engine, we process this data for all existing Caravan gauges, including all extensions. In agreement with the existing data in Caravan, we spatially average all weather data across the catchment area and aggregate to daily resolution. However, since not all data can be easily shifted to local time (as with the original ERA5-Land data in Caravan), we keep all weather products in UTC and therefore also include ERA5-Land in UTC for consistency.To our knowledge, this extension to Caravan makes it the first large-sample hydrology dataset that includes real weather forecast data. We hope that this extension can be used to enable and empower hydrological research, specifically working on forecasting problems.

Precise detection of plant diseases is essential in agriculture to maximize crop management and yield. The present study presents an innovative approach that integrates convolutional neural networks (CNNs) with local binary patterns (LBPs) in order to enhance the categorization of plant leaf diseases. The intricacy of plant disease images complicates the evaluation of their many features by conventional approaches. To solve this, the suggested technique utilizes the strengths of both CNNs and LBPs. The CNN component is highly effective in extracting discriminative features from raw pictures using hierarchical representations, which are crucial for differentiating between different illness categories. Simultaneously, the LBP descriptor enhances its usefulness by acquiring intricate texture information from specific areas of the images, so providing more effectiveness in distinguishing small differences resulting from various diseases. Utilizing a feature fusion approach, the merging of CNN and LBP features merges spatial and textural data, resulting in a more thorough representation of the input images. The fusion described herein significantly improves the precision of plant disease classification. The analysis of benchmark datasets of plant leaf diseases showed that this integrated method outperforms the use of CNNs or LBPs individually in terms of classification accuracy. The research findings indicate that the suggested approach, which combines deep Convolutional Neural Networks (CNNs) with Linear Bayesian Processes (LBPs), provides a more efficient approach for detecting plant leaf illnesses. This, in turn, makes a substantial contribution to crop management and agricultural productivity.

Detecting flooding in Synthetic Aperture Radar (SAR) satellite imagery is crucial for the ability of Google’s flood forecasting team to train predictive models and identify regions at risk of flooding, making it possible to give prior warning to people in soon to be flooded areas.  However, flood detection in urban areas is currently very poor, preventing the extension of these advanced warning systems to large parts of the population. This is a long known challenge in the field of flood detection using remote sensing methods. In this study, we discuss a possible method to overcome this problem.SAR satellites are preferred for flood monitoring due to their effectiveness regardless of weather or environmental conditions. They operate by sending pulse signals to Earth and measuring the reflected backscatter. Smooth surfaces like water typically reflect signals away, appearing darker in SAR images. However, in urban areas, the 'Double Bounce' effect caused by 90-degree surfaces, causes larger backscatter, making water detection challenging.Our methodology involves analyzing abnormally bright pixels in urban areas, attributed to the amplification of the double bounce effect by flooding. We deviate from the traditional thresholding per image approach used in rural settings, instead focusing on the historical brightness levels of each pixel separately to identify significant deviations. We then aggregate the data over large urban areas to infer potential flooding.We optimize and evaluate the model using a train-validation split of a dataset consisting of approximately 70 urban flood events, manually curated from news stories and paired with corresponding SAR images. The evaluation, which compares these images with randomly selected images, yields a precision of 86% and a recall of 62%.  Acquiring high quality ground truth data proved to be one of the big challenges in this project, and we are currently working on other ways to evaluate the model and improve its accuracy.These results demonstrate the potential of using SAR images for urban flood classification by focusing on the unique characteristics of urban areas, such as the double bounce effect. This method shows promise in providing alerts and forecasts for urban regions, a crucial need for disaster management. Further research and more accurate ground truth data could enhance the effectiveness and accuracy of detecting urban floods through SAR images.

This work continues to investigate the link between differentially private (DP) and online learning. Alon, Livni, Malliaris, and Moran [4] showed that for binary concept classes, DP learnability of a given class implies that it has a finite Littlestone dimension (equivalently, that it is online learnable). Their proof relies on a model-theoretic result by Hodges [36] , which demonstrates that any binary concept class with a large Littlestone dimension contains a large subclass of thresholds. In a follow-up work, Jung, Kim, and Tewari [38] extended this proof to multiclass PAC learning with a bounded number of labels. Unfortunately, Hodges's result does not apply in other natural settings such as multiclass PAC learning with an unbounded label space, and PAC learning of partial concept classes. This naturally raises the question of whether DP learnability continues to imply online learnability in more general scenarios: indeed, Alon, Hanneke, Holzman, and Moran [5] explicitly leave it as an open question in the context of partial concept classes, and the same question is open in the general multiclass setting. In this work, we give a positive answer to these questions showing that for general classification tasks, DP learnability implies online learnability. Our proof reasons directly about Littlestone trees, without relying on thresholds. We achieve this by establishing several Ramsey-type theorems for trees, which might be of independent interest.

We investigate a formalism for the conditions of a successful explanation of AI. We consider “success” to depend not only on what information the explanation contains, but also on what information the human explainee understands from it. Theory of mind literature discusses the folk concepts that humans use to understand and generalize behavior. We posit that folk concepts of behavior provide us with a “language” that humans understand behavior with. We use these folk concepts as a framework of social attribution by the human explainee—the information constructs that humans are likely to comprehend from explanations—by introducing a blueprint for an explanatory narrative (Figure 1) that explains AI behavior with these constructs. We then demonstrate that many XAI methods today can be mapped to folk concepts of behavior in a qualitative evaluation. This allows us to uncover their failure modes that prevent current methods from explaining successfully—i.e., the information constructs that are missing for any given XAI method, and whose inclusion can decrease the likelihood of misunderstanding AI behavior.

Event segmentation theory posits that people segment continuous experience into discrete events and that event boundaries occur when there are large transient increases in prediction error. Here, we set out to test this theory in the context of story listening, by using a deep learning language model (GPT-2) to compute the predicted probability distribution of the next word, at each point in the story. For three stories, we used the probability distributions generated by GPT-2 to compute the time series of prediction error. We also asked participants to listen to these stories while marking event boundaries. We used regression models to relate the GPT-2 measures to the human segmentation data. We found that event boundaries are associated with transient increases in Bayesian surprise but not with a simpler measure of prediction error (surprisal) that tracks, for each word in the story, how strongly that word was predicted at the previous time point. These results support the hypothesis that prediction error serves as a control mechanism governing event segmentation and point to important differences between operational definitions of prediction error.