Frontiers of music technologies

Deep Learning has emerged as a powerful set of computational methods achieving great results in a variety of different tasks. Music signal processing, a field with rich commercial applications, seems to benefit too from this data-driven approach. In this paper a review of the state of the art Deep Learning methods applied on music signal processing is provided. A special focus is given in music information retrieval and music generation. In addition, possible future research directions are discussed.

In this paper, we present e-Clips, a framework for the evaluation of content-based indexing and retrieval techniques applied to music video clips. The e-Clips framework integrates different video and audio feature extraction tools, whether automatic or manual. Its goal is to compare the relevance of each type of feature for providing a structured index that can be browsed, finding similar videos, retrieving videos that correspond to a query, and pushing music videos to the user according to his preferences. Currently, over 100 distinct music video clips have been indexed. For each video, shot boundaries were detected and key frames were extracted from each shot. Each key frame image was segmented into visual objects. The sound track was analyzed for basic features. Textual data, such as a song title and its performer was added by hand. The e-Clips framework is based on a client-server architecture that can stream VHS-quality video through an 100 Mbs Intranet. It should help evaluate the relevance of the descriptors generated by content-based indexing tools and suggest appropriate graphical user interfaces for non-specialist end users.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The digitization, analysis, and processing technology of music signals are the core of digital music technology. There is generally a preprocessing process before the music signal processing. The preprocessing process usually includes antialiasing filtering, digitization, preemphasis, windowing, and framing. Songs in the popular wav format and MP3 format on the Internet are all songs that have been processed by digital technology and do not need to be digitalized. Preprocessing can affect the effectiveness and reliability of the feature parameter extraction of music signals. Since the music signal is a kind of voice signal, the processing of the voice is also applicable to the music signal. In the study of adaptive wave equation inversion, the traditional full-wave equation inversion uses the minimum mean square error between real data and simulated data as the objective function. The gradient direction is determined by the cross-correlation of the back propagation residual wave field and the forward simulation wave field with respect to the second derivative of time. When there is a big gap between the initial model and the formal model, the phenomenon of cycle jumping will inevitably appear. In this paper, adaptive wave equation inversion is used. This method adopts the idea of penalty function and introduces the Wiener filter to establish a dual objective function for the phase difference that appears in the inversion. This article discusses the calculation formulas of the accompanying source, gradient, and iteration step length and uses the conjugate gradient method to iteratively reduce the phase difference. In the test function group and the recorded music signal library, a large number of simulation experiments and comparative analysis of the music signal recognition experiment were performed on the extracted features, which verified the time-frequency analysis performance of the wave equation inversion and the improvement of the decomposition algorithm. The features extracted by the wave equation inversion have a higher recognition rate than the features extracted based on the standard decomposition algorithm, which verifies that the wave equation inversion has a better decomposition ability.

O presente artigo busca propor um espaço enunciativo para a análise de videoclipes musicais enquanto objetos da cultura visual incorporados ao universo da arte contemporânea, através do estudo de caso do projeto “Valtari Mystery Film Experiment” (2012), do grupo musical islandês Sigur Rós. Para isto, promove aproximações entre os campos da comunicação e os domínios teóricos das artes visuais, traçando pontos de observação que partem da subjetividade dos músicos e diretores envolvidos até chegarem a questões de reorganização da circulação de obras na contemporaneidade.Palavras-chave: Videoclipes, regime contemporâneo, cultura visual

As the lifespan of virality on TikTok has become increasingly short and operates at a smaller scale, music creators are grappling with what it means to be successful in this later stage of the platform. For this study, I conducted 27 semi-structured interviews with music artists and creators about how short-form video platforms like TikTok have affected the way they work and create music. Many participants believe that while the creation of viral content based on memes and covers of popular songs affords constant visibility, it does not result in the monetary or career milestones that they desire, so they need to use different approaches. In this article, I characterize this new period of online musical labor as a period defined by “platform negotiation,” in which creators evaluate platform demands and work to approach their content creation in ways that prioritize their personal and professional goals. This includes strategic decision-making to negotiate their presence—which can include their identity, brand, image, etc.—across platforms over time. As they engage in relational labor to support continued work, music creators cannot simply rely on the tenets of optimization to be successful. Rather, many are pivoting to utilizing content creation in ways that aim to prioritize fandom and career sustainability. This article examines the ways in which music creators grapple with the challenges of short-form video and how musical labor is changing online.

In this article I discuss the genre of music video re-performance in the context of the formation and constitution of children's cultures and subjectivities. The viral dance-song cover, as I term it, is a digitally recorded and shared imitation of an iconic film or music video clip that formulates transmissible repertoires of movement and so becomes ‘viral’ in the process. Through examples taken from the Disney animation film Frozen and Sia's Chandelier, I think through the ways that contemporary childhood is not only visually construed, but energetically, affectively animated by the diffusive logic of the viral. In doing so I aim to understand how performances of the child-figure are transformed by the digital logic in which it operates, but also how the “traffic” of gesturality so exceedingly visualized by the logic of the viral cover signals a kind of performative compulsion towards transmissibility in which the behaviours of acting like a child become both tested and consolidated.

Women have been reduced to their body and the female body has been dismembered, commodified, and sexually objectified in the media for entertainment and marketing purposes (Bartky 1990; Basow 1992). This restrictive presentation of women as subjective beings who exist solely to be ogled and desired by men has been shown to exert a powerful effect not only on women’s psychological well being but also on men’s perception of women (Basow 1992; Fredrickson and Roberts 1997; Killbourne and Jhally 2001). In the film, Dreamworlds 3: Desire, Sex and Power in Music Video, director Sut Jhally seeks to expose how the music video medium uses this same narrow, objectifying perspective of femininity and the female body to tell a story. His campaign is to alert the audience to messages that they may have unwittingly accepted. Jhally manages to deliver this message in a way that avoids being dry or preachy. He seeks to show and not tell, so to speak, by using music videos themselves to reveal their message of manipulation. Dreamworlds 3 is an extension of Jhally’s work in Dreamworlds 1 and 2, with updated images from more recent music videos as well as the incorporation of video and photographs showing violence against women at the Puerto Rican Pride parade in 2000 and at Seattle’s 2001 Mardi Gras celebration. If you appreciated the earlier analyses of Jhally in Dreamworlds, and in Dreamworlds 2 then you are likely to value his updated version; critics of previous films will find his current offering equally problematic. Jhally, who is a professor of communications at the University of Massachusetts at Amherst, uses an analytic technique similar to that employed by Jean Kilbourne (Kilbourne & Jhally, 2001; Kilbourne, Vitaglione & Stallone, 1979; 1987) in her Killing Us Softly series. Both of these series use a large collection of media images presented with narration to raise the consciousness of the viewer to the impact of these images. Jhally’s analysis is of music video images of women as sexual objects and men as sexual aggressors. Similar to Kilbourne, Jhally emphasizes the feminist socio-cultural perspective that gender is socialized, not innate, and that this socialization occurs through the stories we tell about ourselves as well as the stories told to and about us that are woven into our cultural narratives. Studying these cultural narratives allows us, in Jhally’s words, to “see how our culture teaches us to be men and women.” Following from this, it seems that both Jhally and Kilbourne share the overarching goal of teaching and encouraging media literacy to an audience primarily of young adults and adolescents. Although Jhally’s film uses a similar analysis to that of Kilbourne, the two films exhibit qualitative differences due to the media on which they focus. While Kilbourne analyzes popular advertisements from magazines and prime time TV, Jhally is looking at music videos that are inherently more sexually explicit and graphically violent. While both Kilbourne and Jhally talk about the use of pornographic images, Kilbourne injects far more humor into her commentary. In Dreamworlds 3, on the other hand, there is very guttural feeling of danger that is evoked partly because the music video clips are so full of sound and movement that they engage all of the viewer’s senses therefore eliciting a more visceral response. Additionally, Jhally’s very authoritative commentary weaves over and through the videos like a floating, bodiless narrator who sees through the glitzy surface into the deeper more frightening depths. The fact that the viewer never sees Jhally reinforces the message that the videos themselves Sex Roles (2009) 60:745–747 DOI 10.1007/s11199-008-9573-5

Enhancing the Social Impact of Contemporary Music with Neurotechnology

Timbral autoencoders, a class of generative model that learn the timbre distribution of audio data, are a current research focus in music technology; however, despite recent improvements, they have rarely been used in music composition or musical systems due to issues of static musical output, general lack of real-time synthesis and the unwieldiness of synthesis parameters. This project proposes a solution to these issues by combining timbral autoencoder models with a classic computer music synthesis technique in wavetable synthesis. A proof-of-concept implementation in Python, with controllers in Max and SuperCollider, demonstrates the timbral autoencoder’s capability as a wavetable generator. This concept is generally architecture agnostic, showing that most existing timbral autoencoders could be adapted for use in real-time music creation today, regardless of their capabilities for real-time synthesis and time-varying timbre.

Stacked auto-encoders based visual features for speech/music classification

Chroma features are a popular tool in musical signal processing and information retrieval tasks, providing a compact representation of the tonal content of a piece of music. A variety of approaches to chroma estimation have been proposed, most of which rely on the summation of related frequency partials. However, frequency partials may be incorrectly assigned due to the log/linear relationship of frequency and pitch. Variations of chroma employing overtone suppression strategies are found in the literature. We propose a compositional model of chroma, which considers a coarse modelling of the effects of overtones in the expected chroma vectors of single notes. Synthetic chord recognition experiments indicate the usefulness of the proposed approach.

In the last few years various musicological studies on the Byzantine Chant (BC) have been made via computer analysis. Due to the BC vocal nature and the pitch variability of the BC notes during the performance, the value of a music interval between two such notes in a certain BC mode - “echos”, is ambiguous. Because of that pitch variability a new method of BC intervals measurement is proposed, based on statistical processing of the pitch track that can be done after a proper pitch analysis of the BC recordings. The music scale of a particular BC echos is computed from the histogram of the pitch values by applying an algorithm that calculates a Gaussian Mixture Model approximating the music scale of that BC echos. This new method is applied, as an example, to ten BC recordings - performances of the Cherubic Hymn composed in the fourth echos (aghia) by Petros Lampadarios. The recordings come from the DAMASKINOS Corpus of University of Athens and the analysis results are in agreement with both the theoretical values of the Patriarchal Committee 1881 (approximation of up to 1 moria) and the findings in other similar studies. This method of analysis can be a base for both musicological analysis and music signal processing.

Detecting multiple pitches (F0s) and segregating musical instrument lines from monaural recordings of contrapuntal polyphonic music into separate tracks is a difficult problem in music signal processing. Applications include audio-to-MIDI conversion, automatic music transcription, and audio enhancement and transformation. Past attempts at separation have been limited to separating two harmonic signals in a contrapuntal duet (Maher, 1990) or several harmonic signals in a single chord (Virtanen and Klapuri, 2001, 2002). Several researchers have attempted polyphonic pitch detection (Klapuri, 2001; Eggink and Brown, 2004a), predominant melody extraction (Goto, 2001; Marolt, 2004; Eggink and Brown, 2004b), and instrument recognition (Eggink and Brown, 2003). Our solution assumes that each instrument is represented as a time-varying harmonic series and that errors can be corrected using prior knowledge of instrument spectra. Fundamental frequencies (F0s) for each time frame are estimated from input spectral data using an Expectation-Maximization (EM) based algorithm with Gaussian distributions used to represent the harmonic series. Collisions (i.e., overlaps) between instrument harmonics, which frequently occur, are predicted from the estimated F0s. The uncollided harmonics are matched to ones contained in a pre-stored spectrum library in order that each F0‘s harmonic series is assigned to the appropriate instrument. Corrupted harmonics are restored using data taken from the library. Finally, each voice is additively resynthesized to a separate track. This algorithm is demonstrated for a monaural signal containing three contrapuntal musical instrument voices with distinct timbres.KeywordsHarmonic SpectrumHarmonic SeriesMusic Information RetrievalPitch DetectionSeparate TrackThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The traditional computer-aided design (CAD) software is often not realistic in rendering scenes, such as light, materials, shadows, and other details, which leads designers to spend a lot of energy on manual adjustment.The rendering algorithm based on deep learning (DL) can automatically learn the complex output, and the effect is usually better than traditional methods.In this article, a convolutional neural network (GA-CNN) based on genetic algorithm (GA) optimization is proposed, and it is combined with CAD data and applied to the rendering of environmental art design scenes.This method uses GA to optimize the parameters of CNN to improve its learning effect and generalization ability.From the overall trend, the response speed of GA-CNN algorithm and support vector machine (SVM).The response speed of the traditional CNN method is relatively slow, and the fluctuation range is large.This is because traditional CNN needs a lot of convolution operations and feature extraction when dealing with artistic scenes in complex environments, which leads to a long calculation time.In practical application, it is necessary to select appropriate algorithms and parameter settings according to specific task requirements to achieve the best results.

We develop a theoretical framework for representation and automated generation of South Indian classical music. The foundational part of the latter is based on symbolic dynamics and is implemented by translating the lexicographic rules for a rāga to constraints on a Markov chain whose state space is a layered graph. We analyze the statistical properties of this Markov chain from the point of view of information theory. We also develop several tools in music signal processing, such as, (a) a procedure for automated generation of gamakas or ornamental notes, unique to South Indian classical music, (b) rhythm synchronization, and (c) an algorithm for perceptual scale shifts. The Online Supplement has computer synthesized music from Śankarābharaṇa rāga, rhythm synchronized music for a Mōhana rāga composition, original and synthesized gamakas over madhyama for Begaḍa and Nīlāmbari rāgas, and perceptually scale shifted rāgas Hindoḷa, Madhyamāvati, Śuddha sāveri, and Udaya ravi chandrikā rāgas from the base Mōhana rāga.