• ParkinsonNet: We propose a novel end-to-end network, ParkinsonNet, for automatically qualifying the severity of PD motor symptoms. This network is capable of processing multiple motor tests without any manual feature design. • Temporal Self-Attention Enhancement Module (TAEM): To accurately perceive the gradual progression of motor symptoms, we introduce TAEM, which combines temporal compression with long-term dependency modeling, enabling robust and comprehensive temporal feature extraction. • Similarity Matching Module (SMM): To address the challenges of class imbalance and limited datasets, we design the SMM, that transforms the conventional classification or regression task into a similarity matching problem. This module aligns skeleton features with their most similar text-based features, leveraging semantic relationships for improved performance. • Empirical evaluations: Extensive experiments are conducted on two recent PD datasets, demonstrating the superiority of ParkinsonNet, and providing essential benchmark performance for future algorithm development and evaluation. Parkinson’s Disease (PD) is a progressive neurodegenerative disorder characterized by worsening motor symptoms such as bradykinesia, imbalance, tremor, rigidity, and gait disturbances. Clinician assessments are often time-consuming and costly, and the limited availability of specialists, along with patient mobility issues, complicates frequent evaluations. In this paper, we propose a novel end-to-end network to automatically qualify the severity of motor symptoms in PD, referred to as ParkinsonNet. Unlike most existing methods that focus on isolated tests, ParkinsonNet provides a unified learning framework that is evaluated across multiple PD motor symptoms, as demonstrated on finger tapping and gait. Specifically, to accurately perceive the gradual progression of motor symptoms throughout an entire test cycle (e.g., decrementing amplitude), a temporal self-attention enhancement module is designed by combining temporal compression with long-term temporal dependency modeling. To ease the issues of class imbalance and limited datasets, a similarity matching module is proposed that transforms the conventional classification or regression task into a similarity matching problem, matching the skeleton feature with its most similar texture feature. Additionally, a vector quantization module is incorporated to encode spatiotemporal features into a discrete-valued space, compressing and abstracting motion representations while retaining critical information for more accurate classification. Extensive experiments on two newly identified benchmark datasets demonstrate the superiority of our ParkinsonNet and set new benchmark performance for future algorithm development and evaluation. Our code will be released at this https URL .