Collaborative perception enables connected and autonomous vehicles (CAVs) to overcome individual viewpoint limitations by exchanging perception data, making effective resource allocation crucial for timely transmission. However, existing studies focus on resource allocation within a single collaborative perception group (CPG), limiting their effectiveness in multi-group collaborative perception systems. In such a system, each CPG contributes differently to the overall collaborative perception performance, and it is challenging to evaluate and represent CPG system-level utilities. Meanwhile, competition for shared spectrum resources leads to interference and complicates the joint optimization of collaboration mechanisms and spectrum allocation, which is intensified by their temporal scale misalignment. To address these challenges, we propose a Utility-Aware Hierarchical Reinforcement Learning method (UAHRL) to jointly optimize collaboration mechanisms and spectrum allocation. Specifically, we introduce a hierarchical framework to handle temporally misaligned decisions through joint training. The upper layer optimizes the collaborative relationship and granularity over a longer time scale to enhance system-level collaborative performance, while the lower layer allocates spectrum resources over a shorter time interval to fulfill individual CPG transmission demand and enhance system transmission efficiency. To represent and utilize system-level utility, we leverage a feature-based confidence map to assess CAVs’ perception capability and complementarity. A mixing network in the upper layer further decomposes global performance into individual CPG utilities, enabling utility-aware resource allocation. Simulations show that UAHRL outperforms baseline methods in system-level collaborative perception in multi-group systems.