The increasing proliferation of unmanned aerial vehicles (UAVs) poses significant challenges to airspace security, necessitating the development of effective detection technologies. Passive detection techniques, such as passive radar, offer key advantages including spectrum efficiency and covert operation. However, passive radars that rely on coherent integration are often computationally expensive and dependent on strong Doppler signatures, rendering them ineffective for detecting low-speed or hovering UAVs. To overcome these limitations, we explore the use of channel state information (CSI) time series to characterize UAV presence and propose a temporal variation network for detecting UAV states, including hovering conditions. Our method utilizes digital terrestrial multimedia broadcast (DTMB) signals, which have wide coverage and high transmission power. By capturing DTMB signals with a single receiver, we reduce the complexity of passive detection systems. First, we perform channel estimation on the received signal to obtain CSI, which is arranged in frame order to form a CSI time series. This enables the modeling of interference channels caused by UAVs. We then propose the Channel Estimation and Temporal Variation Network (CETVNet), which leverages an adaptive noise reduction module and a multi-period feature extraction module to process these series for passive UAV state detection. Finally, a real-world signal dataset is collected using a software-defined radio device to train and evaluate CETVNet. Experimental results demonstrate that CETVNet achieves superior performance compared to state-of-the-art methods.