In computer vision (CV), object detection is a basic visual recognition issue that has been extensively researched in recent decades, especially in security surveillance. In recent years, there have been several studies on DL-based object recognition algorithms due to their enormous success in image classification. Accordingly, a new WMDSCBN-MLNet model is proposed in this work for object detection in video surveillance (VS). Initially, input videos are decomposed into individual frames, which are pre-processed using a Wiener filter to enhance quality. Next, the Modified Correlation-based Improved BIRCH (MC-IBIRCH) algorithm is employed for frame segmentation. Feature extraction is then carried out using a combination of Weight Parameter-based Local Gabor Increasing Pattern (WP-LGIP), Median Binary Pattern (MBP), and Multi-Texon features. The aggregated features are fed into the detection module. Finally, object detection is done using weighted MDSC-Batch normalization-based Modified LeNet (WMDSCBN-MLNet). From the study, it is observed that the proposed WMDSCBN-MLNet reached a better specificity of 0.975% which is higher than prevailing methods.