Capsule endoscopy (CE) is a non-invasive way to detect small intestinal abnormalities such as bleeding. It provides a direct vision of the patients entire gastrointestinal (GI) tract. However, a manual inspection of the huge number of images produced thereby is tedious and lengthy, and thus prone to human errors. This makes automated computer assisted decision-making appealing in this context. This paper introduces a novel deep-learning based semantic segmentation approach for bleeding zone detection in CE images. A bleeding image features three regions labeled as bleeding, non-bleeding, and background. Thus, a convolutional neural network (CNN) is trained using SegNet layers with three classes. A given CE image is segmented using our training network and the detected bleeding zones are marked. The proposed network architecture is tested on different color planes and best performance is achieved using the hue saturation and value (HSV) color space. Experimental performance evaluation is carried out on a publicly available clinical dataset, on which our framework achieves 94.42 % global accuracy and 90.69 % weighted intersection over union (IoU), two state-of-the-art classification metrics. Performance gains are demonstrated over several recent state-of-art competiting methods in terms of all performance measures we examined, including mean accuracy, mean IoU, global accuracy and weighted IoU.