Cervical cancer ranks as the fourth most common cancer among females worldwide with roughly 528,000 new cases yearly. Significant progress in the realm of artificial intelligence particularly in neural networks and deep learning help physicians to diagnose cervical cancer more accurately. In this paper, we address a classification problem with the widely used VGG16 architecture. In addition to classification error, our model considers a regularization part during tuning of the weights, acting as prior knowledge of the colposcopic image. This embedded regularization approach, using a 2D Gaussian kernel, has enabled the model to learn which sections of the medical images are more crucial for the classification task. The experimental results show an improvement compared with standard transfer learning and multimodal approaches of cervical cancer classification in literature.