New Siamese Neural Networks for Text Classification and Ontologies Alignment.

Integrating heterogeneous and complementary data in clinical decision support systems (e.g., electronic health records, drug databases, scientific articles, etc.) could improve the accuracy of these systems. Based on this finding, the PreDiBioOntoL (Predicting Clinical Diagnosis by combining BioMedical Ontologies and Language Models) project aims at developing a computer-aided clinical and predictive diagnosis tool to help clinicians to better handle their patients. This tool will combine deep neural networks trained on heterogeneous data sources and biomedical ontologies. The first obtained results of PreDiBioOntoL are presented in this paper. We propose new siamese neural models (BioSTransformers and BioS-MiniLM) that embed texts to be compared in a vector space and then find their similarities. The models optimize an objective self-supervised contrastive learning function on articles from the scientific literature (MEDLINE bibliographic database) associated with their MeSH (Medical Subject Headings) keywords. The obtained results on several benchmarks show that the proposed models can solve different biomedical tasks without examples (zero-shot). These results are comparable to those of other biomedical transformers that are fine-tuned on supervised data specific to the problems being addressed. Moreover, we show in this paper how these new siamese models are exploited in order to semantically map entities from several biomedical ontologies.