The paper discusses the idea of using BERT pre-trained network in bio-medical domain NLP tasks. It lays a path for future applications of Bert in different business verticals. Sharing some points from the review article I wrote on BioBERT on medium (https://medium.com/@raghudeep/biobert-insights-b4c66fde8fa7).

The major contribution is a pre-trained bio-medical language representation model for various bio-medical text mining tasks. Tasks such as NER from Bio-medical data, relation extraction, question & answer in the biomedical field.

For pretraining the model, authors have used a combination of general & medical corpora, which has shown interesting results. They have compared the results of all their combinations.