From 75b2f3fe7c0b2a52a1f25e4dec3af29b5ad870f8 Mon Sep 17 00:00:00 2001
From: Yifan Peng <pengyifan.mail@gmail.com>
Date: Sun, 1 Nov 2020 12:52:39 -0500
Subject: [PATCH 1/3] Create README.md

---
 .../README.md                                 | 63 +++++++++++++++++++
 1 file changed, 63 insertions(+)
 create mode 100644 model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md

diff --git a/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md b/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md
new file mode 100644
index 000000000000..ca989320ac96
--- /dev/null
+++ b/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md
@@ -0,0 +1,63 @@
+---
+language: 
+- en
+thumbnail: 
+tags:
+- bluebert
+- bert
+- 
+license: 
+- PUBLIC DOMAIN NOTICE
+datasets:
+- PubMed
+
+---
+
+# BlueBert-Base, Uncased, PubMed
+
+## Model description
+
+A BERT model pre-trained on PubMed abstracts
+
+## Intended uses & limitations
+
+#### How to use
+
+Please see https://github.com/ncbi-nlp/bluebert
+
+## Training data
+
+We provide [preprocessed PubMed texts](https://ftp.ncbi.nlm.nih.gov/pub/lu/Suppl/NCBI-BERT/pubmed_uncased_sentence_nltk.txt.tar.gz) that were used to pre-train the BlueBERT models. 
+The corpus contains ~4000M words extracted from the [PubMed ASCII code version](https://www.ncbi.nlm.nih.gov/research/bionlp/APIs/BioC-PubMed/). 
+
+Pre-trained model: https://huggingface.co/bert-base-uncased
+
+## Training procedure
+
+*  lowercasing the text
+*  removing speical chars `\x00`-`\x7F`
+*  tokenizing the text using the [NLTK Treebank tokenizer](https://www.nltk.org/_modules/nltk/tokenize/treebank.html)
+
+Below is a code snippet for more details.
+
+```python
+value = value.lower()
+value = re.sub(r'[\r\n]+', ' ', value)
+value = re.sub(r'[^\x00-\x7F]+', ' ', value)
+
+tokenized = TreebankWordTokenizer().tokenize(value)
+sentence = ' '.join(tokenized)
+sentence = re.sub(r"\s's\b", "'s", sentence)
+```
+
+### BibTeX entry and citation info
+
+```bibtex
+@InProceedings{peng2019transfer,
+  author    = {Yifan Peng and Shankai Yan and Zhiyong Lu},
+  title     = {Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets},
+  booktitle = {Proceedings of the 2019 Workshop on Biomedical Natural Language Processing (BioNLP 2019)},
+  year      = {2019},
+  pages     = {58--65},
+}
+```

From a9635b816160cf0919dbfca29a6e78cc82adad20 Mon Sep 17 00:00:00 2001
From: Kevin Canwen Xu <canwenxu@126.com>
Date: Mon, 2 Nov 2020 13:23:21 +0800
Subject: [PATCH 2/3] Update README.md

---
 .../bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md    | 2 --
 1 file changed, 2 deletions(-)

diff --git a/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md b/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md
index ca989320ac96..30028bbc2a36 100644
--- a/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md
+++ b/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md
@@ -1,11 +1,9 @@
 ---
 language: 
 - en
-thumbnail: 
 tags:
 - bluebert
 - bert
-- 
 license: 
 - PUBLIC DOMAIN NOTICE
 datasets:

From adf9b9fdeae881e5b4d801aad6cb2a1d22e70262 Mon Sep 17 00:00:00 2001
From: Julien Chaumond <chaumond@gmail.com>
Date: Thu, 5 Nov 2020 09:03:08 +0100
Subject: [PATCH 3/3] Apply suggestions from code review

---
 .../bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md   | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md b/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md
index 30028bbc2a36..7701bb25f48c 100644
--- a/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md
+++ b/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md
@@ -3,11 +3,10 @@ language:
 - en
 tags:
 - bluebert
-- bert
 license: 
 - PUBLIC DOMAIN NOTICE
 datasets:
-- PubMed
+- pubmed
 
 ---