University of Florida Health, NLP
GatorTron-OG
The model is released alongside GatorTron-S, a similar 345m-parameter cased Megatron checkpoint, but pre-trained on 22B words from the University of Florida SynGatorTron 5B NLG model (a Megatron GPT-3 model) as well as the full Pile dataset [1] and prompted to produce synthetic, de-identified discharge summaries using text sampled from MIMIC-III.
University of Florida Health, NLP
GatorTron-S
The model is designed to provide improved language understanding for downstream clinical tasks. It was trained and is released with a 50K token customized clinical vocabulary trained on the above listed data distribution.
The model is released alongside GatorTron-OG, a similar 345m-parameter cased Megatron checkpoint, but pre-trained on a large selection of de-identified, real-world clinical notes from the University of Florida Health System.