ELECTRA: Pre-Training Text Encoders as Discriminators Rather than Generators

This video explains the new Replaced Token Detection pre-training objective introduced in ELECTRA. ELECTRA is much more compute efficient due to de...