Author Interview - Typical Decoding for Natural Language Generation
#deeplearning #nlp #sampling
This is an interview with first author Clara Meister.
Paper review video hereé
Modern language models like T5 or GPT-3 achieve remarkably low perplexities on both training and validation data, yet when sampling from their output distributions, the generated text often seems dull and uninteresting. Various workarounds have been proposed, such as top-k sampling and nucleus sampling, but while these manage to somewhat improve the generated samples, they are hacky and unfounded. This paper introduces typical sampling, a new decoding method that is principled, effective, and can be implemented efficiently. Typical sampling turns away from sampling purely based on likelihood and explicitly finds a trade-off between generating high-probability samples and generating high-information samples. The paper connects typical sampling to psycholinguistic theories on human speech generation, and shows experimentally that typical sampling achieves much more diverse a
4 views
3
1
1 month ago 00:04:46 1
VVS On What Fans Can Expect From Their Debut, Describe Their Sound & More | Billboard Korea Cover
1 month ago 01:19:54 1
Climate: The Movie (The Cold Truth) Updated 4K version
1 month ago 00:31:38 1
’They’re Obviously Not Antisemitic Protests’: Jewish Yale University Professor Speaks Out
1 month ago 01:20:42 1
‘If Europe does not wake up, you will all be dead,’ says Naomi Wolf, author of Facing the Beast
1 month ago 02:15:58 1
РАZБИРАЕМSЯ С САМАRЦАМИ ПР0 0БМЄN, П0ЧЕМУ ИХ НЕ ZАБИRАЮТ И РАZНИЦА В S0DЕРЖАНИИ @dmytrokarpenko
1 month ago 01:55:10 9
The Truth About iDubbbz
1 month ago 00:16:15 1
Деловой визит представителей компании BK Engineering на завод ГК «Астэк-МТ»
1 month ago 00:01:01 1
Moment police take down Hainault sword attack suspect captured on home CCTV
1 month ago 01:16:41 1
ПРИСЕСТЬ ДОМА-ЭТО МОТИВАЦИЯ ПОЙТИ К ВАМ, ЕЩЁ ВАМ УГР0Ж*ЮТ ПОЛЯКИ, А НЕ МЫ @dmytrokarpenko
1 month ago 00:00:38 1
Jennifer Connelly at Dark Matters red carpet premiere | ScreenSlam
1 month ago 00:19:26 1
Anatomically Modern Human Dating Keeps Getting OLDER #grahamhancock #science #history #ancient
1 month ago 00:29:08 1
Hidden Messages Inside Ancient Amazonian Wall Art #grahamhancock #podcast #science #history
1 month ago 00:15:58 1
Finding Fingerprints From A Lost Ice Age Civilization #grahamhancock #science #history #ancient
1 month ago 00:38:05 1
The Controversial Chambers Under The Great Sphinx #grahamhancock #science #history #ancient #egypt
1 month ago 00:43:37 1
Алексей Шерстобитов (Леша Солдат) новое интервью 2023. Эксклюзив НТВ, программа “Авторитеты“
1 month ago 00:03:42 1
[FREE] Dance x Retro x Pop type beat - “My dear unknown love“
1 month ago 02:18:51 1
Tommy Robinson Revealing the real Britain
1 month ago 00:03:53 1
Israel’s Right To Defend Itself | Dave Rubin
1 month ago 00:41:36 1
Tony Parsons: Wonderfully Gloriously Meaningless
1 month ago 00:01:17 1
The Bloody Story Of How May Day Became A Holiday For Workers | TIME
1 month ago 02:12:27 1
ИДЕЙНОСТЬ И БЕСТ0LКОV0СТЬ рУSSК0ГО МИРА | РАЗГ0ВОР С ЛЕГИОНОМ “СВОБОДА РОССИИ“ | @dmytrokarpenko
1 month ago 00:03:45 4
“НУ ЧТО ЗА МАЩИНА!» СМОЛОВ ЯРКО ВРЫВАЕТСЯ В ИНТЕРВЬЮ ФОМИНА
1 month ago 00:24:57 1
Bart D. Ehrman and Javad Hashmi: Comparing the Historical Problems in the Qur’an and the Bible
1 month ago 00:00:00 1
PM Modi Speaks To NW18 On Zero Electricity Bil, CAA, Artcle 370 & More | N18L | CNBC TV18