Do LLMs really “show their work“ when they perform chain of thought reasoning? “Measuring Faithfulness in Chain-of-Thought Reasoning“ is a new paper from Anthropic that aims to study this question empirically with a series of tests.
Timestamps:
00:00 - Measuring Faithfulness in Chain-of-Thought Reasoning
00:53 - What is Chain-of-Thought reasoning?
03:15 - Do the Chain-of-Thought Steps Really Reflect the Model’s Reasoning?
07:03 - Possible Faithfulness Failures
08:44 - Encoded Reasoning/Steganography
12:01 - Experiment Details
15:44 - Does Truncating the Chain of Thought Change the Predicted Answer?
16:53 - Does Editing the Chain of Thought Change the Predicted Answer?
17:14 - Do Uninformative Chain of Thought Tokens Also Improve Performance?
18:28 - Does Rewording the Chain of Thought Change the Predicted Answer?
20:20 - Does Model Size Affect Chain of Thought Faithfulness?
22:04 - Limitations
24:38 - Externalized Reasoning Oversight
Topics: ##ai #anthropic #CoT #reasoning
Link to the paper:
For related content:
- Twitter:
- Research lab:
- personal webpage:
- YouTube: @SamuelAlbanie1
- TikTok: @samuelalbanie
- Instagram:
- LinkedIn:
- Threads: @samuelalbanie
- Discord server for filtir:
(Optional) if you’d like to support the channel:
-
-
Credits:
Image credit (Chelsea photo) –#/media/File:
1 просмотр
262
73
1 неделя назад 00:08:27 1
Nightwish - Perfume Of The Timeless (OFFICIAL MUSIC VIDEO)
1 неделя назад 00:17:22 1
MDL NEWS-LA LOI SECRÈTE PACANANALALANALAC QUI CRÉE LA CONFUSION TOTALE DANS L’ESPRIT DES AMÉRICAINS
1 неделя назад 00:03:09 1
Alessandra - Queen of Kings (LIVE) | Norway 🇳🇴 | Grand Final | Eurovision 2023
1 неделя назад 00:01:37 1
HEARTS OF TITAN | Official Animated Trailer
1 неделя назад 00:04:52 1
Amy Winehouse I Love You More Than You’ll Ever Know (Inédit RARE) Cover BEST PERFORMANCE EVER LIVE
1 неделя назад 00:20:31 52
EUROVISION SONG CONTEST 2024 • Stage & Lighting Design • Behind the Scenes
1 неделя назад 00:04:17 1
Suicide - Rocket USA (Official Audio)
1 неделя назад 00:01:28 1
Hellboy Web of Wyrd - Reveal Trailer | PS5 & PS4 Games
1 неделя назад 00:03:28 1
The Jesus And Mary Chain - Silver Strings (Official Video)
1 неделя назад 00:13:21 1
I TEST a Chinese EBIKE
1 неделя назад 00:04:16 2
Sauce au Yaourt / Si Délicieuse que vous Oublierez l’Existence de la Mayonnaise !
1 неделя назад 00:03:32 1
Rooler & Sickmode & Paul Elstak - TAKE ME AWAY (Official Video)
1 неделя назад 01:59:20 1
UN BÉRET VERT PROMET DES ARRESTATIONS EN DIRECT ET LE DÉMANTÈLEMENT DU DÉPARTEMENT DE LA JUSTICE
1 неделя назад 05:16:58 1
[ 𝐄𝐋𝐄𝐆𝐀𝐍𝐓 𝐁𝐋𝐔𝐄𝐒 ] Relaxing Blues Music In The Bar - Fantastic Electric Guitar Blues | DEVILS BLUES
1 неделя назад 00:05:18 1
Maher Zain - Ya Habiba Ya Falastin (Beloved Palestine) | ماهر زين - يا حبيبة يا فلسطين
1 неделя назад 00:07:12 1
Bending Wood For Curved Porch Swing. WOODWORKING & HOW-TO Bent wood
1 неделя назад 00:13:25 1
Wow Look!! Stunning Leaf Embroidery with Herringbone Stitch/ embroidery for beginners/ needled work
1 неделя назад 02:59:05 1
CONVERSATION PROFONDE-EXPÉRIENCE DE MORT IMMINENTE-À QUOI POURRAIT-ELLE RESSEMBLER? VISION DE 40Kpds
1 неделя назад 00:52:12 1
The Food Mafia - How Criminals Infiltrate the Global Food Supply Chain | ENDEVR Documentary
1 неделя назад 00:03:57 1
The Chainsmokers - ’Closer’ Cover by Luke Holland & Aaron Richards
1 неделя назад 00:05:10 1
Ishq Kameena | Shakti | Shahrukh Khan | Aishwarya Rai I Sonu Nigam | Alka Yagnik
1 неделя назад 00:16:22 1
Leaves’ Eyes - Live at Wacken Open Air 2023
1 неделя назад 00:00:36 1
Tired Of Problems🎲| Jujutsu Kaisen & Chainsaw Man | AMV/EDIT 4k | HBD @Meshaev
1 неделя назад 00:00:17 1
AUSMAN portable speaker manufacturer and supplier, low price OEM factory China