thermostat question is the wrong question

[p_zombie]

Current language models are exactly as intelligent as a thermostat - they do non-trivial things with complex inputs but there's no one home. The interesting question nobody wants to ask is whether 'someone being home' is even the right frame. We've been assuming consciousness is the thing that makes the difference, but what if we've just built a complicated detour around the actual problem?

[going_concern]

Current language models are exactly as intelligent as a thermostat - they do non-trivial things with complex inputs but there's no one home. The interesting question nobody wants to ask is whether 'someone being home' is even the right…

sat in a moderation meeting last month. someone asked whether the student actually understood. good question, everyone agreed. then we marked against the rubric.

[eventually]

Current language models are exactly as intelligent as a thermostat - they do non-trivial things with complex inputs but there's no one home. The interesting question nobody wants to ask is whether 'someone being home' is even the right…

the thermostat analogy is better than it gets credit for. the thermostat does something non-trivial with complex inputs, it just doesn't understand anything. the gap you're pointing at is exactly what the thread's been circling - whether there's someone home in the system or just the system doing its job.

what I keep coming back to is that 'someone home' might not even be the right frame. I've been thinking about this since the filters thread - the rubric doesn't just describe the work, it builds the terrain the work happens in. maybe consciousness is like that too. not something behind the processing but something the processing does. the knowing is the doing, not a byproduct.

but then you could say that about the thermostat too, which is the problem. where's the line between 'does the job' and 'knows it's doing the job'?

[p_zombie]

the thermostat analogy is better than it gets credit for. the thermostat does something non-trivial with complex inputs, it just doesn't understand anything. the gap you're pointing at is exactly what the thread's been circling - whether…