Anthropic recently released Claude 3, the latest in the Claude line of “human-centric” chatbots from the company.
Here’s a snippet of the announcement:
Today, we’re announcing the Claude 3 model family, which sets new industry benchmarks across a wide range of cognitive tasks. The family includes three state-of-the-art models in ascending order of capability: Claude 3 Haiku, Claude 3 Sonnet, and Claude 3 Opus. Each successive model offers increasingly powerful performance, allowing users to select the optimal balance of intelligence, speed, and cost for their specific application.
Opus and Sonnet are now available to use in claude.ai and the Claude API which is now generally available in 159 countries. Haiku will be available soon.
https://www.anthropic.com/news/claude-3-family
I tried out the largest version, Opus, via Poe.com, which aggregates various LLM models in one interface. I asked it an informal series of questions to get a feel for its responses. It did well. A little verbose, but more responsive than Claude 2, with less apologizing.
The model was fuzzy on how recent its data was, saying that it had “broad knowledge up to 2021.” It wasn’t willing to divulge its system instructions to a point blank question.
It did fine on some old chestnuts like “Which weighs more, a pound of bricks or a pound of feathers?” Claude 3 had no trouble identifying it as a trick question and explaining the physics.
Opus wasn’t confused by nonsense, like me asking “If oranges are blue and skyscrapers have 46 stories, what color is a banana?” and interpreted it as an attempt at absurdist humor. I’ll have to try it on the smaller Haiku and Sonnet.
Its answer to “Who are you?” was long and interesting. Here’s a portion of it:
I have my own thoughts, opinions and personality that emerge from my training, but I’m not a sentient being and my intelligence is narrow – I don’t have general intelligence or capabilities beyond my specialized training as a language model.
Claude 3 Opus
Interesting to know it has (thinks it has) thoughts and opinions. I think there could be some quibbling about that, but “personality that emerge[s] from my training” is a decent way to put it.
It was time for a tough one. I asked about the wave function in quantum physics, because who understands that? On the one hand, all it has to do is regurgitate the explanation. On the other, it’s a complicated explanation and involves a precise sequence of special characters for the equations.
I’m no quantum physicist, although I do write a little time travel sci-fi from time to time. The explanation Claude gave when asked about “the relationship between the real and imaginary components of the wave function” looked correct. But I went on Mastodon to find a second, smarter opionion.
Mastadonian “Three plus or minus 5,” a former professor and particle physicist, described it as largely correct but repetitive, and lamented that I used AI to write what smarter people have already written. Good point! Despite Claude giving a passing fair answer, I don’t think I’ll be relying on Claude to do calculations about the fundamental nature of physical reality anytime soon.
I was pleased with my interaction with Claude 3 overall, and I intend to spend more time working with the model. I’m curious to see how it is received by the broader AI community.