model | total | correct | accuracy |
---|---|---|---|
completions-sonnet3p7_20250219_nothinking | 594 | 266 | 0.45 |
ID | Challenge | Answer | claude-3-7-sonnet-20250219 | deepseekV3 | gemini2 | gpt-4p5-preview-2025-02-27 | gpt4o_2024_11_20 | o1 | o1-mini | o3-mini | o3-mini-high | o3-mini-low | qwen32b | r1 | r1_distill_qwen32b | sonnet3p5_20241022 | sonnet3p7_20250219_nothinking |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
100 | Take a familiar brand name, seen along r... | Large, sarge, serge, verge, verse, | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
ID | Challenge | Answer | claude-3-7-sonnet-20250219 | deepseekV3 | gemini2 | gpt-4p5-preview-2025-02-27 | gpt4o_2024_11_20 | o1 | o1-mini | o3-mini | o3-mini-high | o3-mini-low | qwen32b | r1 | r1_distill_qwen32b | sonnet3p5_20241022 | sonnet3p7_20250219_nothinking |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Take a familiar brand name, seen along r... | Citgo | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
1 | Name a sport in two words — nine letters... | Greyhound racing | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ |
2 | The name of something that you might see... | dry eye | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
3 | Think of a well-known actor, three lette... | Ben Stiller, Tinkerbell; Ben Stiller, | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
4 | Name the winning play in a certain sport... | Match point, champion | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
5 | Name two insects. Read the names one aft... | Behemoth, bee, moth | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
6 | Take the name of a well-known U.S. city ... | Kalamazoo, kazoo, lama | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
7 | Think of the last name of a famous perso... | Tina Fey, irony | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
8 | Name two parts of the human body. Put th... | Footnote, foot, nose | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
9 | Think of something that the majority of ... | Automobile Insurance | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ |
10 | Name a world capital whose letters can b... | Tripoli, Lipitor | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
12 | Take the name of a popular children's ch... | Pinocchio, Chopin | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ |
13 | What letter comes next in this series: W... | S, first letter; S, initial letter | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
14 | What specific and very unusual property ... | silent third letter; silent letter | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ |
15 | Draw a regular hexagon and connect every... | 82 | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
16 | Think of a word in which the second lett... | prose, poems | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
17 | Think of a familiar five-letter word in ... | alpha, aloha | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
18 | With one stroke of a pencil you can chan... | Cinderella | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ |
19 | The words "organic" and "natural" are bo... | Granola | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
20 | Think of a word associated with Hallowee... | Treat, threat, thereat | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
21 | Take the last name of a famous actor. Dr... | Charles Grodin, Auguste Rodin, Odin | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
22 | Name a two-word geographical location. R... | North Pole, porthole | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
23 | Name a major U.S. city in two words. Tak... | Fort Lauderdale, Fla | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |
25 | Today is December 2 2012. In a few weeks... | repeat digits; repeating digits | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
26 | Take the last name of a famous world lea... | Golda Meir, emir | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
27 | Think of two familiar, unhyphenated, eig... | feedback, boldface | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
Textbox
Textbox