The new ChatGPT outperforms Claude. But I don't trust benchmarks. So I ran my own tests: test #1 → analyze a 42-page PDF I asked the 2 of them to read the entire PDF and summarize it. At first, claude did a great job but then completely hallucinated: It gave me 2 steps that I didn't ask for: > a list of 5 important facts > a fictional discussion with a student gpt-4o gave me: > an executive summary > a detailed summary of each chapter It's actually really good. And it was so fast. It feels like groq. Last time I asked gpt-4, I only got a short summary of the PDF. test #1: gpt-4o wins. test #2 → specific graph analysis: "Exposure to AI & to automation in UK" gpt-4o gave me: > a full description of the two panels > a specific percentile details > observations & conclusions And in a bullet-point format. Easy to read. What about claude? claude gave an analysis of the graph but: > shorter description > shorter observation & conclusion > and not in a bullet point format test #2: gpt-4o wins. test #3: making a graph. I wanted to know if gpt-4o can create a graph with the same insights. It cannot. → It gave me false data & makes up words. → It might only work for simple tasks. test #4: write an article I asked both of them to write an article based on the PDF and the graph: gpt-4o gave me a full article with: > titles for each part > call-to-action > SEO keywords claude gave me a shorter article: > less details > no SEO keywords test #4: gpt-4o wins again. Now, test #5: I asked for a Linkedin post. Both LLMs had to get inspired by the PDF. And both of them gave me: > posts that are too long > emojis & question marks I asked for a shorter one and gpt-4o did a better job. claude still didn't. test #5: gpt-4o wins again. Now can they write a tiktok script? And the answer is no. Their generations are awful. gpt-4o scripts starts with "hey everyone" and claude: "hey there". You're not going anywhere with this. So I took a video from Mr Beast as a template and asked for a new script: → claude doesn't have internet access. → gpt-4o gave me a better hook: → "AI is shaking up the job market". But I'm not sure that it actually got the script from the video. It's still much better with a one-shot example. Final conclusion on all these tests: gpt-4o did a better job than claude on most of the tests. It's insanely fast & good at copywriting. But it needs improvements, and a whole lot of one-shot examples to be better. It still lies and hallucinates on some tests. Last thing before I leave this place: Most people copy -paste prompts from the internet. 1) they don't know how to prompt 2) they can't create prompt for their own needs That's why I made the simplest prompt engineering course on how to master ChatGPT in 2024. I sold my masterclass to 3,229+ early-adopters. Download it at rubenhassid.ai.
Sharing is caring ❤️ Press the button "repost". I will create better content. To be your own human-GPT :)
Very interesting. What about copy.ai? On some subjects I believe he can be even better that chatGPT
Hmmmm, I’m a Claude fan.
Hey Ruben, what’s your opinion about GPT4 and GPT4o? It’s an interesting content for a video, huh?
I must admit the ease at which GPT4o follows instructions compared to GPT4 is hugely improved, far less hallucination, it also sounds a lot more human
I am finding GPT-4o slowing down with days. Do you see speed same as day1 or dropping? Ruben Hassid
Have you tested the new video and voice integrations that they showed? Have they started rolling those out yet Ruben Hassid?
Interesting tests!
I’ve been using it. It’s really good.
Master AI before it masters you.
1mo📌 What's the AI Hub? (please like this to keep it at the top) I reached 100+ million people sharing my prompt techniques on the internet. I ended up being an international speaker on prompt engineering for Google (Tel Aviv), LinkedSummit (Denmark) or the B2B Summit (Paris). Today, I share everything I know in one masterclass. Visit @ rubenhassid.ai ### How to prompt ChatGPT 9 videos to know how to prompt. → The basics → Zero-Shot vs. Few-Shot → Tree of Thought Prompting → Multimodality → Chain of Thoughts → Advanced Prompt Engineering → Meta Prompting → Create A Prompt With Me #1 → Create A Prompt With Me #2 ### The copy-paste library My personal Notion prompt library. → Marketing prompts → Writing prompts → Business prompts → Sales prompts ### Bonus All of my Cheat Sheets & resources. My top AI tool list, per category. My favorite & personal GPT agents. Weekly updates for free. Instead of collecting 10,000+ prompts, I made the right ones. Instead of reading academic papers, watch one masterclass in 1h. To master AI, before it masters you.