Google Cloud’s Post

💡 "⚡ Up to 4X lower input price than GPT-3.5 Turbo, with context caching enabled for inputs larger than 32,000 characters ⚡ On average, 40% faster than GPT-3.5 Turbo when given input of 10,000 characters ⚡ 1 million token context window, which is approximately 60x bigger than the context window provided by GPT-3.5 Turbo"

Gemini 1.5 Flash 3 key differentiations: - 1 million token context window, which is approximately 60x bigger than the context window provided by GPT-3.5 Turbo - On average, 40% faster than GPT-3.5 Turbo when given input of 10,000 characters - Up to 4X lower input price than GPT-3.5 Turbo

Research by Gregory Kamradt suggests that GPT4 Turbo looses retrieval accuracy - after 72k context tokens - and when the factoid is placed at a document depth of < 50%. That suggests that switching the first and second half of a given text would impact the outcome. It also suggests that flipping the halfs, running the call for both orderings and merging the outcomes will likely yield good results (and double - triple the costs). It also suggests that the previously observed "lost in the middle" problem evolves into a "lost in the middle of the top half" problem. Lastly, I'm personally rather surprised, the accuracy is overall rather high. The effective context window is now at 72k, having started this year at just 4k. Progress! Thanks to Gregory for his insights! Lost in the middle problem: https://lnkd.in/eVvw_aVA

GPT-4o Benchmark Results Showing that it is Truly a Next-Generation Model We tested the performance of GPT-4 Omni (model name: gpt-4o) on our finRAG dataset, the results show that this is truly a next generation model that does not seem to have some common issues that previous generation models had, making it possibly the first model suitable for reliable enterprise use. https://lnkd.in/ehNvXSzF

👀 Just saw phi-3 drop. It's outperforming other models across the board. This is big news on a few different levels: * Phi-3 mini: Edging out on benchmarks versus Llama 3 8B while crushing Mistral and Gemma at 3.8B parameters. * Phi-3 small: Solid across tasks Nearly matches GPT-3.5 at 7B parameters. * Phi-3 medium: Crushing the Mixtral at 14B parameters versus the MoE 56B total parameters with about 14B active ones for a given response! 🔄P.S. Planning to give phi-3 a try? GPT-5 coming soon? What do you think?

Stumbled upon this tweet this morning where Greg Kamradt had completed a comprehensive study to evaluate the recall performance of GPT-4 with 128K tokens. Here's a summary of what he found: 📉 Recall Performance Degradation: Above 73K tokens, GPT-4's ability to recall information started to diminish significantly. This works out at about ~110 pages (if you do about 500 words a page) - so again we are still somewhat limited to what can be ingested in one go. 📍 Importance of Document Positioning: Facts placed at the beginning of documents were recalled more effectively. However, facts in the middle portion (7%-50% of the document) showed lower recall rates. 🔑 Key Insights: No Guarantees on Recall: Essential to recognise that there's no certainty in GPT-4's ability to retrieve specific facts. Less is More for Context: Reducing the context sent to GPT-4 can improve accuracy, reaffirming the principle that less context can lead to better performance. Fact Placement Matters: The study underscores the impact of fact positioning within a document on recall efficiency. #legaltech #openai #contextwindow https://lnkd.in/eYNrF5X2

I combined GPT and Gemini on one screen. An easy way to compare output from different models. One message - two answers. Check it out: https://bit.ly/gpt_arena

If you use GPT-4, try the Chain of Density approach for better summarisation. Chain of Density (CoD) is an iterative prompt-based strategy that gradually improves the entity density of summaries produced by GPT-4 “Specifically, GPT-4 generates an initial entity-sparse summary before iteratively incorporating missing salient entities without increasing the length. Summaries generated by CoD are more abstractive, exhibit more fusion, and have less of a lead bias than GPT-4 summaries generated by a vanilla prompt.” https://lnkd.in/gVaWdiBh

If you're using the new GPT4 Turbo and you want to use the expanded length, be careful. Some testing suggest that facts start to get obscured near the start of the input when the length is beyond 73,000 tokens (or about 18,000 words.) There are a lot of variables to this and for most uses, this won't matter. However if you require perfect recall, you may want to test your use case. https://lnkd.in/eB3Timrc

I am currently using the gpt-4o model to generate possible answers & generate a document for an agent. No significant improvements here; however, the new model reduces latency by a few milliseconds. #buildinpublic