James Grafton

Want to know more about How Figma's Databases Team Lived to Tell the Scale? Still skeptical of why we built horizontal sharding on RDS instead of <insert-cooler-database-technology>? Wondering if you should copy what we did? Check out out my new podcast with Software Engineering Daily as we dive deep on spicy stories that didn't make the blog post from surviving my first Figma Config conference to debating which database technology comes out on top. https://lnkd.in/gFfh7C-A

150

9 Comments

Stop (over-) engineering your prompts Generative AI is really cool. I use several tools in this category every day for writing suggestions, image generation and more. The tools can do amazing things, especially if given the right prompt. But there's a limit to how much prompt engineering one should do, especially if you're trying to generate text for mainstream consumption. This topic is already a bit of a bandwagon but this topic has been bugging me for about a week so I'm going to write about it. Generative AI relies on a class of machine learning applications broadly described as deep learning. Deep learning boils down to feeding a neural network a ton of content that has been classified into groups and then iterating over the content to allow the network to identify patterns in the content. Basically the network builds up statistical relationships between the words or other elements in the content. So it knows, for example, that if a piece of content is a recipe the word tomato is often followed by the words sauce or paste and very rarely by the word stem. But in a document about farming tomato is often followed by the word seed, starter or stem. The deep learning model has developed an elaborate matrix of statistical relationships between words in different types of content. If you described CHatGPT as an elaborate word guesser you wouldn't be wrong, just over-generalizing. Large language models are trained on billions of documents. So how, you might ask, are the documents classified in order to feed them into the neural network? Did someone categorize the web by hand? Of course not. The developers of the LLM leveraged Natural Language Processing (another deep learning application, btw) and other tools to categorize documents before they got fed into the model. Okay, but, again, what does that have to do with prompt engineering? When you ask an LLM to generate a piece of content, say a blog post about cycling in Northern California, you are specifying a category of content. If you later asked for an "image in Rubinesque style of a man and a woman engaged in flower picking while overhead rages a space battle between starships populated by photorealistic blob-like aliens carrying professional immersion blenders done in the style of pointillism" you have specified many categories. You're essentially asking the LLM to extract elements of the content that it was trained on that match your descriptors and which have mathematical relationships to other adjacent content elements and to include those in your generated image or blog post. And ... prompt engineering? The categories in the content were identified with Natural Language processing. Your prompt should be in natural language, not some over-specified, highly engineered artifical style. Don't over-think it, work with the model, don't try to control it too much. That's it.

2

Excited to share that I have migrated my blog to Github pages using the Jekflix theme from thiagorossener/jekflix-template! It's been a breeze publishing new content and I highly recommend checking it out. Head over to itmusings.com to take a look at some of my old but still relevant blog posts. #githubpages #blogging #webdesign . By the way, this post is written using Linked In AI which is pretty cool :) By the way if you have cached my previous blog in your browser you may get a 404. This is because I had used a 302 redirect before in the old blog. You may want to check this article to remove the 302 from your browser cache. https://lnkd.in/g2-QzUeq

37

3 Comments

Electronic Arts (EA) has recently sparked significant debate by announcing that generative AI could impact about 60% of its video game development processes. It's no coincidence that this statement follows close on the heels of EA's decision to lay off hundreds of employees. While some competitors have pledged to avoid using generative AI, it appears that EA is not only embracing it but also leveraging it as part of broader cost-cutting strategies. As someone who works in software engineering, this feels like a defining moment for generative AI being put to true industry tests. And although I respect the commitment of studios that choose not to adopt generative AI, the technology's rapid advancement suggests they may be on the back foot. Today’s GPT models are just the starting point. They will only become more sophisticated and indistinguishable from human-generated content as time goes on. The real game-changer will be how AI integration can accelerate development cycles. Imagine a world AI tools becoming as essential to game developers as spell checkers are to writers. The studios that harness these tools effectively will likely lead the pack and deliver new gaming experiences much faster than those that don't. I predict that despite public declarations against AI, many studios will gradually and discreetly integrate generative AI into their processes. To me it seems inevitable that generative AI will become a standard across the industry. What are your thoughts? Are we witnessing a new era in software development for gaming?

12

5 Comments

🌶🌶🌶 Hot take… Old problem: if we put more devs on the project, we can deliver it sooner. Same thinking, new manifestation: we can hire more overseas devs for less total money to produce the same result as our US staff. Flaw in that thinking: product development is not an assembly line. Devs are not “interchangeable”. Software is as much an art as it is science. It’s a team sport. The people on your team matters. #tech #staffing #management #hiring #jobs #layoffs #software #startup #teamleadership

25

6 Comments

Technical interviews were never easy to design, but adding LLMs to the process has made it harder. How are folks thinking about using copilot/ChatGPT in tech interviews? Here's how we're thinking about it.

22

7 Comments

Punching Above Their Weight: How Autoregressive Models Outperform Diffusion in Image Generation. Check out my most recent post - https://lnkd.in/evS4bZDk

10

Substrate is live! After nearly a year of development, really honing the developer experience and performance tuning a very unique runtime environment, we're finally releasing what we feel is by far the best way to run multi-inference workloads (e.g. agents, multimodal text, json, image, audio, embedding pipelines, etc). Substrate lets you use a ergonomic SDK (we really obsessed over the abstractions and typeaheads) to describe precise and complex relationships between AI tasks. And—importantly—it's connected to a backend specifically designed to run these graph workloads as efficiently as possible (optimized models, auto batching, insanely fast inter-node RPCs). Given the current state of "AI" models, we really believe that this pattern is the most effective for building capable, reliable, legible, extensible features with AI that go far beyond chatbots. We need both heuristic models like LLMs as well as normal symbolic computation, and a flexible/scalable way to coordinate these two modalities. We've already seen organizations like Substack building some super interesting pipelines that they can easily understand and iterate on, that just work at scale. It's still early in a much more ambitious vision for us, but we think this V1 is not only ready but already a meaningful improvement to many implementations. Reflecting on the last year, I feel so much gratitude for the team that I'm working with: Ben Guo, Liam Griffiths, Rishabh Parikh, 💭 Kyle Pitzen, Chris S., (& Hanwen Wu soon!). With this launch, we're announcing an $8M Series Seed led by Lightspeed (Nnamdi Iregbulem+Guru Chahal) with participation from many others. Feeling a ton of appreciation for the folks that took a bet on us or have lent their time and advice: South Park Commons (Ruchi Sanghvi+Aditya Agarwal), Craft Ventures, Red Swan Ventures (Andy Dunn), Lorimer Ventures, Chris Best, Jairaj Sethi, iqram and the immigrant groove, Andrew Kortina, Guillermo Rauch, Immad Akhund, Sam Lessin, William Gaybrick, Gokul Rajaram, Jana Messerschmidt, Shreyas Doshi, Matthew Hartman, Mike Cohen, Sabrina Hahn, Anne Lee Skates, Shreyans Bhansali, Julian Connor, Namrata Patel + many many others. https://lnkd.in/eHq8KUkA

127

9 Comments

In this week's issue of "Large Scale Web Apps", we share a soundbite from Jem Young (Engineering Manager at Netflix) on how him and his team at Netflix have cultivated a successful testing environment. A few months ago, we asked friends and folks in the industry to share their perspectives on building large-scale web apps with React. Jem Young (Engineering Manager - Web Platform & UI Data Platform at Netflix) was kind enough to share some valuable insights into how testing is integrated into his team’s development culture. Some key takeaways that we love from Jem’s soundbite: 👩‍💻 Testing is not just about the code; it’s about the culture as well. Establishing a culture where testing is viewed as an essential, integrated practice within engineering not only prepares teams to preemptively catch errors but also reinforces a commitment to quality and reliability. This cultural shift ensures that testing becomes a habit rather than an afterthought. 🧪 You need to have a cohesive testing toolset (and strategy). In the modern JavaScript ecosystem, tools change often, which sometimes makes it challenging to keep up with the latest and most effective methods. Despite this, the benefits of standardizing and sticking with a cohesive testing toolset are substantial. A unified set of tools across a development team reduces fragmentation and streamlines both the learning process and the application maintenance. Read the new issue here — https://lnkd.in/gm6VW-pt #programming #javascript #reactjs #testing #developers

34

1 Comment

The music industry is on the cusp of being disrupted by AI generated content. Broadly speaking, there are two types of apps we’re seeing so far: Prompt-generated music like Suno and Udio: 1) "Write a 90’s rap diss track about OpenAI training LLMs on private data” 2) ??? profit ??? More technical music production software like Sonauto (YC W24) that seeks to give lower-level control over the generated music: 1) Upload your own recording of a guitar 2) Prompt “add some drums to fit this vibe” 3) “Ok please tweak it to put it in the key of C major” I’d say the main thesis I have around AI music is it’s going to be a social-first experience, for friends to share / create together, not a juiced-up Ableton. But I haven’t seen much innovation on that front. Where’s the silly app where I can communicate with my friends exclusively through rap battles? Or write heartfelt music birthday notes to my family?

16

1 Comment

Have been brewing up abstractions on Langgraph(from LangChain)! What if there existed a nifty SDK, that allowed you to invoke any agentic design pattern? Why is it that I need to understand "Reflection", why is it that I need to understand "Hierarchical agents"? Possible to improve dev experience, improve the robustness of these agents, and not compromise on configurability? Spun up a quick Loom to braindump this, anyone thinking along these lines?

25

7 Comments

News broke today that Stability AI is laying off 10% of its workforce. Mismanagement by their former CEO is a large contributing factor, but it's not the only one. The series of events that led to these issues in the company are not as cut-and-dried as they appear. Stability AI has had a major disadvantage in the generative AI world from the start. Training models is very computationally intensive. It takes lots of very expensive hardware working for a long time to create a single model. GPUs, the same kind of hardware that is used to render graphics in a video game, are the most efficient way to train these models. OpenAI CEO Sam Altman stated that it took over $100MM to train GPT-4, the latest and most advanced of OpenAI’s models. On Medium, Danielle Franca says that OpenAI used 25,000 Nvidia A100 GPUs for 90 to 100 days in order to train GPT-4. Microsoft lists the A100 hardware as $13.63/hour if you lease the GPU for 3 years. See the breakdown: 90 days * 24 hours/day * $13.63/hr * 25000 units = $736MM That equates to $736MM for hardware alone. Costs for engineers and other things aren’t even factored into this number. But does "over" mean $700M. It turns out that OpenAI and two other generative AI companies get special pricing because of who they invested in them. Microsoft, Amazon and Google are the three largest cloud computing providers. If you need to train a model at this scale you will use one, if not all, of these companies. But, as it so happens, each of these companies has their own vested interest in generative AI. Microsoft has invested $13BN into OpenAI, part of which is huge amounts of compute time. Amazon has invested up to $4BN in Anthropic, the makers of Claude. Google funds its own internal project known as Bard/Gemini. This gives these three AI companies a de facto advantage that is nearly impossible to overcome because they own the hardware. Or, to put it a different way, a generative AI company that doesn’t have a special deal with one of these players is going to have to raise a lot of money. A lot. It almost has the feel of one of these three companies standing in front of your house and saying “That’s a nice house you got there. Be a shame if anything happened to it…” Stability AI raised $101M in venture capital in October of 2022. However, according to the Register, compute costs alone for 2023 were $100MM spread primarily across Amazon and Google. Cash became so tight that Stability was millions of dollars short on paying Amazon in July and may not have intended to pay anything in August. Unfortunately for Stability AI, the game was rigged and I suspect that there wasn’t a lot that could have done to fix it. Without literal billions in cash, Stability AI cannot create a product on the scale of Chat GPT-3, much less the fourth iteration. Stability AI could have run a much tighter ship, but their boat is a dinghy in the Atlantic compared to the aircraft carriers the others are driving.

17

3 Comments

"Your main job is to make your product win." I think this should apply to any product that your company makes. I also like what he says following this statement about what is NOT your main job. I will summarize here: "Your main job is not to make your manager happy, it is not to make execs happy." Also, note, this main job doesn't only apply to PMs, but also to senior Engineering Managers, Tech Leads, and Designers. #makeyourproductwin #leadership

5

1 Comment

I've been testing GPT-4o since its recent release. While I haven't really had a use for the multi-modal (images, audio, and text) capabilities yet, the improvement in speed of response is noticeable. Not just noticeable from a spec sheet perspective either—it's noticeable in how the product works in real use cases. I've been thinking about why that is, and here's my theory: there's a breakpoint somewhere based on latency and response speed. Above this breakpoint, interactions feel sluggish and disruptive. Below it, they become seamless and effortless, akin to a natural conversation. Previously, I posted about how Llama and Gemini models seemed noticeably more usable than GPT-4, with my preference being Llama whenever possible. This new experience with GPT-4o further emphasizes the importance of optimizing both latency and throughput in AI systems. This chart from ArtificialAnalysis.ai shows that GPT-4o has a total response time of 1.5 seconds for 100 tokens, compared to GPT-4's 5.7 seconds. This speed difference isn't just a matter of numbers—it's fundamental to achieving a smooth and efficient user experience. When latency drops below a certain threshold, the interaction becomes fluid and engaging, making the AI more practical and user-friendly. It also makes its mistakes more tolerable. This improvement transforms how we use these models in real-time applications, making AI interactions faster and more intuitive. It underscores that achieving low latency and high throughput is no longer a nice-to-have but a critical requirement for practical AI deployment.

9

3 Comments

Slack adopted a "Fair and Transparent Pricing Model" That sadly is now over. --- Slack's stance was our product offers $X in value so you'll pay $Y per user, and only pay if that user is active that month (on a monthly plan). The idea was brilliant: we don't negotiate because our product is sold at a competitive price and we want to reduce complexity. --- Enter Salesforce acquisition. Salesforce is notorious for overcomplicated SKUs and pricing strategies. If you know how to negotiate with Salesforce you get an amazing contract. If you don't, you're going to overpay for the life of your contract (unless you grow and can trigger a renegotiation) They've decided to start applying this strategy to Slack, especially for the new AI SKUs. --- On one hand, it does mean you can now bundle your total Salesforce contract and get a potential discount. On the other hand, you now have another product that you need a Rubik's cube in order to decipher what you should be paying. --- I think we're in for a world where we'll continue to see add-on SKUs for Slack, which may push companies with a strong Microsoft footprint closer to Teams over Slack. What do you think? Do you prefer Slack's old pricing model or moving towards negotiation and bundles? #salesforce #businesssystems #slack

24

12 Comments

The reality of system design no one will tell you is: Companies and interviewers exist on a spectrum somewhere between opinionated and dogmatic: about tech choices, products, and you. Telling a system design interviewer from Amazon that you want to use SQL, is akin to telling a hiring manager at Apple you prefer Android. Who knew? It's obvious once it's spelled out, but only because we sometimes assume these big brand-name tech companies (along with their interviewers and interview processes) are faultless. This was an eye-opening clip, Victor Moreno and Kevin Landucci! But I like the full interview even more (link in the description) 🛎 Subscribe to ApplyPass for more content like this

22

2 Comments

I've noticed that Shravan Tickoo often says, "I've seen products that solve real problems and have great UI and UX, but still end up failing." This made me wonder why they fail despite they are excellent . After discussing with Dev, Ashu, and other product managers, it became clear that products without a strong network effect are more likely to fail. ➡️ What is a Network? Andrew Chen introduces the concept of an "atomic network" – the smallest stable network from which all other networks can be built. For example, Zoom started with just two participants in video conferencing. ➡️ Do Networks Help in Business Building? Absolutely! Building a network is crucial for network-based businesses. ➡️ How to Prevent Products or Services from Failing The key to saving a product or service from failure is to build and leverage strong network effects. Here are some strategies by Andrew Chen: - Identify and Build the Atomic Network ▫️ Focus on the Core Experience: Start by creating a compelling experience for the smallest viable group. This ensures significant value, crucial for retention and organic growth. 𝘌𝘹𝘢𝘮𝘱𝘭𝘦: Facebook began exclusively for Harvard students, creating a close-knit community that saw immediate value. ▫️ Ensure Stability and Self-Sustainability: The atomic network must be stable and self-sustaining, motivating users to stay engaged without constant external input. - Expand Strategically ▫️ Create Repeatable Strategies: Once the atomic network is established, replicate this success to build additional networks, gradually expanding the market. 𝘌𝘹𝘢𝘮𝘱𝘭𝘦: Airbnb targeted specific events like conferences, building stable networks before expanding globally. ▫️ Leverage Network Effects: Use the growing user base to attract more users. - Overcome Challenges ▫️ Avoid Negative Network Effects: Prevent issues like overcrowding or spam that can degrade the user experience. 𝘌𝘹𝘢𝘮𝘱𝘭𝘦: Twitter has struggled with spam and harassment issues as it has grown, which negatively affects user experience. Addressing these issues is crucial to maintaining a healthy network. ▫️ Solve the Chicken-and-Egg Problem: Attract initial users on both sides of a marketplace or network with incentives or innovative marketing. 𝘌𝘹𝘢𝘮𝘱𝘭𝘦: Uber initially offered discounts and incentives to both drivers and riders to build up both sides of their marketplace, solving the chicken-and-egg problem effectively. - Focus on Retention and Engagement ▫️ User Retention: Retain users with a valuable service. 𝘌𝘹𝘢𝘮𝘱𝘭𝘦: WhatsApp offers reliable messaging. ▫️ Engagement Metrics: Track metrics like DAU and MAU. In wrapping up, These strategies offer a roadmap for businesses to fortify their standing in the market. By capitalizing on network effects, companies can transform challenges into opportunities for growth and resilience.

39

11 Comments

This is a good piece by Andrew Chen. It's so easy to think that the world needs your solution and is waiting for it, to realise you're building something nobody wants or is willing to pay for with their time or money. This is maybe the most important lesson you learn, and it's hard to get right even if it's not your first attempt: > "Dual insights needed: (1) an insight about customers that gives them product/market fit (2) an insight about distribution that creates traction." -- so, what would you do to ensure your entire team works together to solve it? This is not a marketing problem, a sales problem, or a product problem. It's everyone's problem.

1