The Pragmatic Engineer

What tools do devs use for AI-assisted development? In our survey (conducted May 2024) ~70% of respondents mentioned either ChatGPT, or GitHub Copilot for software development-related work. All other tools were mentioned less, combined. Two other interesting findings from this survey: 1. Tooling preference seems to have changed very little in a year! ChatGPT and GitHub Copilot still dominate. (At the same time, we would not be surprised to see Claude to quickly gain ground, thanks to Claude 3.5 Sonnet having standout coding abilities.) 2. Workflows also have not changed that much, versus 12 months ago. AI agents for dev tasks are a new one, though! Read more details in today's deepdive: https://lnkd.in/eMXcs36k

Are relational databases here to stay as good fits for AI? A recent paper by two database experts suggests this will be the case (and already is.) With the rise of large language models (LLMs,) vector database solutions are more relevant than before because embeddings are at the core of LLMs. An embedding is a vector that is a multi-dimensional representation of a token (basically a piece of text, image, or similar.) Operations like retrieval augmented generation (RAG) calculate the embedding of the input, and try to find previously stored embeddings (chunks of texts) in a vector database, so these vector databases are now very useful. We previously covered RAGs in more detail. Lots of venture capital has flowed into vector database startups, with Pinecone one of the best-known cases, along with Chroma, Weaviate, and others. The paper “What goes around comes around… and around,” was authored by Michael Stonebraker — who is a computer scientist (and currently a professor at MIT) with decades of experience in database systems: the cofounder of Ingres, Vertica and VoltDB, and the recipient of the 2014 Turing Award — and Andrew Pavlo — the cofounder of AI-powered SQL optimization startup, Ottertune, and an associate professor at Carnegie Mellon university. They analyzed the evolution of database management systems, and interestingly concluded that relational database management systems add vector support surprisingly rapidly, and that vector database systems must become more relational in order to stay competitive: “After LLMs became “mainstream” with ChatGPT in late 2022, it took less than one year for several RDBMSs to add their own vector search extensions. In 2023, many of the major RDBMSs added vector indexes, including Oracle, SingleStore, Rockset, and Clickhouse. There are two likely explanations for the quick proliferation of vector indexes. The first is that similarity search via embeddings is such a compelling use case that every DBMS vendor rushed out their version and announced it immediately. The second is that the engineering effort to introduce a new index data structure is small enough that it did not take that much work for the DBMS vendors to add vector search. Most of them did not write their vector index from scratch and instead integrated an open-source library (e.g., pgVector, DiskANN, FAISS). We anticipate that vector DBMSs will undergo the same evolution as document DBMSs by adding features to become more relational-like (e.g., SQL, transactions, extensibility). Meanwhile, relational incumbents will have added vector indexes to their already long list of features and moved on to the next emerging trend.” The paper is worth reading, and makes the compelling case backed by data that relational databases are here to stay. The full paper: https://lnkd.in/eHc3c8Pk This was an expert from yesterday's The Pulse. The full issue: https://lnkd.in/erXvTKj5

A win-win: Volkswagen to get good software, and Rivian to get much-needed cash? Rivian is a popular EV maker in the US, producing the Rivian R1T; the first electric pickup truck. But: While the vehicle is known for its impressive range of around 400 miles / 650km, off-road capability, and innovative features - Rivian seemed to be in financial trouble, following a successful 2021 IPO. As we covered in April (https://lnkd.in/gfit5kcJ), the company is at risk of running out of money in a few years time. The company’s current market cap reflects the risk of a potential bankruptcy down the road: in 2021 RIvian was valued at $150B: but a week ago, it was down to $10B. This week, things changed for the better. Volkswagen announced a joint venture with Rivian, in which the German automaker provides $5B of capital, and in return, Rivian’s software powers Volkswagen cars. This partnership seems like an ideal solution to both company’s problems. Rivian has great vehicles and software, but needs money. Volkswagen has plenty of cash, but its software is known to be terrible and buggy (e.g. https://lnkd.in/ddpnhe44), to the point that it costs the company customers. I previously test drove a Volkswagen EV, the ID3 model, and its unresponsive software – alongwith reviews stating the same – was enough to not spend more time evaluating it. This must be what a “win-win” looks like! Congrats to Rivian; and hopefully Volkswagen’s customers also benefit from this venture. --- This was one of many topics from today's The Pulse issue in The Pragmatic Engineer. Subscribers can read the full issue here: https://lnkd.in/gDVSg64b

Today's free issue is the first-of-its kind of survey on GenZ software engineers... from GenZ devs themselves! Including how they see their "older" colleagues. Read the full issue for interesting (and sometimes amusing) observations. It might change how you think about the youngest folks to enter the software engineering workforce: https://lnkd.in/eD2aZeM8

How do AI software engineering agents work? No better place to start than "at the source:" the state-of-the-art open source AI coding agent: SWE-bench has quietly become the industry standard AI coding agent benchmark. The team behind SWE-bench built SWE-agent (at release, state-of-the-art open source AI agent) and with Ofir Press (a member of the team building SWE-bench and SWE-agent), we dive deep on how it works: Read the deepdive here: https://lnkd.in/e3Ha39Rz

Oxide is a very ambitious startup: they are building their own "cloud computer" hardware, complete with their software stack. The team shared how they built two separate servers from scratch (and why) which powers their "cloud computer." Today's issue is a deepdive, with input from CTO Bryan Cantrill and other Oxide team members. It's also a peek behind the scenes on what it takes to build a fully custom server, in a way that very few companies have ever done before (and you cannot buy those servers from Big Tech companies like Meta or Google!) Read it here: https://lnkd.in/eYv35SF6

How did Cloudflare not notice an additional 12M requests/second hitting their services? Cloudflare engineering director Benjamin Yule shared this interesting non-event (in the photo below). I have so many questions as 12.5 million requests per second is a huge load! A few thousand requests per second is usually considered high, and handling a load in the millions surely requires thousands of servers (if not more!) to be spun up to handle it. How much load a single server can handle depends on how long a request takes to handle, how much memory a request uses, and then calculating whether a request is limited in CPU or memory. Whichever resource (CPU capacity or memory) runs out first determines a server’s maximum load. You can push things pretty far with clever workarounds; it’s how the WhatsApp team served 2 million requests per second on a server with 24 cores and 100GB RAM, back in 2012! (Details here: https://lnkd.in/e6nUbzRN) In this case, Cloudflare soaked up the load by using Cloudflare Workers, a serverless application platform, which spun up 40,000 CPU cores (an average of 320 requests/second per CPU core.) The tweet (https://lnkd.in/eGxiCSD4) says the Cloudflare Workers team didn’t even notice the increase, which suggests they could be handling 50-100x more load on the service! If we assume a 100x load, that could mean operating 4 million CPU cores or more simultaneously, just for Workers. By comparison, we previously covered travel booking service Agoda operating a total of 300,000 physical cores and 600,000 virtual ones (we covered more on Agoda's private cloud: https://lnkd.in/ehD8m-22) It’s likely Cloudflare is operating more than 10x this number. Realistically, they probably have far far more than 10x! --- This was one out of several topic covered in this week's The Pulse. Subscribers can read the full issue here: https://lnkd.in/eUtSTVz4

Bluesky is a Twitter-alternative. It launched two years ago, and has already grown to an impressive 5.5 million registered users: with only ~10 developers building it. But how was it built? With Elin Nilsson we spent the last couple of weeks talking with Paul Frazee and Daniel Holmgren (two founding engineers at the startup) to get a sense of how this platform went from zero to where it is today. A few interesting notes: - The core of the protocol and app were built by 2-3 engineers over 9 months - Bluesky moved from AWS to on-prem for cost-saving and performance reasons - The team migrated from PostgreSQL to... SQLLite! For maintainability and performance reasons. And they are very happy with this move. Read the details in today's free deepdive: https://lnkd.in/ezqHQvND

Five myths about security engineering that software engineers should probably know: 1. Security is only the responsibility of security engineers. False! 2. Security through obscurity is sufficient. False! 3. More security measures makes software more secure. Not always! 4. Once secure, always secure. False! 5. Penetration testing by itself ensures security. False! For a deepdive into security engineering, see today’s issue by security engineer Nielet D'Mello. Read it here: https://lnkd.in/dJXvWsDN

It is now day 5 of an Italian bank’s systems being down, hard, following a system upgrade over the weekend. It’s hard to resist asking the question: Could the underlying reason be: “Spaghetti code”? On a more serious note, we cover more details and analysis in today’s bonus The Pulse issue. Read it here: https://lnkd.in/ey_raKG5