Grigory Cherny

Check out the latest in our "Debuggability" blog series! This blog is authored by Kevin Downey Jason Johl and me and talks about how we built a Debuggability Paved Road at Intuit using Ephemeral Containers and Argo Workflows. We'd love to hear your thoughts on the topic. Also, what tools do you build as-a-platform to help debug service, compute and network issues in Kubernetes at scale? #intuit #kubernetes #debug

23

Hey everyone, I am an investor and advisor to Katanemo.... They are reinventing the infrastructure primitives for GenAI and launched Arch - https://lnkd.in/gCrbbkEV - an intelligent prompt gateway. Give the project a look on Github, and star the project if it resonates

88

3 Comments

With the announcement of S3-native-streams (Freight clusters), here is a commentary on Confluent strategy regarding object storage, streaming and an open data architecture. https://lnkd.in/dTg6DKVy

49

6 Comments

sharpening writing skill is one of killer skill!

2

Can KV cache optimization go beyond just reducing memory footprint? Our latest work, SwiftKV, does exactly that—cutting prefill computation by half and significantly accelerating enterprise inference workloads! We’ve open-sourced the SwiftKV-optimized model checkpoints for the Llama 3 family and vLLM implementations, enabling efficient inference and making it easier than ever to integrate these advancements into your applications. - Tech blog: https://lnkd.in/gZbPBum3 - Paper: https://lnkd.in/g__9NpBd We hope SwiftKV inspires new possibilities for enterprise LLM efficiency for you!

460

5 Comments

Windows App SDK 1.6 is just released! I'm mostly excited about Native ahead-of-time (AOT) support - where we see 50% reduction in start time and ~8x reduction in package size! See more details below and join WinUI Community Call next week

18

What does it feel like to be an engineering manager at Amazon: Since the RTO announcement it became popular to write shit about Amazon, but it's still one of the best companies around the world. A few weeks ago, Gilad Naor shared with the 'Leading Developers' readers 3 lessons from his time as an engineering manager at Amazon. Here is the first one, read the full article here: https://lnkd.in/dHBxv2Ha 𝟭. 𝗦𝗼𝗹𝘃𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺𝘀 𝘄𝗶𝘁𝗵 𝗺𝗲𝗰𝗵𝗮𝗻𝗶𝘀𝗺𝘀 My first meeting at Amazon was baffling. I entered a room with about a dozen Software Development Managers (SDMs) and Product Managers (PMs). Everyone sat in silence and looked at their laptops or the printed papers they had before them. It was my second day at Amazon, and I picked up one of the extra sheets and started reading. After those first ten awkward minutes, everyone finished reading and the meeting began. Why do Amazon meetings always start this way? This is a specific example of how Amazon approaches problem-solving. It is the "Good Intentions Don't Matter" approach. In a famous internal video, Jeff Bezos talks about why good intentions are irrelevant. When things don't work, it does not matter what people wanted. All that matters is fixing the problems with mechanisms. For example, many busy managers come to meetings without first reading the prep material. They all intended to read the 6-Pager before the meeting. They just got sidetracked by the latest emergency. Scolding the managers won't solve the problem. A mechanism will. This is why all Amazon meetings start with a study period. The remaining meeting time is then spent on high-quality decision-making. Meta solves the same problem by creating cultural norms. I can't count the number of times that I heard a senior leader cut off the presenter: "Stop, assume that everyone here already read the slides. Let's get to the point." This is a culture-first approach. I have seen this work well in some teams, and not work at all in other teams. At Amazon, the solution is part of the company's "operating system". ------------------------- Anton here - by now everyone is familiar with the 6-pager concept, but that "operating system" thinking can be used almost anywhere. Think about something annoying that your developers do, and how can you solve it. For example, I remember being annoyed that people don’t update the Jira tickets. Finally, after requesting multiple times, now we just take 20 seconds from the standup to update them.

56

8 Comments

This is the upside of having a monorepo. Successes like this depend on being able to change code at scale safely and with clear understanding of performance impact. The changes themselves are done by a fairly small team, but the systems that enable us to continuously roll changes with critical company wide impact like this are key to make this possible.

38

9 Comments

Large Language Models (#LLMs) are evolving rapidly, and the demand for running them at scale is skyrocketing. #Ollama is stepping up to meet this need with its official container image on Docker Hub, excelling in local #deployments. However, there’s a catch for containerized environments: downloading #models requires the ollama serve command to be running within the #container. This extra step complicates automated deployments via #CI #CD #pipelines, making seamless model access challenging. To take your #model to production, it needs to be #container-ready—a feature Ollama has yet to fully support. The gap in #AIOps for smooth deployments gets blurred with the inference services available as an easy way to take it to production. Goal is to overcome obstacles and deliver solutions 😉 so here is to get you started on having your (O)#llama #production ready in a private #deployment from your #local to #cloud all in #oneshot . Enjoy Medium: https://lnkd.in/g9jqBQ8d dev.to: https://lnkd.in/gi_gGwam #AI #LLM #Ollama #llama3 #phi3 #mistral #S6overlay #AIOps #CI #CD #oneshot #container #docker #podmanEnjoy

11

Project update! My application tracking extension is now doing fuzzier matching and injecting new information into LinkedIn (and other sites) to keep me from doubling up on applications. Next up: local storage and bulk import/export operations...

12

2 Comments

While there are certainly individual variations in skill and experience when discussing modern software development the story is much more nuanced than pure individual effort.  * software development is a team sport -- individual performance is heavily impacted by the system in which they are asked to operate in.  * we pair skill and experience to this model of understanding performance by expecting people to be able to influence and adapt in direct proportion to their seniority, aka the more senior you are the more responsible you are for the outcomes of your work, the less senior you are the more your manager and management chain are responsible.  * humans want to succeed, we're social animals who derive purpose from demonstrating skill, and working collectively. All of which is to say there is no way you have over 1000 people underperforming at your company without senior leadership deeply implicated in.  If nearly 10% of your company is failing, versus just having the market move against you, you need to fire the CEO.

150

9 Comments

"Over 25% of all new code at Google is now generated by AI", according to CEO Sundar Pichai in today's earnings call. But is this shift aimed at speeding up the development of complex software, or simply cutting engineering costs? One potential indicator will be if Google reduces headcount by 25% while boosting revenue per employee. Here are some details behind this productivity metric in Google's own research page . Majority of this '25% code' is in the area of code completion, not in code creation. "LLM-based inline code completion is the most popular application of AI applied to software development..." Watch the video to understand further on this metric.

16

3 Comments

AI tools like Amazon Q Developer are evolving rapidly, with new features and capabilities reshaping how we approach software development. But it’s not all about exponential progress - there are nuanced stories and limitations worth exploring. Here are some of the upcoming features for Amazon Q Developer: - Gradle support, expected in Q1. - Java 21 compatibility, also slated for Q1. What’s more interesting is the overall approach Amazon Q Developer takes. Each action, like "transform" , is handled by a dedicated agent designed specifically for that task. General-purpose agents just don’t cut it. The limitations imposed on these agents are equally fascinating. To avoid shortcuts that might create bigger problems, agents are given a restricted toolset. For example, the transform agent doesn’t have bash access. Why? Because instead of carefully resolving issues with Java library upgrades, it might just delete files to "fix" things. And yet, even with months of development and fine-tuning, these specialized agents deliver only 10-50% time savings. The complexity of the codebase or the task itself heavily influences their effectiveness. The more complex the work, the more human oversight and manual intervention are required to ensure success (or even getting it to build). Now, before the AI bros jump in with "this will get exponentially better," let’s inject some realism. Progress doesn’t come for free. As we’ve seen with o3, scaling these agents up for more complex scenarios gets significantly more expensive. Handling something as intricate as a legacy migration is far more challenging than churning out the millionth iteration of the same app. The cost and time investment for building highly specialized agents don’t magically disappear. Looking ahead to 2025, we’ll undoubtedly see more agents emerge, but they’ll be increasingly niche and take significant effort to create to be useful. Even then, humans will remain an essential part of the equation, especially for tasks that go beyond repetitive, predictable workflows. AI can assist, but it’s not about to replace the need for expertise and adaptability in complex scenarios. Let’s celebrate the progress, but let’s also keep it grounded in reality.

3

I first joined Amazon in December of 2005. Even though I left for two years, my "Amaversary" still coincides with the end of the year and a natural moment for reflection. As I reached my 17 year mark this December, I also had the privilege of working with Amazon Careers to share some of my journey and thoughts about individual contributor technical leader and people manager career paths. In my ruminating, I put together far more content than could be used for a single article. So, I've decided to post here as a series of Q&As - in case others might find my observations, experiences, and perspectives interesting or useful. Below is the first of 6. 1. 𝘽𝙧𝙞𝙚𝙛𝙡𝙮 𝙨𝙝𝙖𝙧𝙚 𝙮𝙤𝙪𝙧 𝙘𝙖𝙧𝙚𝙚𝙧 𝙟𝙤𝙪𝙧𝙣𝙚𝙮; 𝙬𝙝𝙖𝙩 𝙢𝙤𝙩𝙞𝙫𝙖𝙩𝙚𝙙 𝙮𝙤𝙪 𝙩𝙤 𝙩𝙖𝙠𝙚 𝙤𝙣 𝙙𝙞𝙛𝙛𝙚𝙧𝙞𝙣𝙜 𝙘𝙖𝙧𝙚𝙚𝙧 𝙥𝙖𝙩𝙝𝙨, 𝙄𝘾 𝙫𝙨 𝙢𝙖𝙣𝙖𝙜𝙚𝙧? I started my career as a software engineer 25 years ago. I've worked for startups, enterprise companies, and 7 different organizations over 17 years and two tours at Amazon. I've written firmware for robotic camera mounts; built highly scalable distributed systems; designed restaurant marketing, ordering, logistics, and operations software; created automated test harnesses for mobile games; tested HD DVD encoding; helped launch grocery delivery to customers around the globe - just to name a few of the many cool things I've had the privilege to work on. I've been an entry-level software engineer, a technical lead of teams, a software manager on a couple of occasions, chief software architect reporting to a CTO, and (for the last few years) Amazon Principal Engineer directly supporting a VP with about 200 engineers within my remit. I first became a manager because I mistakenly thought it was the only way to continue growing as a leader, and in many ways I failed. I learned some very humbling lessons about leadership through that experience, which I used to be a better individual contributor technical leader on my path to Senior Engineer. My second time as manager was because my organization had a need that I could fill. I had grown enough as a leader that this time I was much more successful. I said yes to an opportunity that started with managing a small team through delivery of a complex and critical project. Once I grew a high performing team that delivered that initial project, I took on more scope and expanded into multiple teams. After a couple of years of success growing and investing in my teams so that we could deliver greater value to our customers and business, I decided to prioritize being close to my family. For me, at that time, it meant giving up my people management role and returning to the role of an individual contributor technical leader that supports a globally distributed organization. Even though I really enjoyed my time as a people manager, there is no better job for me than that of a Principal Engineer. #amazon #careerdevelopment #technicalleadership #peoplemanagement

91

3 Comments

Clear and rich information on RAG variations. I think this will be useful to most practitioners who have gone beyond the intro and demo workflows, and now need to consider how to refine their flows.

10

Waymo is probably both more expensive and longer waits on average than Uber or Lyft today, but they are close enough, and rapidly getting better. These markets will flip to fully autonomous when the tech is ready.

2

ModernBERT is an open-source embedding model which represents a significant leap in encoder-only transformer architecture merging: > Long context (8k tokens) > Local-global attention for efficiency > Up-to-date code and text training data (2 trillion tokens) > FlashAttention & Unpadding to push performance on consumer and server GPUs > High retrieval and classification accuracy that sets new benchmarks across GLUE, BEIR, code tasks, and more. Learn more about this exciting new embedding model here: https://lnkd.in/en5ud9VE

2