Stanford Engineering: Transformers & LLMs

The New Year holidays are over, and it’s the perfect time to start learning something new 🤔.

Stanford Engineering fully opened a course CME 295: Transformers & LLMs with explanation of LLMs core components, their limitations and how to use them effectively in real-world applications.
Course instructors are engineers with work experience in Uber, Google and Netflix, so they they really know what they are talking about.

Topic covered in the course:
- Transformers architecture
- Decoding strategies & MoEs
- LLMs finetuning & optimizations
- Results evaluation & Reasoning
- RAG & Agentic workflows

To really understand a topic, I need to know how everything is organized under the hood and what the core architectural principles are. That's why I really like such courses as they provide a structured and systematic view of the topic with all the necessary theory.

#ai #engineering

A2UI Protocol

Google has introduced a new protocol for AI - A2UI. It allows agents to generate rich user interfaces that they can be displayed in different host applications. Now Lit, Angular, and Flutter renderers are supported, others are in the roadmap.

The main idea is that LLMs can generate a UI from a catalog of predefined widgets and send them as a message to the client.

The workflow looks as follows:
🔸 User sends a message to an AI agent
🔸 Agent generates A2UI messages describing the UI (structure + data in JSON lines format)

{"surfaceUpdate": 
  {"surfaceId": "booking", 
    "components": [
      {"id": "root", "component": {"Column": {"children": {"explicitList": ["header", "guests-field"]}}}},
      {"id": "header", "component": {"Text": {"text": {"literalString": "Confirm Reservation"}, "usageHint": "h1"}}},
      {"id": "guests-field", "component": {"TextField": {"label": {"literalString": "Guests"}, "text": {"path": "/reservation/guests"}}}}
]}}

🔸 Messages stream to the client application
🔸 Client renders it using native components (Angular, Flutter, React, etc.)
🔸 User interacts with the UI, sending actions back to the agent
🔸 Agent responds with updated A2UI messages

According to the article, the main benefits of the protocol are:
🔸 Security: No LLM-generated code, there is only a declaration passed to the client.
🔸 LLM-friendly: Flat structure, easy for LLMs to generate incrementally
🔸 Framework-agnostic: Separation of the UI structure from the UI implementation.

Right now the project is in early public preview. It looks promising, especially once REST protocol will be supported (now only A2A and AG-UI are supported).

#ai #news

Make Your Docs Ready for AI

Several years ago, we wrote documents for humans. Now we write documents for AI 😀.

And, to be honest, machines require much more structured text than humans do. Poorly structured content leads to incorrect chunking and low quality answers during RAG context evaluation.

Common recommendations for modern docs:
🔸 Use structured format: md, html, asciidoc.
🔸 Provide metadata: important dates, tags, context and document goal.
🔸 Define glossary: describe key terms and abbreviations.
🔸 Organize content hierarchy: create clear structure using descriptive headings and subheadings.
🔸 Use lists: prefer bulleted or numbered lists instead of comma-separated enumerations.
🔸 Include text description for visual information: duplicate important details from diagrams, charts, and screenshots via text.
🔸 Use simple layout: do not use complex visuals and tables, prefer simple headings, lists and paragraphs.
🔸 Keep related information together: design content in a way that each section contains sufficient context to be understood independently.
🔸 Describe external references: when referencing external concepts, provide brief context and explanation.

As you can see, the recommendations are quite simple. And moreover, it feels like I became a machine a long time ago: I hate long paragraphs without clear structure and prefer well-structured documents for years.

#ai #documentation

AI-Ready Repos: AGENTS.md

Structuring project documentation helps build a good knowledge base, but it's not enough to work effectively with the codebase. In practice, agents also need extra instruction files: AGENTS.md and SKILLS.md. Let's start with AGENTS.md: what it is for and how to cook it properly.

AGENTS.md is a markdown file that provides context, instructions, and guidelines for AI coding agents working with the repo.

Its content is added to the initial prompt (system context) when LLM session is created. What matters that this is a standard that widely adopted by different agents such as Cursor, Codex, Claude Code and many others.

Common structure:
🔸 Project overview: project description, tech stack with particular versions, key folders and dependencies.
🔸 Commands: list of build and test commands with required flags and options.
🔸 Code Style: describe preferred code style.
🔸 Testing: commands to run different types of tests and linters.
🔸 Boundaries: do's and don'ts (e.g., never touch secrets, env configs).
🔸 Extra: PR guidelines, git workflow details, deployment instructions, etc.

Common recommendations:
🔸 Keep it short (~150 lines)
🔸 Continuously update it with code changes
🔸 Be specific, prefer samples over description
🔸 Improve it iteratively by adding what really works and removing what doesn't
🔸 Use nested AGENTS.md files in large codebases. The agent reads the closest file to the work it is doing.

Sample:

# Tech Stack
- Language: Go 1.24+
- API: gRPC
- Database: PostgreSQL 18
- Message Queue: RabbitMQ 4.2, Apache Kafka 4.1.x
- Observability: OpenTelemetry, Jaeger, Prometheus, Grafana
- Security: JWT, OAuth2, TLS
- Deployment: Docker, Kubernetes, Helm
  
# Build & Test Commands
- Build: `go build -o myapp`
- Test `go test` 
  
# Boundaries
- Never touch `/charts/secrets/` files. 
- Avoid adding unnecessary dependencies.

# PR Submission

## Title Format (MANDATORY)
Issue No: User-facing description

Samples from opensource projects:
- RabbitMQ Cluster Operator
- Kubebuilder
- Airflow
- Headlamp

Additionally I recommend reading How to write a great agents.md: Lessons from over 2,500 repositories from Github blog. I didn't get how they measured the effectiveness of analyzed instructions, but anyway the overall recommendations can be helpful.

#ai #agents #engineering #documentation

Fighting the Calendar Chaos

The more responsibilities you have, the more meetings you get. Unfortunately, it's a typical story.

At some point of time there are so many meetings in the calendar, it's not clear how to manage the day.

To bring it back to a manageable state, I use the following tips:
🔸Book slots in the calendar for my working tasks
🔸Decline meetings that I think I don't need to attend (no clear agenda, already delegated, etc.)
🔸 Use color coded categories to classify the meetings. I mostly sort them out according to the importance and urgency, for example:
- Black: urgent and/or important, I must attend.
- Red: not urgent but important, need to attend.
- Orange: not so important, make sense to attend if I have time (e.g., some regular meetings or where I believe my team can make a decision without me).
- Gray: not needed, for my info that meeting is scheduled.

You can use whatever categories are convenient for you. But I don't recommend having more than 5 categories, it’s hard to keep them all in your head. Additionally, meetings categorization helps to review the week if there is enough time for important tasks. If not, it's a sign to reflect and change something.

#softskills #tips #productivity

AI-Ready Repos: Skills

Previously we checked how to cook AGENTS.md, today we'll check another important part of agent configuration - skills.

Agent skill is a detailed workflow description or checklist for performing a specific task. Technically a skill is a folder containing a `SKILL.md` file, scripts, and additional resources. Most imprtantly, it's a standard already supported by different AI agents.

Skills structure in a repo:

skills/
├──my-skill/ 
  ├── SKILL.md # Required: instructions + metadata  
  ├── scripts/ # Optional: executable code 
  ├── references/ # Optional: documentation 
  └── assets/ # Optional: templates, resources

SKILL.md is a standard md file used to define and document agent capabilities. It consists of:
- Metadata (~100 tokens): name(skill name) and description (when to use).
- Instructions (< 5000 tokens recommended): The main part of the file that is loaded when the skill is activated.

The overall flow is as follows::
1. Discovery: At startup the agent loads the name and description of each available skill.
2. Activation: When a task matches a skill’s description, the agent reads the full `SKILL.md` instructions into context.
3. Execution: The agent follows the instructions, loading referenced files or executing bundled code if needed.

The main idea is to load instructions lazily to prevent prompt sprawl and context rot, when LLMs lose focus on specific tasks and start making mistakes.

What I like about skills is that it's an artifact with clear specification and development lifecycle. It allows not only collecting knowledge about tools and procedures, but also makes AI outputs more testable and predictable.

#ai #agents #engineering #documentation

The Words You Use

Have you ever been in the meetings where people say the same thing in different words and still don’t understand each other? Or maybe you were in another situation: you explain something, but the audience doesn't get it, then someone else repeats the same idea and surprisingly it clicks.

Strangely enough, it all comes down to the words you use.
Words reflect the picture of the world in someone’s head.

To make communication effective, pay attention to what a person says and how they say it, what definitions they use and what terms they like. If this is someone you work with regularly, you can do a deeper analysis and understand what is important for that person and why.

Then start using the same terms, metaphors, and definitions. If you fit into a person’s picture of the world, then it become much easier to share your ideas to that person. You start talking on the same language using the same vocabulary and stop wasting time on pointless arguing .

#tips #communications

Can you see the forest for the trees?

How often have you seen developers stuck in code review discussing some method optimization or "code excellence"? Spending hours or even days trying to make code perfect? Did that really help build a well-architected solution?

Developers often get stuck in small details and completely lose sight of the bigger picture. That's a very common mistake that I see in engineering teams. As a result, there are perfect classes or functions and complete mess in overall structure.

A proper review should always start with a bird’s-eye view:
🔸 Component structure: Are changes implemented in the components\services that are actually responsible for this logic?
🔸 Module structure: Are changes in the modules you expect them to be (public vs private, pkg vs internal, etc.)?
🔸 Public contracts: Review how your APIs will be used by other parties. Are they clear, convenient, easy to use, and easy to extend?
🔸 Naming: Are module, class and function names clear and easy to understand? Don't they duplicate existing entities?
🔸 Data model: Is the domain modeled correctly? Does the model follow single responsibility principle?
🔸 Testing: Are main cases covered? What about negative scenarios? Do we have proper failure handling approach?

In most cases, there’s no point in reviewing code details until the items above are finalized. The code will likely be rewritten, maybe even more than once.
That’s why specific lines of code should be the last thing to check.

Details are cheap to fix.
Structure and contracts are not.

#engineering #codereview

1001 GenAI Use Cases

Over the past year, AI has been everywhere, even the coffee machines seem to be talking about it.
Google collected over 1001 GenAI use cases from their clients. For me it's really interesting to see how different businesses adopted AI capabilities.

So I picked the top-5 the most impressive to share with you:
🔸 In-car assistant (Volkswagen of America, Mercedes Benz): Voice assistant that can explore user's manual and answer questions like "How do I change a flat tire?" or "What does this red indicator mean?". I think that's an absolutely amazing idea especially for drivers like me 😃
🔸 Intersatellite communication (Satlyt): In-orbit AI workloads to orchestrate intersatellite communication and routing. This is a real "rocket science" example.
🔸 Emotional temperature of conversations (704 Apps): Audio content from car rides goes to an LLM and risky language patterns generate alerts. The main idea is to improve safety for both drivers and passengers.
🔸 Driver safety (Geotab): AI analyzes billions of vehicle data points for fleet optimization, safety, and sustainability on the roads.
🔸 Logistics networks (UPS): building a digital twin of the distribution network (live simulation environment) to show where delivery is at any time.

#ai #usecase

Self-Doubting

Have you ever doubted your decisions?
I do all the time.

Previously I thought that the reason was the lack of expertise or relevant experience. But now I understand that the answer is much simpler: self-doubt is OK.

Doubts motivate us to ask more questions, discover alternative solutions, closely collaborate with the team, listen and, most importantly, hear another points of view. Doubts help us to stay curious and not rush to choose the first answer that looks convincing.

A leader who doesn't doubt their decisions is dangerous. They are blind and deaf.

But be careful: self-doubt is often a contributor to imposter syndrome. By understanding your limiting beliefs and triggers, you can look at your self-doubt more rationally.

Funny fact: according to the US National Science Foundation the average person has about 12,000 to 60,000 thoughts per day, and 80% of them are negative 😲. Doubting itself can be a survival mechanism, as it push us to carefully assess risks and potential consequences before taking action.

#leadership #softskills

Scale Cube

In IT, we work in a very complex domain. We have to keep a lot of things in our heads at once: technologies, patterns, trade-offs, limitations.
That's why I like simple models that help me stay focused and remember technical concepts.

One such model is Scale Cube. This is the model introduced in 2009 in the book "The art of scalability" and it suggests 3 dimensions for scaling:
🔸 Horizontal scaling: duplicate similar things, clone data, add more replicas.
🔸 Functional decomposition: split application to multiple services.
🔸 Sharding: split data into subsets by region, tenant, hash, or range.

The starting point is always monolith.
The end point is near-infinite scale with all 3 dimensions implemented together (see diagram in the post).

That's it. It's very simple and powerful. If you want to scale something you have only 3 strategies to do that 😎. So you don’t need to waste time reinventing the wheel.

#architecture #engineering #scalability

Residual Architecture

The business environment is dynamic and complex: markets change, new regulations appear, technologies evolve. And systems we design must operate within this complexity. That's where Residuality Theory of Architecture starts from.

The theory was developed by Barry O'Reilly and was presented at multiple conferences. One of the talks was on GOTO 2025 Residues: Time, Change & Uncertainty in Software Architecture.

The main idea:

Randomly stressing an architecture produces architectures that are more likely to survive unexpected events in their environment.

Let's go into more details:
🔸 Software is rigid and ordered, business environment is complex and fluid.
🔸 Unpredictable events can destroy the architecture. Such events are called stressors.
🔸 Parts of the system that stay functional after the stress are called residues. The number of residues defines the system's future state.
🔸 A complex system has a limited number of states that it is able to reach. These states are called attractors.
🔸 The number of attractors is much less than the possible number of stressors.
🔸 The number of attractors is defined by the following parameters:
- N - number of services or modules.
- K - number of connections or integrations between the services.
- P - probability of some behavior based on APIs, specs, contracts and policies.
🔸 The higher N and K, the more attractors the system can reach.
🔸 The main tool to define attractors is stressor analysis: generate a list of undesired events, define how our system will react, identify what will be broken, and add missing elements to the architecture.
🔸 The more attractors and residues we define, the more resilient our final architecture is.

The theory is good, and I like that the author tries to make software architecture domain more scientific. But in practical terms, this theory can be described as a structured form of 'design for failure'.

One more thought about this talk: the main assumption is that business context is dynamic and software is rigid. But in the era of AI, it looks like this assumption may become wrong as our software becomes more and more dynamic. It means that the number of attractors will increase and in addition to APIs and contracts we will need new types of constraints and boundaries to make software resilient.

#architecture #resilience

Leading with Let Them

Leading with Let Them is a separate guide on how to apply The Let Them Theory in leadership. The guide is published separately from the book and co-authored with David Gerbitz, an executive coach who previously held senior leadership roles at Amazon, Microsoft, Yahoo, and Qurate Retail Group.

Key recommendations:
🔸 Lead yourself (Let Me). Leader defines the mood of the team: "If you’re not managing your own emotions, mindset, and reactions, you can’t expect your team to manage theirs."
🔸 Choose inspiration over micromanagement (Let Them). Let Them means start creating the conditions where the team can grow: let them think, decide, take ownership and accountability of their work.
🔸 Choose influence over control (Let Me). Leadership is about building connection, trust and understanding. Recognize our triggers, test our beliefs, pause before responding, lead by example.
🔸 Focus on outcomes (Let Me). Provide a clear vision of what should be achieved (the “what” and the “why”) and support the team on their way when needed.
🔸 Reframe mistakes as growth opportunities (Let Me).
🔸 Be personable and present (Let Me). Give people our time and our attention and encourage the team to do the same for each other. Put away the phone and focus on the moment.
🔸 Be consistent (Let Me). Consistency is a base of certainty, certainty is the base of safety. Safe environment is a foundation of successful teams.

As you can see, most of the items relate to the Let Me part. This highlights that a leader should control and adjust their own behavior to make the team effective.
In addition to the main principles the guide contains practical recommendations like do's and don'ts. So it can be really helpful when applying those principles in real life.

From my perspective, the guide is very useful and it's definitely a must-read for leaders, especially those who are just starting their leadership career.
When I read it I realized it describes many things I already do in practice: sometimes intuitively, sometimes based on experience. I just know it works and it reflects my leadership style. And now I have a very good reference with this knowledge to recommend.

#softskills #leadership

The Developer Skill Shift

New AI tools appear every day, new coding models become smarter and smarter, AI agents can automate more and more work. It seems impossible to keep up with everything and easy to get lost in all this diversity. And the common question is "Are we gonna be replaced by AI soon?"

In the last few months I attended multiple AI events, watched the latest talks from conferences, read a lot of blogs and discussions. So what I can say for sure is that the developer role is changing. And this change is not about a future where anyone can just vibe-code anything. No. What we give as an input for LLMs will be multiplied in the final result. Garbage in -> garbage out.

This means that developers must have much more professional skills than ever. And it's not the knowledge of particular framework or technology.

New paradigm prioritizes a different set of skills:
🔸 System Design: knowledge of architectural patterns, trade-offs and cost analysis.
🔸 System Thinking: ability to understand a system as a whole, analyze consequences from the changes, build the system of checks and balances for AI agents.
🔸 Critical Thinking: ability to challenge AI generated results.
🔸 Product Thinking: understanding the business context and real problem to solve, product roadmap and evolution.
🔸 Communication Skills: ability to clarify requirements, discuss architecture, clear and in detail explain our intensions to AI assistants, remember "garbage in -> garbage out."

Of course, all those skills are not really new. But previously we expected them from more senior positions like senior developers, architects and leads. Now they are expected from every engineer.

But the important thing is that we are still engineers. We still transform our knowledge and expertise into working products.

AI doesn't replace expertise. It gives us tools to produce better results faster, making strong engineers even stronger.

#engineering #ai

10 Tips for AI-Assisted Coding

10 Tips To Level Up Your AI-Assisted Coding is an absolutely wonderful talk from the latest NDC London conference. Alex Stensby shares practical recommendations on how to improve interaction with coding agents:

🔸 Context is King. Context is a limited resource, so we must actively manage it:
- Provide enough context for the task: role, file references, specs, schemas, logs, etc.
- Explicitly request the agent to ask additional questions if more information is needed. It helps to prevent hallucinations.
- Start a new context for each new feature to keep agent focused.
- Summarize the progress and save it to md file for future work.
🔸 Rules & Docs make all the difference. Actively document contracts, specifications, database schema, use AGENT.md to set the initial context of the project, build a library of reusable skills.
🔸 Make a plan. Always use plan mode. It forces the LLM not to go with the first suitable solution but to reflect on the task. Review the plan, give feedback on it, "make it your plan".
🔸 Break it down. Break complex tasks into smaller manageable pieces of work, keep track of them in md file or github issues, track the progress.
🔸 Pick the right model. Use a cheaper model for simpler tasks. But Alex said that he uses the latest Opus 4.x for all coding tasks now.
🔸 Use the tools. Turn repetitive prompts into skills or slash commands, use subagents to run work in parallel or focus on specific task types (e.g., architect, code reviewer, QA specialist)
🔸 Git everything & learn from rabbit holes. Ask AI to learn from and reflect on its mistakes, save results in md files and add it to the memory.
🔸 Power up with MCPs. Use MCPs carefully, especially with databases or other resources where AI can make dangerous changes, better to give it read-only access only.
🔸 Release the agents. Use multiple agents to solve different tasks, orchestrate them, fork their context to explore other options, send them for remote execution when needed.
🔸 Be the human in the loop. Agents make mistakes. That's a fact. So we must review and verify work results.

I definitely recommend watching the full video, as it contains not only common tips for using AI agents but it also has a lot of practical examples of Claude Code usage.

#engineering #ai

AI-Generated Architecture Diagrams

If you've ever built architecture diagrams, you know how time-consuming it can be.
I use drawio and sometimes I spend hours aligning elements to make the picture compact, clear and not overloaded with elements and connections between them (hello to my perfectionism).

The great news is that AI now can help with this task also using Drawio MCP. Initially I was quite skeptical. Anyone who has tried to generate a diagram by architecture description will understand me 🙂.

Surprisingly, the result is good. Not perfect, but really good.

I tested it with creating new diagrams and modifying existing ones according to some template. New diagrams are a bit clumsy, so you need to clean them up. Modification according to the template shows better results, it aligned around 10 different diagrams to the same template within minutes.

Of course as with any AI agent task you need to tune the output, explain mistakes and save lessons learnt to the memory. But finally I got desired result much faster then doing that on my own.

#engineering #ai #tips

Claude Code: Behind the Scenes

Claude Code is one of the most popular developer tools today. But did you know that initially it was just a side project inside the company?

This and many other interesting details are discussed in an interview with Boris Cherny, the creator and head of Claude Code at Anthropic.

Key insights from the interview:
🔸 Boris built Claude Code as a bash chat-based tool when he was learning public Anthropic APIs and how people use the model. It quickly became popular among other employees, that led to a future public success.
🔸 Claude writes ~80% of the code at Anthropic on average.
🔸 Boris ships 20-30 PRs a day by running 5 parallel Claude instances. He starts with a plan mode, iterates over the plan, then let the agent do the implementation. Since the Opus 4.5 release, Claude writes 100% of his code.
🔸 There is no "right way" to use Claude Code. According to Boris, "The way we build cloud code is to be hackable because we know every engineer's workflow is different. There's no one way to do things. There's no two engineers that have the same workflow."
🔸 The Claude team doesn't write Product Requirement Documents (specs). They just build dozens of working prototypes before shipping a feature.
🔸 Claude Code reviews every pull request at Anthropic and it catches ~80% of bugs. There are 2 rounds of review: the first is performed by AI agent, the second is always done by human who finally approves the changes.
🔸 Claude Cowork is intended to provide Claude capabilities for non-engineers. The tool was built in ~10 days and the main engineering complexity here is about safety: building classifiers, a shipping VM, OS-level protections against accidental file deletion, and rethinking the permission model for non-technical users.
🔸 There are no technical grades inside Anthropic. Everyone has the same title "Member of Technical Staff", that highlights the assumption that everyone can do everything: product, design, infrastructure, research.

During the interview Boris repeated multiple times that it's more important to ship fast and get user feedback early than to wait and deliver a fully featured product. That's why they built their engineering culture around prototyping. I think it's one of the reasons Anthropic products are so successful.

By the way, if you haven't tried Claude Code yet I highly recommend to do so. And as a starting point you can use Claude Code In Action Anthropic course.

#ai #engineering #usecase

Uber: Agent-Centric Organization

Looking for inspiration on how to apply AI in your team? Then check Uber: Leading engineering through an agentic shift where the Dev Platform team presents Uber's AI adoption approach and its current state.

Uber's agent platform includes:
🔸 MCP Gateway & Registry. Central MCP gateway to expose external and internal MCPs and provide a secure sandbox for experiments.
🔸 ML Michelangelo platform. An agent builder with no code or SDK solutions with built-in visualization, telemetry, tracing.
🔸 AIFX. A tool to access internal agents infrastructure: provisioning, discovery, configuration, background tasks.
🔸 Minion. Background agent platform to integrate them with CI\CD, slack, PRs.
🔸 Code Inbox. Unified inbox for PRs developers need to review. It tries to find the most relevant person to review the code, track review SLOs, help reassign PRs or make escalation if necessary.
🔸 uReview. Review pipeline that enriched with internal context, best practices, guidelines.
🔸 Autocover. A system to generate unit tests. 3x higher code test quality than generated by generic agent, 5000+ tests generated per month.
🔸 Automigrate. A tool to implement large-scale changes. It consists of problem identifier, code transformer (openwrite, piranha or agents), validation, and campaign manager (route PRs to reviewers, split changes on reasonable PRs, rebases, changes prioritization)

As you can see the team uses AI to support the entire development lifecycle. The whole strategy sounds like "Enable Uber engineers to focus on creative work by eliminating toil". By toil they mean upgrades, migration, bugfixes, writing docs, cleanup.

To sum up, Uber has a very reasonable strategy for AI adoption. Of course, they made a huge investment to their agent platform. But one simple recipe suits everyone: define the most tedious repeatable tasks and give it to AI. People are burnt out, agents are not.

#ai #engineering #usecase

Are You Ready for Coding Agents?

Coding agents are booming. Only the lazy haven’t yet talked about how they built their own agent. But the reality is much more complex. To get the benefits from AI your development ecosystem should be ready for it.

Garbage in -> garbage out, remember?
Low coverage, flaky tests, undefined code style, long verification cycle, poor documentation. Add AI-generated code to this and you will increase the entropy and reduce overall system stability.

To produce predictable results the engineering infrastructure must be stable.

By infrastructure I mean:
🔸 Linters and automated code style verification.
🔸 High unit tests coverage (>=80%).
🔸 Contract tests for all public APIs.
🔸 System, integration, and E2E tests that run at least once a day.
🔸 No flakiness. You must fully trust your tests and CI process otherwise you cannot guarantee that agent won't break anything.
🔸 Security gates. Secret management, vulnerability checks, SAST verification.
🔸 Documentation. Requirements, architecture, guides, internal agreements. Everything that helps the agent understand how we work.

The most non-obvious part here is test flakiness.
What's the problem with just rerunning the test?
Developers know the context, the agents do not. It means that they will try to fix the test, making it weaker, or modify the code, introducing a bug. The overall result is worse code generation and increased maintenance overhead. So each rerun must be treated as a bug report, not a solution.

If you check how different companies adopt AI, you can notice that all success stories are based on existing powerful CI\CD processes that can safely check AI agent output (Google, Claude Code, Uber, Google, Airbnb).
AI adoption doesn't just bring new tools and processes but also forces the best engineering practices we already have.

#engineering #ci #ai #agents