Project Hail Mary
Technical books and articles are great, but sometimes my brain needs a break. Especially now, when AI is generating more and more new things to learn every day. One of my favorite ways to recharge is reading fiction, and I recently finished the very popular Project Hail Mary by Andy Weir.
I'm not a big sci-fi fan, but I definitely enjoyed this book.
Thanks to the recent movie adaptation, the story is probably familiar to many.
A man wakes up alone on a spaceship with no memory of who he is or why he's there. As his memories gradually return, he discovers that he's a scientist on a mission in another star system.
Humanity is facing extinction. The Sun is losing energy because of a mysterious organism called Astrophage. Nearby stars are also infected except Tau Ceti. A crew is sent there to find out why it's different and, hopefully, save Earth.
Unfortunately, only the main character survives the journey.
He starts his scientific research of the star and eventually noticed a spacecraft on his radar.
And then the almost impossible happens: first contact with an alien. The problem is, how do you communicate when you don't even share the same way of producing speech? The answer is physics. I really liked the idea that laws of physics are universal, making math and science the foundation for building communication between two civilizations.
I won't spoil the rest, but the story is really engaging. Despite being a disaster novel, it contains a good dose of humor and places a strong emphasis on friendship, kindness, and mutual help. And when you finish it, you're left with a surprisingly warm feeling.
I watched the movie after finishing the book, and for once I can say the adaptation is actually good. Of course, it's much more compact and some details are simplified, but it stays remarkably close to the original story while preserving its emotional depth.
Overall, I loved it. Highly recommended both the book and the movie.
#offtop #booknook
Technical books and articles are great, but sometimes my brain needs a break. Especially now, when AI is generating more and more new things to learn every day. One of my favorite ways to recharge is reading fiction, and I recently finished the very popular Project Hail Mary by Andy Weir.
I'm not a big sci-fi fan, but I definitely enjoyed this book.
Thanks to the recent movie adaptation, the story is probably familiar to many.
A man wakes up alone on a spaceship with no memory of who he is or why he's there. As his memories gradually return, he discovers that he's a scientist on a mission in another star system.
Humanity is facing extinction. The Sun is losing energy because of a mysterious organism called Astrophage. Nearby stars are also infected except Tau Ceti. A crew is sent there to find out why it's different and, hopefully, save Earth.
Unfortunately, only the main character survives the journey.
He starts his scientific research of the star and eventually noticed a spacecraft on his radar.
And then the almost impossible happens: first contact with an alien. The problem is, how do you communicate when you don't even share the same way of producing speech? The answer is physics. I really liked the idea that laws of physics are universal, making math and science the foundation for building communication between two civilizations.
I won't spoil the rest, but the story is really engaging. Despite being a disaster novel, it contains a good dose of humor and places a strong emphasis on friendship, kindness, and mutual help. And when you finish it, you're left with a surprisingly warm feeling.
I watched the movie after finishing the book, and for once I can say the adaptation is actually good. Of course, it's much more compact and some details are simplified, but it stays remarkably close to the original story while preserving its emotional depth.
Overall, I loved it. Highly recommended both the book and the movie.
#offtop #booknook
👍3🔥2❤1
"AI won't take your job. Someone using AI will."
This quote caught my attention and made me watch A Leader’s Guide to Advanced Team Structures in an Agentic World from the recent AWS Summit Sydney.
It's a very sobering talk on the current state of the industry, AI adoption, and the future of engineering roles.
The central question of the talk is: "How should we build teams to work in this new AI world?"
To answer it, the author proposes a framework based on four elements:
Economics
The market has changed. Timelines are compressing. A small team of senior engineers can replace an entire existing product. This creates real risks for businesses that fail to adapt in time. That's why AI decisions should be driven by economics and business value, not hype.
Talent
Previously, career growth in tech was mostly about writing code and building features. Today, the most valuable skill is understanding the business, customers, and product. In other words, AI rewards expert generalists. One person can now handle analysis, backend, and frontend, reducing collaboration overhead and the need for deep specialization of large teams.
Another interesting point is the future of junior engineers. The speaker argues that we must keep the junior pipeline alive. Otherwise, we won't have senior expertise in 2034.
Structure
Current IT operations are optimized for determinism. But agents are non-deterministic. So operating model has to shift: variance in execution, focus on outcome and guardrails around thing you actually care about. The best operational model there is platform engineering.
Governance
The author highlights several areas that organizations need to address:
- Agent Identity Management. Every agent should have a verifiable identity traced to a named human.
- Risk Assessment. Clearly define what an agent is allowed to do and ensure it operates within those boundaries.
- Multi-agent Coordination. Control what happens when agents disagree, escalate, or find emergent behavior we don't expect.
- Deskilling Prevention. Employees should maintain core skills even if agents automate routine work. Someone still needs to validate results, audit actions, and take responsibility for decisions.
Overall, the talk is a good reality check on what is actually happening and how businesses and teams need to change to remain successful. Much of it resonates with my own observations, so I would definitely recommend to watch the full video.
#ai #leadership #engineering
This quote caught my attention and made me watch A Leader’s Guide to Advanced Team Structures in an Agentic World from the recent AWS Summit Sydney.
It's a very sobering talk on the current state of the industry, AI adoption, and the future of engineering roles.
The central question of the talk is: "How should we build teams to work in this new AI world?"
To answer it, the author proposes a framework based on four elements:
Economics
The market has changed. Timelines are compressing. A small team of senior engineers can replace an entire existing product. This creates real risks for businesses that fail to adapt in time. That's why AI decisions should be driven by economics and business value, not hype.
Talent
Previously, career growth in tech was mostly about writing code and building features. Today, the most valuable skill is understanding the business, customers, and product. In other words, AI rewards expert generalists. One person can now handle analysis, backend, and frontend, reducing collaboration overhead and the need for deep specialization of large teams.
Another interesting point is the future of junior engineers. The speaker argues that we must keep the junior pipeline alive. Otherwise, we won't have senior expertise in 2034.
Structure
Current IT operations are optimized for determinism. But agents are non-deterministic. So operating model has to shift: variance in execution, focus on outcome and guardrails around thing you actually care about. The best operational model there is platform engineering.
Governance
The author highlights several areas that organizations need to address:
- Agent Identity Management. Every agent should have a verifiable identity traced to a named human.
- Risk Assessment. Clearly define what an agent is allowed to do and ensure it operates within those boundaries.
- Multi-agent Coordination. Control what happens when agents disagree, escalate, or find emergent behavior we don't expect.
- Deskilling Prevention. Employees should maintain core skills even if agents automate routine work. Someone still needs to validate results, audit actions, and take responsibility for decisions.
Overall, the talk is a good reality check on what is actually happening and how businesses and teams need to change to remain successful. Much of it resonates with my own observations, so I would definitely recommend to watch the full video.
#ai #leadership #engineering
YouTube
A leader’s guide to advanced team structures in an agentic world | AWS Events
As AI agents transform the workplace, organizations must adapt their structures and methodologies to harness new opportunities. The probabilistic nature of AI requires continuous iteration and intelligent oversight, creating new ways of working across business…
👍2🔥2❤1
A Few Words About Context
New major model releases regularly promise bigger context windows. Sounds great until you realize it's mostly marketing. A bigger context window doesn't mean better results. It often means more data, more noise, and more AI slop.
According to multiple studies, models effectively use only about 30–50% of their available context. For example, a model with a 200K-token context window may already show noticeable quality loss at around 50K tokens.
Why this happens:
🔸 Context rot. Output quality gradually degrades as the context grows.
🔸 Reasoning shift. The model spends less effort on reasoning. The answers sound more confident, but their quality often gets worse.
🔸 The lost-in-the-middle effect. Information in the middle of the context can be overlooked during later reasoning.
🔸 Attention dilution. The model's attention is spread across different instructions, making it harder to focus on what actually matters.
The practical takeaway is simple: keep your context clean:
🔸 Start a new conversation for each new task (
🔸 During long-running tasks, use /compact regularly to collapse intermediate reasoning and keep only the important things.
🔸 Store large data in long-term memory or relevant documentation, and bring it into the context only when it's actually needed.
Useful references:
- https://www.morphllm.com/context-rot
- https://www.zenml.io/llmops-database/context-rot-evaluating-llm-performance-degradation-with-increasing-input-tokens
- https://arxiv.org/html/2601.11564v1
#engineering #ai #tips
New major model releases regularly promise bigger context windows. Sounds great until you realize it's mostly marketing. A bigger context window doesn't mean better results. It often means more data, more noise, and more AI slop.
According to multiple studies, models effectively use only about 30–50% of their available context. For example, a model with a 200K-token context window may already show noticeable quality loss at around 50K tokens.
Why this happens:
🔸 Context rot. Output quality gradually degrades as the context grows.
🔸 Reasoning shift. The model spends less effort on reasoning. The answers sound more confident, but their quality often gets worse.
🔸 The lost-in-the-middle effect. Information in the middle of the context can be overlooked during later reasoning.
🔸 Attention dilution. The model's attention is spread across different instructions, making it harder to focus on what actually matters.
The practical takeaway is simple: keep your context clean:
🔸 Start a new conversation for each new task (
/new in Claude).🔸 During long-running tasks, use /compact regularly to collapse intermediate reasoning and keep only the important things.
🔸 Store large data in long-term memory or relevant documentation, and bring it into the context only when it's actually needed.
Useful references:
- https://www.morphllm.com/context-rot
- https://www.zenml.io/llmops-database/context-rot-evaluating-llm-performance-degradation-with-increasing-input-tokens
- https://arxiv.org/html/2601.11564v1
#engineering #ai #tips
👍4🔥2❤1
Akrites Project
Over the past year, we've seen how much the industry depends on small open source projects. In many cases, software used by thousands of companies is maintained by one or two people working on it in their free time.
The release of the Mythos and Fable models made the situation even worse. They demonstrated how many vulnerabilities AI can find across thousand of projects, putting the whole enterprise IT infrastructure at risk.
Last Thursday (June 25), the Linux Foundation and a set of big IT companies announced a new initiative called Akrites, "a coordinated effort to remediate and disclose vulnerabilities in critical open source software."
The goal is simple: bring engineering resources together to identify, fix (!!!), and responsibly disclose vulnerabilities in critical open source software before they can be exploited.
If a package has no active maintainer, Akrites will serve as a "maintainer of last resort so fixes to the latest version reach everyone in a timely fashion".
And of course, they will do that with the help of AI frontier models.
The list of members is impressive: AWS, Anthropic, Cisco, Google, Microsoft, NVIDIA, IBM, OpenAI, Red Hat, and many others. It's one of the largest coordinated industry initiatives we've seen in recent years.
Personally, I think this is a very important shift. For years, the biggest challenge was finding enough people to make security fixes on time. AI has made the situation much more critical by flooding maintainers with newly discovered vulnerabilities.
The great thing about this initiative is that engineering resources are finally being focused not on reporting vulnerabilities, but on fixing them.
So the idea looks promising. Let's see how well it works in practice.
#news #opensource #security
Over the past year, we've seen how much the industry depends on small open source projects. In many cases, software used by thousands of companies is maintained by one or two people working on it in their free time.
The release of the Mythos and Fable models made the situation even worse. They demonstrated how many vulnerabilities AI can find across thousand of projects, putting the whole enterprise IT infrastructure at risk.
Last Thursday (June 25), the Linux Foundation and a set of big IT companies announced a new initiative called Akrites, "a coordinated effort to remediate and disclose vulnerabilities in critical open source software."
The goal is simple: bring engineering resources together to identify, fix (!!!), and responsibly disclose vulnerabilities in critical open source software before they can be exploited.
If a package has no active maintainer, Akrites will serve as a "maintainer of last resort so fixes to the latest version reach everyone in a timely fashion".
And of course, they will do that with the help of AI frontier models.
The list of members is impressive: AWS, Anthropic, Cisco, Google, Microsoft, NVIDIA, IBM, OpenAI, Red Hat, and many others. It's one of the largest coordinated industry initiatives we've seen in recent years.
Personally, I think this is a very important shift. For years, the biggest challenge was finding enough people to make security fixes on time. AI has made the situation much more critical by flooding maintainers with newly discovered vulnerabilities.
The great thing about this initiative is that engineering resources are finally being focused not on reporting vulnerabilities, but on fixing them.
So the idea looks promising. Let's see how well it works in practice.
#news #opensource #security
Akrites
Coordinated, confidential vulnerability remediation for the open source software critical infrastructure depends on
👍2🔥2❤1
Who Is Responsible for AI-Generated Code?
I hear conversations like this more and more often. The funny part is that as soon as someone says a feature was "vibe coded," expectations immediately drop. At least among developers 😃.
But that raises an interesting question: who is actually responsible for that feature being released?
The answer hasn't changed. It's still the teamlead or techlead.
And here's what we actually have:
On one hand, we can generate much more code, and naturally the business expects features to be delivered faster and cheaper.
On the other hand, we now have much more code to review and many more bugs to catch.
Yes, agents can review code too (just look at how many articles have been written about it).
But here's a simple question:
Would you personally sign off on these changes?
What if it's a B2B product with contractual penalties?
Or a mission-critical system in aviation or healthcare?
For now, code review is one of the main bottlenecks. Developers spend more and more time verifying what the agent generated.
The cognitive load grows. Before, you might have reviewed one feature a day. Now you may need to review five. AI review can help, just like tests, linters, and other engineering practices help.
But today it still doesn't replace human responsibility.
That's why the real challenge is to find the right balance between speed and quality.
And we should measure not only the number of features delivered and development speed, but also the resulting bug rate.
#ai #engineering
"This thing works terribly."
"Well... it was vibe coded."
"Ah, that explains it."
I hear conversations like this more and more often. The funny part is that as soon as someone says a feature was "vibe coded," expectations immediately drop. At least among developers 😃.
But that raises an interesting question: who is actually responsible for that feature being released?
The answer hasn't changed. It's still the teamlead or techlead.
And here's what we actually have:
On one hand, we can generate much more code, and naturally the business expects features to be delivered faster and cheaper.
On the other hand, we now have much more code to review and many more bugs to catch.
Yes, agents can review code too (just look at how many articles have been written about it).
But here's a simple question:
Would you personally sign off on these changes?
What if it's a B2B product with contractual penalties?
Or a mission-critical system in aviation or healthcare?
For now, code review is one of the main bottlenecks. Developers spend more and more time verifying what the agent generated.
The cognitive load grows. Before, you might have reviewed one feature a day. Now you may need to review five. AI review can help, just like tests, linters, and other engineering practices help.
But today it still doesn't replace human responsibility.
That's why the real challenge is to find the right balance between speed and quality.
And we should measure not only the number of features delivered and development speed, but also the resulting bug rate.
#ai #engineering
❤4🔥2👍1
ROI of AI-Assisted Development
One of the biggest questions around AI adoption is what business value does it actually bring?
I'm currently introducing AI into the SDLC across my teams, so measuring its real impact is something I'm really interested in.
Back in February 2026, DORA published a dedicated report on this topic: ROI of AI-assisted Software Development report.
Key takeaways:
🔸 Measure your baseline first. Before evaluating the impact of AI, you need to understand where you're now. Without "before" metrics, it's impossible to measure "after".
🔸 Expect a J-curve. Most organizations experience a temporary productivity drop before seeing benefits. Main reasons for that:
- The learning curve. Time to master new tools and workflows.
- The verification tax. Time to verify AI-generated code and establish trustworthiness of agents output.
- Pipeline adaptation. Scaling other downstream processes like testing and change approval to handle increased number of code changes.
🔸 Invest in engineering practices. The quality of an internal platform, clear workflows, reliable test automation, and strong engineering culture become critical to produce predictable delivery. The rule
🔸 Measure both positive and negative outcomes. Higher throughput is valuable only if it doesn't come with higher instability, more defects, or lower quality.
🔸 Measure the following business values:
- Cost efficiency
- Productivity
- Developer experience
- User experience
- Business growth
The report provides guidance on how to measure these areas and combine them into an ROI calculation. It also includes an online calculator where you can check your own numbers.
🔸 AI adoption takes time. According to the report, the average adoption journey takes around 8 months, in large enterprise rollout can take even 12–18 months.
Overall, the report is great: no hype, just a practical and rational approach.
It also strongly aligns with my own view of AI adoption. You can't simply buy AI licenses for developers and expect the investment to pay for itself.
Successful AI adoption requires improvements across the entire SDLC: better development processes, a stronger engineering culture, higher test coverage, and investments in guardrails and workflows that keep software delivery stable and predictable.
#ai #engineering #ai4sdlc
One of the biggest questions around AI adoption is what business value does it actually bring?
I'm currently introducing AI into the SDLC across my teams, so measuring its real impact is something I'm really interested in.
Back in February 2026, DORA published a dedicated report on this topic: ROI of AI-assisted Software Development report.
Key takeaways:
🔸 Measure your baseline first. Before evaluating the impact of AI, you need to understand where you're now. Without "before" metrics, it's impossible to measure "after".
🔸 Expect a J-curve. Most organizations experience a temporary productivity drop before seeing benefits. Main reasons for that:
- The learning curve. Time to master new tools and workflows.
- The verification tax. Time to verify AI-generated code and establish trustworthiness of agents output.
- Pipeline adaptation. Scaling other downstream processes like testing and change approval to handle increased number of code changes.
🔸 Invest in engineering practices. The quality of an internal platform, clear workflows, reliable test automation, and strong engineering culture become critical to produce predictable delivery. The rule
garbage in -> garbage out is still actual.🔸 Measure both positive and negative outcomes. Higher throughput is valuable only if it doesn't come with higher instability, more defects, or lower quality.
🔸 Measure the following business values:
- Cost efficiency
- Productivity
- Developer experience
- User experience
- Business growth
The report provides guidance on how to measure these areas and combine them into an ROI calculation. It also includes an online calculator where you can check your own numbers.
🔸 AI adoption takes time. According to the report, the average adoption journey takes around 8 months, in large enterprise rollout can take even 12–18 months.
Overall, the report is great: no hype, just a practical and rational approach.
It also strongly aligns with my own view of AI adoption. You can't simply buy AI licenses for developers and expect the investment to pay for itself.
Successful AI adoption requires improvements across the entire SDLC: better development processes, a stronger engineering culture, higher test coverage, and investments in guardrails and workflows that keep software delivery stable and predictable.
#ai #engineering #ai4sdlc
Google Cloud
DORA: ROI of AI-assisted Software Development
Unlock the true value of AI in dev, Learn to navigate the J-curve, translate metrics into financial outcomes, and reinvest recovered capacity
🔥5❤1👍1
AI Adoption: What to Measure First
In the previous post, we looked at how DORA recommends measuring the ROI of AI adoption.
But what if you don't know the revenue generated by your features or other business metrics? Yet you're still expected to show the effectiveness of AI adoption.
Let's bring the DORA approach down to the engineering team level.
I would split AI adoption into two phases: AI adoption itself and getting business benefits. These phases have different goals, metrics and outcomes.
It doesn't make much sense to measure business impact of AI until the team has actually gone through the adoption phase.
What to measure to understand the AI adoption state:
🔸 Active users. How many engineers actually use AI in their daily work?
🔸 Monthly usage per user. How actively do engineers use AI tools? Token consumption or API cost per engineer can be a good indicator. AI Gateway solutions such as LiteLLM can help collect these metrics.
🔸 AI-generated code ratio. What percentage of code is generated by AI? Important note: this metric measures only how widely AI is being used. It says nothing about whether AI is being used effectively or producing good results.
🔸 AI-assisted feature ratio. How many features are developed with AI assistance? The metric counts if AI was used in design, implementation, testing, documentation, or code review.
🔸 Engineering readiness. Is the engineering ecosystem ready for agents? Here I would use criteria similar to an Agent Readiness Framework: documentation,
Giving developers AI licenses does not mean AI has been adopted.
Not after one month. Not after three. Not after five.
AI adoption happens at the team level. The goal of all these metrics is to understand whether the teams actually change the way they work. And it requires training people, overcoming resistance, changing engineering practices, and adapting the development process.
In the next post, I'll share my approach to measuring AI impact during the second phase using only tools that almost every engineering team already has.
#ai #engineering #ai4sdlc
In the previous post, we looked at how DORA recommends measuring the ROI of AI adoption.
But what if you don't know the revenue generated by your features or other business metrics? Yet you're still expected to show the effectiveness of AI adoption.
Let's bring the DORA approach down to the engineering team level.
I would split AI adoption into two phases: AI adoption itself and getting business benefits. These phases have different goals, metrics and outcomes.
It doesn't make much sense to measure business impact of AI until the team has actually gone through the adoption phase.
What to measure to understand the AI adoption state:
🔸 Active users. How many engineers actually use AI in their daily work?
🔸 Monthly usage per user. How actively do engineers use AI tools? Token consumption or API cost per engineer can be a good indicator. AI Gateway solutions such as LiteLLM can help collect these metrics.
🔸 AI-generated code ratio. What percentage of code is generated by AI? Important note: this metric measures only how widely AI is being used. It says nothing about whether AI is being used effectively or producing good results.
🔸 AI-assisted feature ratio. How many features are developed with AI assistance? The metric counts if AI was used in design, implementation, testing, documentation, or code review.
🔸 Engineering readiness. Is the engineering ecosystem ready for agents? Here I would use criteria similar to an Agent Readiness Framework: documentation,
AGENTS.md, test coverage, CI stability, guardrails, and so on.Giving developers AI licenses does not mean AI has been adopted.
Not after one month. Not after three. Not after five.
AI adoption happens at the team level. The goal of all these metrics is to understand whether the teams actually change the way they work. And it requires training people, overcoming resistance, changing engineering practices, and adapting the development process.
In the next post, I'll share my approach to measuring AI impact during the second phase using only tools that almost every engineering team already has.
#ai #engineering #ai4sdlc
🔥3❤1👍1
AI Adoption: Measuring Feature Development
Let's assume your team has already completed AI adoption and now management expects to see positive business results.
The good news is that AI doesn't magically change your business metrics. You don't need to invent new KPIs. You just need to track how your existing ones change (or finally start collecting them).
I suggest focusing on quality metrics first, then gradually shift toward delivery speed and cost optimization.
Since I lead platform engineering teams, we have two major types of work: new features development and L4 support. These activities are different, so they should be measured differently. Let's start with product development.
What can be measured:
🔸 Team Budget. The cost of the team in $, mandays, or FTEs.
🔸 AI Cost ($). How much does your team spend on AI? Those jokes about "it being cheaper to hire a junior" may stop being jokes soon. :)
🔸 Bug Density (defects/LOE). AI helps us deliver more code, but it can also introduce more defects. The first goal is to make sure quality doesn't get worse.
🔸 Test Coverage (unit, integration, E2E). AI is very good at writing tests. Increasing test coverage across different levels is usually one of the easiest wins.
🔸 % of Toil Budget. How much engineering effort goes into routine work such as CI maintenance, vulnerability fixes, upgrades, and similar operational tasks? AI should gradually reduce this type of work.
🔸 Feature Delivery Rate (per sprint, release, or quarter). I'm personally skeptical about this metric. In R&D, one feature may take two days while another takes two months, so averages often tell you nothing. But for some teams it can be a useful indicator.
🔸 Time to Market. The average time to implement and deliver a feature. This is also difficult to measure in R&D, but it can be useful for teams who develop customer-facing features.
So don't focus on measuring AI. Measure the results of your work: cost, quality, and time against your baseline (baseline is the metrics value before AI adoption). If you want to show the value of AI in the future, you need to start collecting those metrics today.
#ai #engineering #ai4sdlc
Let's assume your team has already completed AI adoption and now management expects to see positive business results.
The good news is that AI doesn't magically change your business metrics. You don't need to invent new KPIs. You just need to track how your existing ones change (or finally start collecting them).
I suggest focusing on quality metrics first, then gradually shift toward delivery speed and cost optimization.
Since I lead platform engineering teams, we have two major types of work: new features development and L4 support. These activities are different, so they should be measured differently. Let's start with product development.
What can be measured:
🔸 Team Budget. The cost of the team in $, mandays, or FTEs.
🔸 AI Cost ($). How much does your team spend on AI? Those jokes about "it being cheaper to hire a junior" may stop being jokes soon. :)
🔸 Bug Density (defects/LOE). AI helps us deliver more code, but it can also introduce more defects. The first goal is to make sure quality doesn't get worse.
🔸 Test Coverage (unit, integration, E2E). AI is very good at writing tests. Increasing test coverage across different levels is usually one of the easiest wins.
🔸 % of Toil Budget. How much engineering effort goes into routine work such as CI maintenance, vulnerability fixes, upgrades, and similar operational tasks? AI should gradually reduce this type of work.
🔸 Feature Delivery Rate (per sprint, release, or quarter). I'm personally skeptical about this metric. In R&D, one feature may take two days while another takes two months, so averages often tell you nothing. But for some teams it can be a useful indicator.
🔸 Time to Market. The average time to implement and deliver a feature. This is also difficult to measure in R&D, but it can be useful for teams who develop customer-facing features.
So don't focus on measuring AI. Measure the results of your work: cost, quality, and time against your baseline (baseline is the metrics value before AI adoption). If you want to show the value of AI in the future, you need to start collecting those metrics today.
#ai #engineering #ai4sdlc
👍4❤2🔥2
AI Adoption: Measuring Support
Continuing the previous post, let's look at how AI adoption can be measured for support teams.
The principles are exactly the same: cost, quality, and time.
Here are the metrics I'd track:
🔸 Team Budget. Same as for development teams: the cost of the team in $, mandays, or FTEs.
🔸 AI Cost ($). How much does the team spend on AI, including autonomous agents if you're using them.
🔸 Incoming Load. The number of incoming tickets. At the beginning, this metric probably won't change much. But in the future it can show whether the overall quality is improving or getting worse. I recommend tracking it as a percentage change from the baseline before AI adoption.
🔸 Backlog Size. The number of non-resolved tickets. This metric should always be viewed together with the team budget and SLA.
🔸 Time to Resolution. How quickly customers receive a solution to their problem.
🔸 Reopen Rate. Has the percentage of reopened tickets increased? I've seen exactly this happen when teams introduced AI-based ticket assessment. Resolution time went down, but the number of reopened tickets increased several times. A clear sign that support quality had dropped.
As you can see, none of these metrics are AI-specific. Most tracking systems can calculate them out of the box, while a few may require collecting and analyzing historical data.
And one final point: never look at these metrics in isolation. Resolution time is decreased -> reopen rate doubles, team budget decreased -> backlog increased -> resolution time increased.
So it's the combination of metrics that tells you whether AI is actually improving support process or not.
#ai #engineering #ai4sdlc
Continuing the previous post, let's look at how AI adoption can be measured for support teams.
The principles are exactly the same: cost, quality, and time.
Here are the metrics I'd track:
🔸 Team Budget. Same as for development teams: the cost of the team in $, mandays, or FTEs.
🔸 AI Cost ($). How much does the team spend on AI, including autonomous agents if you're using them.
🔸 Incoming Load. The number of incoming tickets. At the beginning, this metric probably won't change much. But in the future it can show whether the overall quality is improving or getting worse. I recommend tracking it as a percentage change from the baseline before AI adoption.
🔸 Backlog Size. The number of non-resolved tickets. This metric should always be viewed together with the team budget and SLA.
🔸 Time to Resolution. How quickly customers receive a solution to their problem.
🔸 Reopen Rate. Has the percentage of reopened tickets increased? I've seen exactly this happen when teams introduced AI-based ticket assessment. Resolution time went down, but the number of reopened tickets increased several times. A clear sign that support quality had dropped.
As you can see, none of these metrics are AI-specific. Most tracking systems can calculate them out of the box, while a few may require collecting and analyzing historical data.
And one final point: never look at these metrics in isolation. Resolution time is decreased -> reopen rate doubles, team budget decreased -> backlog increased -> resolution time increased.
So it's the combination of metrics that tells you whether AI is actually improving support process or not.
#ai #engineering #ai4sdlc
👍2🔥2❤1
Agent Plugins Spec
Agent Plugins - a new specification intended to standardize how skills and MCP servers are packaged across different agent harnesses.
The spec is a result of collaboration between Cursor, Microsoft, OpenAI, Vercel, and AWS, and looks like an attempt to provide an alternative to the growing plugin ecosystem around Claude Code.
According to the spec, a plugin should have the following structure:
And that's basically where the specification ends.
There are no answers to questions like: How should users discover available plugins? How should plugins be distributed and installed? How to deliver a new version and roll it out to clients? All of that is left to each particular harness implementation.
In other words, Agent Plugins standardizes the package format, but not package management.
And this makes it difficult to operationalize across an organization. Moreover, Claude Code, for example, doesn't support it at all, and there is currently no obvious reason for Anthropic to adopt it.
Specifications are good. I actually really like them because they bring some order to the chaos of different integrations. But at the moment, Agent Plugins is far behind APM packages or the Claude Code plugins ecosystem.
Overall, it looks promising, but for now it's more something to watch than something you can build a sustainable process around. Let's see.
#ai #engineering
Agent Plugins - a new specification intended to standardize how skills and MCP servers are packaged across different agent harnesses.
The spec is a result of collaboration between Cursor, Microsoft, OpenAI, Vercel, and AWS, and looks like an attempt to provide an alternative to the growing plugin ecosystem around Claude Code.
According to the spec, a plugin should have the following structure:
my-plugin/
├── plugin.json
├── skills/
│ └── summarize/
│ ├── SKILL.md
│ ├── scripts/
│ └── references/
├── mcp.json
└── com.example.client/
└── hooks/
And that's basically where the specification ends.
There are no answers to questions like: How should users discover available plugins? How should plugins be distributed and installed? How to deliver a new version and roll it out to clients? All of that is left to each particular harness implementation.
In other words, Agent Plugins standardizes the package format, but not package management.
And this makes it difficult to operationalize across an organization. Moreover, Claude Code, for example, doesn't support it at all, and there is currently no obvious reason for Anthropic to adopt it.
Specifications are good. I actually really like them because they bring some order to the chaos of different integrations. But at the moment, Agent Plugins is far behind APM packages or the Claude Code plugins ecosystem.
Overall, it looks promising, but for now it's more something to watch than something you can build a sustainable process around. Let's see.
#ai #engineering
Agent Plugins
A portable package format for reusable components that extend AI agents.
👍2❤1🔥1👀1