I really don't understand how people given access to a pile of tools and data sources and unleash them on customers. It's horrible UX in my experience and at times worse than a phone tree.
My view is that you need to transition slowly and carefully to AI first customer support.
1. Know the scope of problems an AI can solve with high probability. Related prompt: "You can ONLY help with the following issues."
2. Escalate to a human immediately if its out of scope: "If you cannot help, escalate to a human immediately by CCing bob@smallbiz.co"
3. Have an "unlocked agent" that your customer service person can use to answer a question and evaluate how well the agent performs in helping. Use this to drive your development roadmap.
4. If the "unlocked agent" becomes good at solving a problem, add that to the in-scope solutions.
Finally, you should probably have some way to test existing conversations when you make changes. (It's on my TODO list)
I've implemented this for a few small businesses, and the process is so seamless that no one has suspected interaction with an AI. For one client, there's not even a visible escalation step: they get pinged on their phone and take over the chat!
The purpose of customer support is to convince the customer that it is not worth their time to pursue support. A worse experience achieves that goal faster.
Using GenAI is a huge breakthrough in this field, because it is a socially acceptable way to tell someone you don't care about their issue.
You've articulated it better than I could. I think, reading through this author's post, they've misunderstood the objectives.
The purpose has been achieved, in that there is a large drop rate. The product manager has met their goals, cut costs, and might be looking forward to their bonus.
It would be far more expensive to make the LLM behave effectively than it would be to do nothing. Any product manager that sincerely cared about customer support wouldn't be inflicting a personalised callous disregard for service. Instead they'd be focusing on improving documentation, help, and processes. But that's not innately quantifiable in a way that leads to bonuses, and therefore goes unnoticed.
A lot of the agent tools/frameworks don't dare to have an agent on the site to answer user questions. For those who dares, it sucks. eg. Mastra.ai is supposed to be a framework for building agents but their agent on the website cannot answer any question ( i asked ~20 questions and got 0 satisfactory answers)
>> really don't understand how people given access to a pile of tools and data sources and unleash them on customers
It’s pretty simple. When a non-tech person sees faked demos of what it can do - it looks epic and everyone extrapolates results and thinks AI is that good.
Nice framing for PMs, but technically it is way too rosy. MCP is real but still full of low utility services and security issues, so “skills as plug-ins” is not production ready. A2A protocols were only just announced this year (Google, etc.) and actual inter-agent interoperability is still research grade, with debugging across agents being a nightmare. Orchestration layers (skills, workflows, multi-agent) look clean in diagrams but turn into brittle state machines under load. LLM “confidence scores” are basically uncalibrated logits dressed up as probabilities.
In short: nice industry roadmap, but we are nowhere near robust, trustworthy multi-agent systems yet.
The idea of giving an LLM with a tool any kind of control over an actual user's account remains (though you put this more elegantly) batshit insane to me.
Even assuming you've correctly auth'd the user contacting you (big assumption!), allowing that user to very literally prompt a 'semi-confident thing with tools' - however many layers of abstraction away the tool is - feels very, very far away from a real-world, sensible implementation right now.
Just shoot the tool prompts over to a human operator, if it's so necessary! Sense-check!
I MVP'd one of these (a basic sequence of LLM customer support 'agents') at my last job, I guess spring 2024. So much has changed since then!
'Routing through increasingly specialised agents' was my approach, and the only thing that would've done the job (in MVP form) at the time. There weren't many models that would fit our (v good) CS & Product teams' dataset of "probable queries from customers" into a single context window.
I never personally got my MVP beyond sitting with it beside the customer support inbox, talking to customers. And AFAIK it never moved beyond that after I left.
Nor should it have been, probably - there are (wild, & mostly ineffable) trade-offs that you make the moment you stop actually talking to users at the very moment they get in touch. I don't remember ever making a trade-off like that where it was worthwhile.
I _do_ remember it as perhaps the most worthwhile time I ever spent doing product-y work.
I say that because: To consider a customer support query type that might be 0.005% of all queries received by the CS team, even my trash MVP had to walk a path down a pretty intricate tree of agents and possible query types.
So - if you believe that 'solving the problems users have with your product' = 'making a better product'. then talking to an LLM that was an advocate for a tiny subset of users, and knew very intimately the details of their issue with your product, that felt really good. It felt like it was a very pure version of what _I_ should be to devs, as any kind of interface between them and our users.
It was very hard to stay a believer in the idea of a 'PM' after seeing that, at least. As a person who preferred to just let people get on with things.
I enjoyed the linked post; it's really interesting to see how far things have come. I'm surprised nobody has built 'talk to your customers at scale', yet - this feels like a far more interesting problem than 'avoid talking to your customers at scale'.
I'm also not surprised, I guess, since it's an incredibly bespoke job to do properly, I imagine, for most products.
> I enjoyed the linked post; it's really interesting to see how far things have come. I'm surprised nobody has built 'talk to your customers at scale', yet - this feels like a far more interesting problem than 'avoid talking to your customers at scale'.
This sounds hard to pull off in a very similar way to getting good data through surveys.
I generally don't want to talk to my tools. If I'm motivated to talk to you, it's probably because something went wrong. And even if I talked to you when not annoyed, I'd struggle to articulate more than "it's working good" at any given moment - when what you really want as a product person is to know "it's working good, but I had to internalize this workaround for something for my use case that no I don't even think about but originally I found offputting and almost bounced because of" or whatever.
The author's inner PM comes out here and makes some wild claims. Calibration is something we can do with traditional, classification models, but not with most off-the-shelf LLMs. Even if you devised a way to determine if the LLM's confidence claim matched it's actual performance, you wouldn't be able to calibrate or tune it like you would a more traditional model.
I'm typically pretty critical of PM oriented pieces, but I found this to be a decent overview of how to reason about building these systems from first principles + some of the non-tech pain points + how to address them.
reading this as a security engineer trying to get ahead of misguided PMs who buy into the AI hype and don't know 1) it's immature 2) it's not secure & 3) whether their business use case is viable for the R&D we're about to put into it.
I get the feeling there's going to be either 1) a great revert of the features, 2) a bunch of hurried patches, or 3) a bunch of legacy systems operating on MCP v0.00-beta (metaphorically speaking)
We need to take the focus off cost savings. None of this tech is anywhere near mature enough to replace humans yet.
Far better to focus on enhancing human capabilities with agents.
For example while a human talks to a customer on the phone, AI is fetching useful context about the customer and suggesting talking points to improve the human conversation.
One example of a direct benefit for business using AI this way is reducing onboarding times for new employees
What does the PM title even mean at this point? It's a bit surprising to see a deep dive into technical architecture - though there is massive value in understanding what's involved - as a PM responsibility, this is more in TPM (technical program manager) land which is a different job.
In my book they ideally focus on understanding scope, user needs and how to measure success, while implementation details such as orchestration strategies, evaluation and making sure your system delivers the capabilities you want in general, are engineering responsibilities.
There are bad PMs and good PMs, and bad engineers and good engineers. If you treat an entire profession with disdain, don’t be surprised if you get treated like that too.
I know you probably feel you're being fair, but you're not.
There's a dichotomy in development where bad PMs can prosper in a way bad engineers can't.
There's no skill test for PMs, unlike engineers. Bad PMs can look like good PMs to senior management simply because they hold tons of meetings, kiss ass, over promise or steal credit. Any of those bad traits can fool senior management. But those are bad PMs.
On top of that, when you have a bad PM, there's a good chance the Devs themselves will step into the role and still deliver a product.
The bad PM will still take credit, obviously. A bad PM is often circumvented instead of exposed.
Conversely the opposite doesn't work, a good PM + bad Devs turns into never ending dev cycles. The PM looks bad even though there's nothing he can really do, unless he can fire/hire. The good PM cannot circumvent bad engineers.
And in the end, to find bad engineers you can just look at their code. If you don't have the skill to do that, or don't employ someone you know that can, you probably shouldn't be in the software development business.
Well sure, I never said they were equivalent in all respects. Just that you can have good and bad versions of both. For sure the failure modes are different.
I challenge the idea that there is no skill test for PMs, though - take a PM interview at a serious product company some day.
And the PM role is of course more than just delivery. If they dropped dead the product would still get shipped. But then what? Someone would need to talk to customers, dig into data and figure out the roadmap. Other people can do it, but in a sufficiently complex company you might as well get people who are good at it and want to devote their time to it.
I understand why some engineers don’t like PMs. But it is exactly the same reason as why some PMs (and C-suites) view engineers as fungible resources who waste time on abstractions instead of shipping, and pad estimates and refuse to discuss practical tradeoffs to move quicker - it’s an unfair generalisation based on bad experiences.
I just think more respect all around wouldn’t hurt.
That's not a skill test, or you'd point to the "test" instead of telling me to interview for a job role. That's a skilled judgement.
It is a generalization, but it's not unfair. That's the mistake you're making. Is it "unfair" to call the British people Roast Beef, or calling French people Froggies. Those are generalizations but are fair (or were at least). British people genuinely eat a disproportionate amount of Roast Beef and French people genuinely eat Frogs legs.
And there are genuinely more bad PMs than good ones and lots of developers have experience "managing" their PM and trying to ensure they don't do too much harm, like the GP that started this discussion.
Don't worry, most engineers will quickly realize when a PM is good and let them do their job without "managing" them. In fact, it's a delight working with one as they do genuinely make the dev process so much better.
I’m really not sure what you’re arguing. You want a precise test for being a good PM, that can be marked like an examination with correct and incorrect answers? It’s not engineering - it’s a role largely to do with learning and measuring and facilitating streams of work across multiple different (highly opinionated!) types of professionals - user researchers, engineers, designers, marketers, copywriters, data scientists - all of whose expertise is needed to ensure good outcomes. The fact that it can’t be measured as a multiple choice test doesn’t mean it isn’t skill. But if you really want to go down that route, then you’d ask a PM to explain some ways of proving the value of a potential feature, or the different ways to prioritise a roadmap, or how to manage challenging stakeholders, or indeed how to get good outcomes from colleagues who insist that only they are the people with any kind of skill…
Don’t worry, PMs are also used to working with engineers who view their profession as the only special one. Managing that is part of how to get good outcomes.
If you’ve mainly encountered bad PMs, then hey I’m sorry for you. Find somewhere to work with better colleagues?
But you’ll not convince me that one profession is just inherently better than another. That’s silly, and speaks to a lack of empathy that is, if you’re still looking for a checkbox test for the role, the type of thing that would cause you to fail it immediately.
Every PM I've ever met has been a loser who has failed in their primary desired career and then made the switch to project management. I have no respect for PMs whatsoever.
Stop trying to treat these things as more than they are. Stop trying to be clever. These models are the single most complex things ever created by humans; the summation of decades of research, trillions in capex, and the untold countless hours of thousands of people smarter than you and I. You will not meaningfully add to their capabilities with some hacked together reasoning workflows. Work within the confines of what they can actually do; anything else is complete delusion.
This is a nonsensical opinion by a person who doesn't know what they're talking about, and probably didn't read the article.
These models are tools, and LLM products bundles these tools with other tools, and 90% of UX amounts to bundling these well. The article here gives a great sense of what this takes.
The AI bundling problem is over. The user interface problem is over. You won't need a UI for your apps in a few years, agents are going to drive _EVERYTHING_. If you want a display for some data, the agent will slap together a dashboard on the fly from a composable UI library that's easy to work with, all hot loaded and live-revised based on your needs.
I use agents to do so much stuff on my computer, MCPs are easy to roll so you can give them whatever powers you want. Being able to just direct agents to do stuff on my computer via voice is amazing. The direct driving still sucks so they're not a general UI yet, and the models need to be a bit more consistent/smarter in general, but it'll be there very soon.
I use them as an intelligence layer over disk cleanup tools, to manage deployments/cloud configs, I have big repo organization workflows, they can manage my KDE system settings, I use them as editors on documents all over my filesystem (to add comments for revision, not to rewrite, that's not consistent enough), I use them to do deep research on topics and save reports, to look at my google analytics and seo data and suggest changes to my pages. Frankly if I had my druthers I wouldn't use a mouse, the agent would use visual tracking (eye/hand) along with words and body language to just quickly figure out what I want.
My claim is that the "useful assistant for menial tasks" is the Wright brothers flyer to what we'll have in a few years. If you have voice chat with an agent on your phone that can just do everything you'd need an app for, what's the point of an app? And it's gonna happen, because if your app doesn't let people's agents handle their business and your competitors' does, people are gonna switch if they can. The computer interfaces of the future are going to be made for agents first.
> My claim is that the "useful assistant for menial tasks" is the Wright brothers flyer to what we'll have in a few years.
I agree with that.
But what you originally wrote was, "The AI bundling problem is over. The user interface problem is over." It would probably make more sense to say "...will be over."
People tend to be sensitive to those kinds of claims because there's a lot of hype around all this at the moment. So when people seem to imply that what we have right now is much more capable than it actually is, there tends to be pushback.
Except that the main blocker on the star trek computer is the hooks we wire into the agent to manage the computer. Current gen models are almost smart enough, though their long context support and ability to use tools are a little shaky in general (I have walked a lot of agents through using tools, correct shell command use needs more RL for sure). None of this is outlandish advances, it's all just the natural progression of the track we're on.
I won't use agents for everything. Why would I expect tasks to use agents for everything? This is like saying everything is on the web. No, there is substantial number of things on the web, but not everything.
Who maintains that UI library? Or does the AI create it on the fly too? Why even bother with a library at that point? Just do a bespoke implementation.
The library will exist to maintain high quality/consistency and reduce load times. Also, it's faster to generate a page with parameterized components than to recreate all the components. It's a win all around from an engineering perspective, and nobody has to maintain them, there could be an artifact registry where people publish their components and you or AI can just select nice ones for the given use case.
A widget != a UI. I don't need a stripe app, but things like visualizations are still useful. I want to be able to pull up a graph of my sales on stripe over the last 72 hours using a specific type of plot, cross referenced with my promotions in a dashboard side by side with consistent colors so it's easy to scan. The agent will be able to pull high quality plots of the right type that theme according to my preferences and slot into my dashboard neatly, and I won't have to hassle with stripe or my adtech or analytics or any of that except to configure the agent.
I have a hard time determining if you are in support or critiquing the article. I'm 60% confident it is a critique (I jest, a play on the content :) ).
I really don't understand how people given access to a pile of tools and data sources and unleash them on customers. It's horrible UX in my experience and at times worse than a phone tree.
My view is that you need to transition slowly and carefully to AI first customer support.
1. Know the scope of problems an AI can solve with high probability. Related prompt: "You can ONLY help with the following issues."
2. Escalate to a human immediately if its out of scope: "If you cannot help, escalate to a human immediately by CCing bob@smallbiz.co"
3. Have an "unlocked agent" that your customer service person can use to answer a question and evaluate how well the agent performs in helping. Use this to drive your development roadmap.
4. If the "unlocked agent" becomes good at solving a problem, add that to the in-scope solutions.
Finally, you should probably have some way to test existing conversations when you make changes. (It's on my TODO list)
I've implemented this for a few small businesses, and the process is so seamless that no one has suspected interaction with an AI. For one client, there's not even a visible escalation step: they get pinged on their phone and take over the chat!
The purpose of customer support is to convince the customer that it is not worth their time to pursue support. A worse experience achieves that goal faster.
Using GenAI is a huge breakthrough in this field, because it is a socially acceptable way to tell someone you don't care about their issue.
You've articulated it better than I could. I think, reading through this author's post, they've misunderstood the objectives.
The purpose has been achieved, in that there is a large drop rate. The product manager has met their goals, cut costs, and might be looking forward to their bonus.
It would be far more expensive to make the LLM behave effectively than it would be to do nothing. Any product manager that sincerely cared about customer support wouldn't be inflicting a personalised callous disregard for service. Instead they'd be focusing on improving documentation, help, and processes. But that's not innately quantifiable in a way that leads to bonuses, and therefore goes unnoticed.
A lot of the agent tools/frameworks don't dare to have an agent on the site to answer user questions. For those who dares, it sucks. eg. Mastra.ai is supposed to be a framework for building agents but their agent on the website cannot answer any question ( i asked ~20 questions and got 0 satisfactory answers)
>> really don't understand how people given access to a pile of tools and data sources and unleash them on customers
It’s pretty simple. When a non-tech person sees faked demos of what it can do - it looks epic and everyone extrapolates results and thinks AI is that good.
Doubly so if the person deciding what gets implemented doesn't really get what their staff actually do.
LLMs ability to give convincing sounding answers is like catnip for service desk managers who have never actually been on the desk itself
Nice framing for PMs, but technically it is way too rosy. MCP is real but still full of low utility services and security issues, so “skills as plug-ins” is not production ready. A2A protocols were only just announced this year (Google, etc.) and actual inter-agent interoperability is still research grade, with debugging across agents being a nightmare. Orchestration layers (skills, workflows, multi-agent) look clean in diagrams but turn into brittle state machines under load. LLM “confidence scores” are basically uncalibrated logits dressed up as probabilities.
In short: nice industry roadmap, but we are nowhere near robust, trustworthy multi-agent systems yet.
The idea of giving an LLM with a tool any kind of control over an actual user's account remains (though you put this more elegantly) batshit insane to me.
Even assuming you've correctly auth'd the user contacting you (big assumption!), allowing that user to very literally prompt a 'semi-confident thing with tools' - however many layers of abstraction away the tool is - feels very, very far away from a real-world, sensible implementation right now.
Just shoot the tool prompts over to a human operator, if it's so necessary! Sense-check!
I MVP'd one of these (a basic sequence of LLM customer support 'agents') at my last job, I guess spring 2024. So much has changed since then!
'Routing through increasingly specialised agents' was my approach, and the only thing that would've done the job (in MVP form) at the time. There weren't many models that would fit our (v good) CS & Product teams' dataset of "probable queries from customers" into a single context window.
I never personally got my MVP beyond sitting with it beside the customer support inbox, talking to customers. And AFAIK it never moved beyond that after I left.
Nor should it have been, probably - there are (wild, & mostly ineffable) trade-offs that you make the moment you stop actually talking to users at the very moment they get in touch. I don't remember ever making a trade-off like that where it was worthwhile.
I _do_ remember it as perhaps the most worthwhile time I ever spent doing product-y work.
I say that because: To consider a customer support query type that might be 0.005% of all queries received by the CS team, even my trash MVP had to walk a path down a pretty intricate tree of agents and possible query types.
So - if you believe that 'solving the problems users have with your product' = 'making a better product'. then talking to an LLM that was an advocate for a tiny subset of users, and knew very intimately the details of their issue with your product, that felt really good. It felt like it was a very pure version of what _I_ should be to devs, as any kind of interface between them and our users.
It was very hard to stay a believer in the idea of a 'PM' after seeing that, at least. As a person who preferred to just let people get on with things.
I enjoyed the linked post; it's really interesting to see how far things have come. I'm surprised nobody has built 'talk to your customers at scale', yet - this feels like a far more interesting problem than 'avoid talking to your customers at scale'.
I'm also not surprised, I guess, since it's an incredibly bespoke job to do properly, I imagine, for most products.
> I enjoyed the linked post; it's really interesting to see how far things have come. I'm surprised nobody has built 'talk to your customers at scale', yet - this feels like a far more interesting problem than 'avoid talking to your customers at scale'.
This sounds hard to pull off in a very similar way to getting good data through surveys.
I generally don't want to talk to my tools. If I'm motivated to talk to you, it's probably because something went wrong. And even if I talked to you when not annoyed, I'd struggle to articulate more than "it's working good" at any given moment - when what you really want as a product person is to know "it's working good, but I had to internalize this workaround for something for my use case that no I don't even think about but originally I found offputting and almost bounced because of" or whatever.
> Confidence calibration: When your agent says it's 60% confident, it should be right about 60% of the time. Not 90%, not 30%. Actual 60%.
With current technology (LLM), how can an agent ever be sure about its confidence?
The author's inner PM comes out here and makes some wild claims. Calibration is something we can do with traditional, classification models, but not with most off-the-shelf LLMs. Even if you devised a way to determine if the LLM's confidence claim matched it's actual performance, you wouldn't be able to calibrate or tune it like you would a more traditional model.
I was about to say "Using calibrated models", then I found this interesting paper:
Calibrated Language Models Must Hallucinate
https://arxiv.org/abs/2311.14648
https://www.youtube.com/watch?v=cnoOjE_Xj5g
I'm typically pretty critical of PM oriented pieces, but I found this to be a decent overview of how to reason about building these systems from first principles + some of the non-tech pain points + how to address them.
As an engineer, I like this framework but can think of approximately zero PMs who could use it to build a product.
reading this as a security engineer trying to get ahead of misguided PMs who buy into the AI hype and don't know 1) it's immature 2) it's not secure & 3) whether their business use case is viable for the R&D we're about to put into it.
I get the feeling there's going to be either 1) a great revert of the features, 2) a bunch of hurried patches, or 3) a bunch of legacy systems operating on MCP v0.00-beta (metaphorically speaking)
:lol_sob:
We need to take the focus off cost savings. None of this tech is anywhere near mature enough to replace humans yet.
Far better to focus on enhancing human capabilities with agents.
For example while a human talks to a customer on the phone, AI is fetching useful context about the customer and suggesting talking points to improve the human conversation.
One example of a direct benefit for business using AI this way is reducing onboarding times for new employees
What does the PM title even mean at this point? It's a bit surprising to see a deep dive into technical architecture - though there is massive value in understanding what's involved - as a PM responsibility, this is more in TPM (technical program manager) land which is a different job.
In my book they ideally focus on understanding scope, user needs and how to measure success, while implementation details such as orchestration strategies, evaluation and making sure your system delivers the capabilities you want in general, are engineering responsibilities.
This post does not do a deep dive into technical architecture.
When it comes to LLMs and agents, this is the technical architecture: orchestration patterns, memory, context handling, MCP, etc.
The PM's role is to whip devs until the requirements are met. That seems apt here. Even if the requirements make zero sense
[flagged]
There are bad PMs and good PMs, and bad engineers and good engineers. If you treat an entire profession with disdain, don’t be surprised if you get treated like that too.
I know you probably feel you're being fair, but you're not.
There's a dichotomy in development where bad PMs can prosper in a way bad engineers can't.
There's no skill test for PMs, unlike engineers. Bad PMs can look like good PMs to senior management simply because they hold tons of meetings, kiss ass, over promise or steal credit. Any of those bad traits can fool senior management. But those are bad PMs.
On top of that, when you have a bad PM, there's a good chance the Devs themselves will step into the role and still deliver a product.
The bad PM will still take credit, obviously. A bad PM is often circumvented instead of exposed.
Conversely the opposite doesn't work, a good PM + bad Devs turns into never ending dev cycles. The PM looks bad even though there's nothing he can really do, unless he can fire/hire. The good PM cannot circumvent bad engineers.
And in the end, to find bad engineers you can just look at their code. If you don't have the skill to do that, or don't employ someone you know that can, you probably shouldn't be in the software development business.
Well sure, I never said they were equivalent in all respects. Just that you can have good and bad versions of both. For sure the failure modes are different.
I challenge the idea that there is no skill test for PMs, though - take a PM interview at a serious product company some day.
And the PM role is of course more than just delivery. If they dropped dead the product would still get shipped. But then what? Someone would need to talk to customers, dig into data and figure out the roadmap. Other people can do it, but in a sufficiently complex company you might as well get people who are good at it and want to devote their time to it.
I understand why some engineers don’t like PMs. But it is exactly the same reason as why some PMs (and C-suites) view engineers as fungible resources who waste time on abstractions instead of shipping, and pad estimates and refuse to discuss practical tradeoffs to move quicker - it’s an unfair generalisation based on bad experiences.
I just think more respect all around wouldn’t hurt.
That's not a skill test, or you'd point to the "test" instead of telling me to interview for a job role. That's a skilled judgement.
It is a generalization, but it's not unfair. That's the mistake you're making. Is it "unfair" to call the British people Roast Beef, or calling French people Froggies. Those are generalizations but are fair (or were at least). British people genuinely eat a disproportionate amount of Roast Beef and French people genuinely eat Frogs legs.
And there are genuinely more bad PMs than good ones and lots of developers have experience "managing" their PM and trying to ensure they don't do too much harm, like the GP that started this discussion.
Don't worry, most engineers will quickly realize when a PM is good and let them do their job without "managing" them. In fact, it's a delight working with one as they do genuinely make the dev process so much better.
I’m really not sure what you’re arguing. You want a precise test for being a good PM, that can be marked like an examination with correct and incorrect answers? It’s not engineering - it’s a role largely to do with learning and measuring and facilitating streams of work across multiple different (highly opinionated!) types of professionals - user researchers, engineers, designers, marketers, copywriters, data scientists - all of whose expertise is needed to ensure good outcomes. The fact that it can’t be measured as a multiple choice test doesn’t mean it isn’t skill. But if you really want to go down that route, then you’d ask a PM to explain some ways of proving the value of a potential feature, or the different ways to prioritise a roadmap, or how to manage challenging stakeholders, or indeed how to get good outcomes from colleagues who insist that only they are the people with any kind of skill…
Don’t worry, PMs are also used to working with engineers who view their profession as the only special one. Managing that is part of how to get good outcomes.
If you’ve mainly encountered bad PMs, then hey I’m sorry for you. Find somewhere to work with better colleagues?
But you’ll not convince me that one profession is just inherently better than another. That’s silly, and speaks to a lack of empathy that is, if you’re still looking for a checkbox test for the role, the type of thing that would cause you to fail it immediately.
Here here. Well said
Well said!
PMs that can hire/fire are pretty common, but again how do they know who?
This is silly; PM is a more broad role than SWE.
Every PM I've ever met has been a loser who has failed in their primary desired career and then made the switch to project management. I have no respect for PMs whatsoever.
Ouch! So who will do the work of the PM in the team? The engineers?
Stop trying to treat these things as more than they are. Stop trying to be clever. These models are the single most complex things ever created by humans; the summation of decades of research, trillions in capex, and the untold countless hours of thousands of people smarter than you and I. You will not meaningfully add to their capabilities with some hacked together reasoning workflows. Work within the confines of what they can actually do; anything else is complete delusion.
This is a nonsensical opinion by a person who doesn't know what they're talking about, and probably didn't read the article.
These models are tools, and LLM products bundles these tools with other tools, and 90% of UX amounts to bundling these well. The article here gives a great sense of what this takes.
> This is a nonsensical opinion by a person who doesn't know what they're talking about, and probably didn't read the article.
Ok, but can you please make your substantive points without putting others down? Your comment wouold be fine without this bit.
https://news.ycombinator.com/newsguidelines.html
The AI bundling problem is over. The user interface problem is over. You won't need a UI for your apps in a few years, agents are going to drive _EVERYTHING_. If you want a display for some data, the agent will slap together a dashboard on the fly from a composable UI library that's easy to work with, all hot loaded and live-revised based on your needs.
You must be an easy person to market to.
I use agents to do so much stuff on my computer, MCPs are easy to roll so you can give them whatever powers you want. Being able to just direct agents to do stuff on my computer via voice is amazing. The direct driving still sucks so they're not a general UI yet, and the models need to be a bit more consistent/smarter in general, but it'll be there very soon.
What do you do with agents?
I use them as an intelligence layer over disk cleanup tools, to manage deployments/cloud configs, I have big repo organization workflows, they can manage my KDE system settings, I use them as editors on documents all over my filesystem (to add comments for revision, not to rewrite, that's not consistent enough), I use them to do deep research on topics and save reports, to look at my google analytics and seo data and suggest changes to my pages. Frankly if I had my druthers I wouldn't use a mouse, the agent would use visual tracking (eye/hand) along with words and body language to just quickly figure out what I want.
> they can manage my KDE system settings
Why do you even have KDE installed if AI has replaced GUIs?
You’re saying you’ve found a useful assistant for menial tasks. That’s not consistent with the strong claims you were making upthread.
My claim is that the "useful assistant for menial tasks" is the Wright brothers flyer to what we'll have in a few years. If you have voice chat with an agent on your phone that can just do everything you'd need an app for, what's the point of an app? And it's gonna happen, because if your app doesn't let people's agents handle their business and your competitors' does, people are gonna switch if they can. The computer interfaces of the future are going to be made for agents first.
> My claim is that the "useful assistant for menial tasks" is the Wright brothers flyer to what we'll have in a few years.
I agree with that.
But what you originally wrote was, "The AI bundling problem is over. The user interface problem is over." It would probably make more sense to say "...will be over."
People tend to be sensitive to those kinds of claims because there's a lot of hype around all this at the moment. So when people seem to imply that what we have right now is much more capable than it actually is, there tends to be pushback.
The Juicero moment for software
Tell me you don't want to go hands free and have the star trek computer do everything for you. We could be there in ~5 years.
We also could have warp drives next year!
Except that the main blocker on the star trek computer is the hooks we wire into the agent to manage the computer. Current gen models are almost smart enough, though their long context support and ability to use tools are a little shaky in general (I have walked a lot of agents through using tools, correct shell command use needs more RL for sure). None of this is outlandish advances, it's all just the natural progression of the track we're on.
You’re either a decent troll, or absolutely delusional.
I genuinely do not want this, it sounds like shit
I won't use agents for everything. Why would I expect tasks to use agents for everything? This is like saying everything is on the web. No, there is substantial number of things on the web, but not everything.
why would anyone want more non-determinism than absolutely necessary?
Who maintains that UI library? Or does the AI create it on the fly too? Why even bother with a library at that point? Just do a bespoke implementation.
The library will exist to maintain high quality/consistency and reduce load times. Also, it's faster to generate a page with parameterized components than to recreate all the components. It's a win all around from an engineering perspective, and nobody has to maintain them, there could be an artifact registry where people publish their components and you or AI can just select nice ones for the given use case.
Why are people publishing their components when the UI problem is over and no one builds UIs anymore?
A widget != a UI. I don't need a stripe app, but things like visualizations are still useful. I want to be able to pull up a graph of my sales on stripe over the last 72 hours using a specific type of plot, cross referenced with my promotions in a dashboard side by side with consistent colors so it's easy to scan. The agent will be able to pull high quality plots of the right type that theme according to my preferences and slot into my dashboard neatly, and I won't have to hassle with stripe or my adtech or analytics or any of that except to configure the agent.
Contrary to my other comment, I 100% agree to this.
I have a hard time determining if you are in support or critiquing the article. I'm 60% confident it is a critique (I jest, a play on the content :) ).