Mark Smalley on building a healthy relationship with AI

Quick chat 

⁠Mark Smalley has spent his career studying something he calls IT paradigmology: the way we think about IT, and why that thinking keeps needing to change. So when AI arrived, he wasn't just interested in the technology. He was interested in what happens between the technology and the people using it, and has written a book titled AI and the being between us.

⁠For Mark, the question was never just what AI can do. It's about the relationship between the people using it and the technology itself, and whether we're thinking about that correctly. To him, the question is not "how do we master AI?", but "how do we build a healthy relationship with it?" 

⁠In this conversation, he traces that idea through IT readiness, agentic risk, cognitive sovereignty, and where ITSM goes from here.

⁠What do most IT leaders get wrong about AI?

⁠That it can and should be mastered, otherwise it will master us. I think it's more about evolving with AI. You shouldn't think about AI as is, but about the relationship between the person using AI and AI. So instead of thinking in terms of mastery goals - like everybody should use AI for 50 % of their work - leaders should think about setting relationship goals, and whether every person is able to articulate when they trust AI or not.

⁠How should IT teams be thinking about their relationship with AI right now?

⁠Things are moving so quickly, hardly anyone is ready for AI but it is important to start thinking about it seriously - it’s like that story about outrunning the bear. You don't need to be faster than the bear, you just need to be faster than your competitors.

⁠I’d recommend looking at three areas: the IT department, the users, and the leadership. The IT department's going to be concerned about readiness for building and running AI safely and reliably, looking at technology, skills, data, privacy, security, those kinds of aspects. 

⁠Beyond that, it’s essential to think about the people using it. The key question there is, think, can your people work wisely and confidently with AI? Do they trust AI? Are they ready for using and benefiting from it? Where do you stand with AI literacy? It is equally important to pay attention to strategic questions around whether you can turn your AI capability in your firm to sustainable but also ethical value. Think about strategic alignment, governance, accountability, what kind of ethical stance are you taking on AI and do you have an ROI model that ties the efforts back into benefit. 

⁠What's the biggest risk that agentic AI introduces?

⁠Agentic AI represents a fundamental shift from reactive automation to proactive decision-making. Traditional automation follows rules, if then else. Agentic AI pursues goals, but you don't know how. And that autonomy is powerful, but it introduces risks, particularly accountability boundaries. When an agent makes a decision that you didn't anticipate using logic that you can't fully trace, who owns the outcome? Accountability is the biggest difference that agentic AI brings and is something IT teams must think about. 

⁠What does it actually take for IT teams to build enough trust in agentic AI to let it run?

⁠A big barrier that might prevent IT teams from adopting agentic AI is lack of trust. Autonomy is scary. Who gets blamed when it goes wrong? To tackle this, you need what’s called safe or bounded autonomy. You need the right cultural norms, you need governance structures, and technical guardrail. These help to start small in domains where the stakes are low with clear boundaries. So start off by developing experience with human-agentic collaboration before scaling to more critical systems. 

⁠If an IT leader wanted to dip their toes into agentic AI, where should they start?

⁠I'd go back to first principles. Look at where agentic AI acts well, where people are cognitively burdened with work that doesn't involve human strengths and AI weaknesses. You're trying to find the sweet spot between AI and human capabilities. We've got plenty of data to work on - repetitive work and ones where you've got feedback loops that are rich enough for continuous learning. These processes consume lots of cognitive energy from human beings, really rarely require the human empathy or moral reasoning that humans are good at. So that's probably a good test bed to play around with that concept of safe autonomy. So start in AI's sweet spot and leave people in their sweet spot.

⁠Before an IT leader hands any autonomy over to an AI agent, what questions do they need to have answered?

⁠I focus on bounded or safe autonomy, safe agents. Before you roll out any agentic AI, create a one page document that answers questions like, what can this agent do, what must remain human, how will we know if it's working, who owns the outcomes, etc. Accountability is a big issue. The bigger aspect to think about will be how we will retain our cognitive sovereignty.  That's a topic to think about. How do you retain your cognitive sovereignty and not outsource it to AI?

⁠If cognitive erosion is a real risk, what does that mean for how IT leaders develop and protect their teams?

⁠IT leaders will need to develop skills that will turn them into what I call stewards of cognitive sovereignty. If you don't actively protect human cognitive vitality, you risk operational failure due to over-reliance on automation, indefensible decisions due to eroded human judgment, diminished innovation due to loss of original human thinking, and ethical negligence due to failing to intervene as leaders as human capability quietly erodes. And this all results in reputational damage which you simply can't afford. It is important for IT leaders to not just look at it mastering the machine but staying fully present in how we use it. You can allow your employees to become zombies, or you can help them develop what truly makes them human.

⁠Where does ITSM go from here as agentic AI takes hold?

⁠Over the next five years, agentic AI could trigger the emergence in IT service management of a new role as relationship therapist. And that might sound totally weird, but think about what relationship therapists actually do. They help both parties understand each other and develop healthy relationships. And I think that's just what ITSM needs to become. Not here's your AI or let me configure that for you, but let me help you, the users, let me help you develop a healthy, productive relationship with these intelligent systems. It's not just the tool, it's the interaction.