Section name - QuickChat

One of the loudest conversations around AI has nothing to do with its capability. It is about ethics. 

As AI systems become more autonomous, the values they operate with are shaped by the humans who design, deploy, and train them. And because these systems learn in ways that scale quickly, ethical lapses at the beginning can grow into organisational risks later.

We speak with AI ethicist James Finister about readiness, misconceptions, trust, and what leaders need to get right early.

James Finister

AI Ethicist

“I'm that unusual beast, an AI ethicist. I help large organizations and small organizations think about ethical ways of governing AI in their organizations.”

• How prepared are today’s IT teams for AI?

Most organizations aren't ready for AI. They don't know what they want AI to do. They don't know how to evaluate whether or not AI is doing what they asked it to do. And when we move on to something as complex as agentic AI, they're certainly all at sea and prone to follow the herd, shall we say.

• What are the biggest ethical and trust-related hurdles to adopting agentic AI?

One of the big ones is that we simply don't know what the best use cases are yet in many organizations. And it's quite a scary thing to let agentic AI out in your organization unsupervised, potentially interacting with other agents who may be inside or outside your organization. And that includes other human beings. 

Are we actually happy with these agents interacting with our human counterparts, be that in our supplier chain, our customers, our users, the business, even external to our organization? That's a big one. Do we trust it enough to be able to allow it to do that? And do we actually have the confidence that we know what it should be doing? Some of them are technical in that organizations don't yet understand what's required to roll them out, particularly in the case in complex supply chains, SIAM, service integration management environments, where it may need to work across different vendors and different technical systems.

But all of it is about trust. Are organizations yet ready to place the amount of trust that you need to make effective use of agentic AI, where you effectively allow it to behave in the same way that you'd allow a member of your staff to behave?

• What sets agentic AI apart from automation when it comes to behaviour, judgement, and how it works with people?

You have to look at what agentic AI does, as opposed perhaps to the technology of it. I've got a non-technical definition of it. First, it has to be able to cope with multiple steps without human supervision - to look ahead and decide what chain of actions it's going to take. It's got to be able to explain its reasoning behind those choices, and it's got to be able to interact with both human agents and other AI agents seamlessly. It's also got to be able to work without a predetermined workflow. If we can program something for a predetermined workflow, then it's probably not agentic AI. And finally, it's got to be able to interact both with other human agents and other AI agents. 

• What ethical guardrails are essential for agentic AI?

Your existing guardrails have to be robust, because if not, because if not, agentic AI is going to break them. But I think a big area to look at here is probably the area of ethics and culture. We need to be honest about the fact that we’re using agentic AI, and the people are interfacing with an AI model and not a human being. We also need to be careful that we monitor how it behaves, and watch for behaviors that are unacceptable, such as being discriminatory.

Also be sure to not be overly optimistic. Don’t start telling people that you’re going to fix this incident in 30 seconds, when actually we all know it's going to take perhaps several hours to fix it. I would look at some of the cultural and ethical aspects as the main guardrails which are missing at the moment.

• What’s the safest first step for AI adoption?

Where I would start is those areas where you're fairly confident that your existing ways of working are good. Good AI makes things better, but bad AI makes things worse. So you want to apply AI where you're already reasonably competent as an organization. And for me, that's probably coordination of major instances behind the scenes. So actually doing a lot of running backwards and forwards between different vendor teams, between different technical teams to the service desk, particularly, rather than some of more outward facing communication towards the customer. And of course, it's also got a role to play in taking out the basic admin overhead of all the teams involved in IT service provision. 

• How will agentic AI change the skills teams need?

The workforce is going to need to be trained in how to use agentic AI successfully. I think in an ideal world, what we would see is still agentic AI in that co-pilot role, freeing up time, freeing us up from routine tasks potentially to let us do the things that human beings do well. What I don't think we'll see is agentic AI de-skilling the workforce. I think it will be an up-skilling exercise.

• Where do you see agentic AI adding value in the next five years?

We'll see it being used in outward facing roles and in more complex scenarios. I want to move it away from just being about IT failures but move into more managerial roles. For instance, can we use agentic AI to give us some basic information during the course of service management reviews. That would be an interesting area to go into.

• What advice would you give leaders who are planning AI adoption?

I think the number one piece would be to put in place measures in advance that you can actually measure and that will tell you whether or not it's been successful. Don't just put it in and hope that it's going to deliver results. Have some idea of the type of results you expect it to deliver. Experiment. Learn to walk before you learn to run with it.