AI transformation is a double-edged sword. Here's how to avoid the risks

A new year means a fresh focus on IT investments. Tech analyst Gartner predicts worldwide IT spending will hit $5.74 trillion in 2025, an increase of 9.3% from 2024. The analyst says explorations into generative AI (Gen AI) will help drive this rise.

How to use ChatGPT to write: Resumes | Excel formulas | Essays | Cover letters 
Most of us will have dabbled with Gen AI by now. Whether polishing text, creating photos, or generating code, the technology's capabilities can feel like magic.

You got to prove your hypothesis

However, James Fleming, CIO at the Francis Crick Institute, is one digital leader who isn't letting his organization get carried away by the hype.

He told ZDNET that using emerging technologies to power revolutionary scientific discoveries isn't straightforward. This challenge means the rise of Gen AI has not led to a major shift in working methods at his world-leading research organization.

Also: Your AI transformation depends on these 5 business tactics

"Doing scientific AI, as opposed to creating publicly available large language models, is quite a different discipline. You're operating in a tightly bounded scientific world where you've got to prove your hypothesis," he said.

"It's not good enough to say that you've thrown a model out there that's almost right. Most of the time, it's got to be exactly right under certain conditions. And you've got to demonstrate that you're basing your work on a fundamental understanding."

Fleming described the use of AI in science as a "double-edged sword." While emerging technology can help speed up the research process, any new conclusions must be generated and presented with a high degree of certainty.

"Provability is critical, particularly if you're thinking about something with a pathway to real-world impact in a clinic or as a medical device," said Fleming.

"If you're putting an innovation in front of a clinician and saying, 'I think this tool can predict the evolution of cancer,' for example, they're going to say, 'Can it? Show me why.'"

Also: 5 ways AI is changing baseball - and big data is up at bat

However, while explainability is critical to scientific research, it's a focus at odds with the black-box working methods and hallucinations of many popular AI models.

Use an iterative approach 

So, to demystify emerging technology processes, Fleming said the Crick uses an iterative approach to help its researchers embrace AI models confidently.

"You've got to work slowly and incrementally towards the goal," he said. "We take a much more focused approach that builds provenance and trust from day one."

The Crick's incremental approach helps researchers deploy AI models in two ways.

First, enhancing existing scientific methodologies. Fleming said the institute started its work here five years ago in microscopy.

Also: Integrating AI starts with robust data foundations. Here are 3 strategies executives employ

The Crick's microscopy facility analyzes cryogenic electrons and produces incredibly dense and beautiful images of tissues, cells, and individual molecules.

However, producing a nice picture is just the starting point. The image must be turned into data, such as details that can help show the differences between a cancerous cell and one that isn't.

Fleming's team has worked iteratively to prove that the right models can produce game-changing research results faster.

"AI models can do a lot of the grunt work for you, such as feature analysis and extraction and turning an image into data that you can work with to derive understanding," he said.

Also: How your business can best exploit AI: Tell your board these 4 things

The second area where Crick uses AI is in a discovery context, but a tightly bounded one.

Fleming gave the example of one lab's work on Parkinson's. The research team created a classifier that could identify which patients had the disease in a population of stem cells. However, the researchers couldn't explain why -- so they worked backward, iteratively.

"Having trained the model, there was then a process of reverse interrogating that model with various statistical methods to say, 'Actually, the dominant thing is the ellipticity of the cell. They're more oval. And there was also a whole set of other features that the model then extracted.'"

He said these results in isolation aren't the answer, but they do prompt the next line of inquiry: "'OK, cell morphology is different. Why is that? What's our next round of experimentation?' And that's where the iterative piece starts to come in."

Also: 4 ways to turn generative AI experiments into real business value

Test and hone

Fleming said the Crick's careful process of incrementally testing and honing AI technologies is now leading to a more sophisticated approach where multiple AIs are brought together to power trusted research programs.

The biggest project is led by Samra Turaljic, whose Cancer Dynamics Laboratory focuses on understanding how kidney cancer evolves.

The team uses AI to predict the genomic evolution of a tumor from pathology images. Fleming said that effort has involved training multiple AIs and cross-training models with genomic databases that span 10 years of research.

"The result is you create something that can clinically predict the evolution of the kidney," he said.

"But in each of those processes, you're both meticulously building up a sub-component to the point that you can trust it, and you're also working through layers and layers of data and getting closer and closer to real-world population scale as you work."

Also: The future of computing must be more sustainable, even as AI demand fuels energy use

Fleming said this super-detailed approach is important because results can be affected by biological and acquisitional variations, such as different intensities of a stain on a pathology slide.

This stage-by-stage process proves that the key to unlocking faster results from AI in the long term is working slowly and methodically in the shorter term.

That's a perspective we can all learn from in an age of AI, where vendor hyperbole suggests brilliant solutions to intractable challenges are just a click away.

"We start with a small data set, understand it, get it predictive and working, and bring in more data," said Fleming.

"Then, when we add more data, we hone the model again. This iterative process is critical because if you don't do it, you don't build the understanding and the provenance."