5 Questions To Ask Before Implementing Gen AI

If the topic of Generative AI has been hogging an increasing share of your meeting agendas, you’re hardly alone. Since it was unleashed on the general public two years ago, GenAI has become the subject of endless news stories that seem to alternate between predictions of game-changing benefits for the modern enterprise and dire warnings of gloom and doom.

GenAI, even in its infancy, represents not merely an incremental step forward but a generational leap that might even genuinely warrant the hype and buzz it has engendered. Among its potential gifts, it promises to reduce product development cycles and enable innovation at scale, as well as automate all manner of laborious tasks, thus streamlining operations, reducing costs and freeing up humans to focus on strategic tasks.

It offers even small and midsize companies the ability to offer personalized and interactive customer experiences. “Amazon could always have written a great chatbot that feels value-added,” says Tariq Shaukat, co-CEO of Sonar, former president of Google Cloud and Bumble and board member at Gap and Public Storage. “Now, any company can get a chatbot that sounds like a human, more or less, to answer customer service questions. So, it really does level the playing field and create a lot of opportunity for midsize companies.”

But with the technology so new—and changing so rapidly—many CEOs are trying to get up to speed quickly and worried about launching projects whose risks have not all been identified. If you’re trying to get a better grasp on how GenAI will benefit your company here are some questions to pose to your executive team. 

1. How is GenAI aligned with our company’s strategic objectives? (Or, why are we doing this?)

As with all shiny new technologies, companies must resist the FOMO-induced urge to leap before they look and remember that any investment in AI needs a concrete business case. “You need some clarity around what the company is trying to achieve,” says David Garfield, global head of industries for AlixPartners. “Is it to generate insights, enhance productivity or generate incremental revenue—or maybe some combination of those things?”

Without a clear strategy, much capital will be wasted, says Richard Boyd, cofounder and CEO of AI company Tanjo and cofounder and CEO of Ultisim, a simulation learning company that utilizes gaming technology and AI. “I’ve already seen a lot of projects, and it reminds me of early ERP systems at the turn of the century, when people were implementing them, but they really weren’t ready yet. It created a lot of failed projects that cost tens of millions of dollars and were just disasters.”

Bob Rogers, CEO of supply chain AI company Oii and cofounder and chief scientific officer for BeeKeeperAI, observed something similar when he was chief data scientist at Intel. “I saw companies building huge data transformation projects, which ended up with huge negative ROIs because they were trying to do, basically, everything, everywhere, all at once rather than having one or two focused ROI opportunities.”

That said, most companies should be experimenting with the technology. Most experts recommend a small pilot project involving one group within the company. “Something you can trial and then very rigorously evaluate and test and learn from that experience, and then decide whether to expand or to modify before you take further steps,” says Garfield. “Then you look at KPIs and metrics, technical indicators of whether the software is performing the way it was intended.”

The pilot project should be contained “in a sandbox,” says Nate Thompson, founder of The Disrupted Workforce and cohost of its podcast. “It should be something that’s not connected to the broader corporate network or has a very limited connection so there could be no way that this technology could somehow expose a broader data set or in any way run wild on a corporate network.”

It’s a tightrope walk to find the balance between bleeding-edge adopter and missing-the-boat latecomer—but it’s a strategy CEOs and boards have to find together, says Herman Bulls, international director and vice chairman, Americas at JLL and board member at Host Hotels & Resorts, USAA and Comfort Systems USA. “You can’t go too fast, you can’t go too slow. It’s got to be just right.”

2. What are our policies around AI use?

Having clear policies around AI usage—who can use it, how and for what purpose—is critical for ensuring that the deployment of AI is aligned with the company’s values, mission and the expectations of stakeholders. By proactively addressing issues such as bias, fairness and accountability, the company can not only get ahead of regulatory scrutiny and legal challenges but also maintain public trust, a key asset in the digital age, where consumers and partners are increasingly concerned about data privacy, security and ethical implications.

“Every company needs to make their AI principles and acceptable-use policies incredibly explicit,” says Shaukat. “At Google Cloud, we listed it—here are the principles, and we will not do any deal that does not meet these principles.” Sonar’s policy not only dictates acceptable use cases but also spells out specifics on AI usage. “Like, you have to use this approved tool with this enterprise contract, etcetera. It can’t just be the Wild West.”

With the genie out of the bottle and apps like ChatGPT, Copilot and Jasper making GenAI increasingly ubiquitous on desktops and smartphones, formal policy has become that much more critical to safeguard the company against a host of risks that employees, perhaps unwittingly, invite. Cyberattacks are already a chief concern, says Gary LeDonne, board member with MVB Financial. “To me, this just kind of takes that to the next level. What algorithms can hackers develop to continually try different avenues into systems? That has always been a concern, and it’s an even bigger concern for me today.”

Reid Blackman, an AI ethics advisor and host of the podcast “Ethical Machines,” adds: “Make sure that your organization has an AI risk program that’s enterprise-wide, that systematically and comprehensively identifies and mitigates the risks of AI.” He adds that some of the known risks—bias, hallucinations, some privacy violations—are built into the technology. “It’s the nature of the beast. These risks are not mere possibilities—they’re probable.”

Ben Waber, CEO of Humanyze, cautions leaders to understand the weight of all the potential downsides of any AI project before going live. He points to the recent example of British package delivery company DPD’s public embarrassment after its AI chatbot swore at a customer and criticized its own company. “They probably lost way more than they saved from the amount they pay call-center workers. So, you need to understand the systemic risk that caused the problem. If you can’t answer those questions, doing something whole-hog seems incredibly foolish.”

3. Where is our data coming from?

As the saying goes, garbage in, garbage out. Data is the lifeblood of generative AI projects; it’s what fuels the intelligence and adaptability of these systems. Quality data is what enables AI to learn, discern patterns and make decisions. The data must be sourced ethically and legally, and structured for easy access, processing and analysis. Disorganized data can lead to inefficiencies or inaccuracies in learning by even the most sophisticated (and most expensive) large learning models (LLMs).

Before investing in any AI project, “make sure you have the infrastructure that will support it,” says EY’s former CEO Mark Weinberger, who sits on the boards of Johnson & Johnson, MetLife and Saudi Aramco. “Do we have our data in a way that we can access it and use it? Do we have the skillset, the software engineers? Are we partnering with others who do this? Do we have them lined up to help us to really understand and apply this new thinking that AI will provide? Those are the fundamentals you need before you have the end use case.”

While you don’t want to get too far into the weeds, you can ask for the sources of data that the system is being trained on, says Ann Skeet, senior director of leadership ethics at the Markkula Center for Applied Ethics and coauthor of Ethics in the Age of Disruptive Technologies: An Operational Roadmap. “[Directors] can also advise proactively choosing an AI system that has an identifiable training data set.”

Since most companies will acquire AI services rather than build from scratch in-house, those vendors need to be identified as potential sources of risk, says Paola Zeni, chief privacy officer for cloud communications company RingCentral. “If the strategy is to rely a lot on third-party AI, I would ‘double-click’ on what exactly have we done to vet those third parties, and what kind of criteria have we identified to vet them? And who is in charge of making sure that we have the right terms and conditions in place with that vendor?”

That includes understanding what happens to the data your company feeds the AI system, says Flavio Villanustre, global chief information security officer for LexisNexis Risk Solutions. “Depending on the contractual arrangement with the generative AI service provider, prompts used to interact with the models could be captured and used to further improve the system, which could lead to privacy or security issues.”

4. How are we addressing the potential impact of AI on our workforce?

Most experts agree that while some roles will no longer be needed in an AI world—just as the Industrial Revolution displaced blacksmiths and handloom operators—AI will not replace humans en masse any time soon. But it will create all new jobs to manage and interpret AI outcomes, as well as fuel a demand for soft skills like problem-solving, communication and emotional intelligence.

“If we do this right, by the time jobs are eliminated by AI, those people will be upskilled, reskilled and future skilled in a way that they’ve already pivoted to higher value and higher-impact tasks,” says Thompson, who recommends having a plan for growing talent as the fight for AI skills erupts into an all-out war and spoils go to those with the deepest coffers. Unlike more complex technologies, expertise with GenAI will be easier to learn. “There are people who can learn this if they have an affinity, if they have the right exposure. Do not wait and hope that you will find a unicorn. Start developing your talent now.”

In the meantime, it’s critical that this nascent technology never be allowed to take the wheel or be the last set of eyes on anything it creates. Waber offers an example of an HR department using an LLM to write the latest version of their employee handbook. “These models are trained to reduce the amount of sexualized content and so imagine that, because of that, it doesn’t output a section forbidding sexual harassment in the workplace,” he says. That omission might be missed if a human does not carefully read every word. Later on, if an employee does something out of line, you may not be able to fire them. “Now, are you going to be able to pin that on the fact that you used a large language model?” asks Waber. “Probably not.”

The key is to have AI take first, not last, crack at any task, says LeDonne. “We can let AI be the flag, and then those flags can go to those people who are skilled in fraud to make more judgment-based assessments. I would call data analytics an aid to decision-making, never the sole source.”

5. Who will own it?

For any serious implementation, experts say it’s essential to identify which individual or team will oversee AI implementation, identifying risks and opportunities and providing accountability for ethical standards, compliance and performance outcomes. Given AI’s far-flung consequences across the enterprise, identifying one person is not a simple exercise. RingCentral opted to create an AI governance council, which gathers leaders from across departments once a month to discuss every AI initiative in play.

“The purpose of the council is loosely to make sure that there is a shared awareness and understanding of what the company is doing with AI so that different teams could identify what it means for them,” says Zeni. “So, if I hear an update about how AI impacts a product strategy, I can immediately think, ‘okay, what are the potential legal implications and potential risks?’ Someone from the communication office would think, what are the communication opportunities or challenges around this strategy?” And so on. The agenda varies month to month but typically includes product updates “as well as updates on best practices or processes that have been introduced to manage, for instance, AI risks.”

One of the biggest benefits of the council meetings is that they have fostered collaboration between teams so that no opportunity, or risk, is missed. “It has created a lot of opportunities for a deeper dive,” Zeni says. “The discussions also trigger operational conversations at the lower level in the organization. What you ultimately want is not only alignment on the strategy but for the different organizations to not to be siloed, to work together.” Having a team focused on AI also ensures staying current on a technology with an exponential rate of change. 

It’s also key to benchmark and keep an eye on how other companies in your industry are using AI—but not copy, “because really, no one knows what they’re doing,” says Ben Waber, who is a visiting scientist at MIT Media Lab. “The floor is littered with horrible decisions that came from people just copying each other.”