Generative AI Business Opportunities, Risks and Ethics: Panel
At COLLIDE 2023, a panel of data and AI leaders from Kellogg's, Nike, Randstad Enterprise and Stardog, moderated by Carter's, weighed generative AI's opportunities against bias, hallucinations and privacy risk. They urged AI principles, testing, human oversight and caution with company data.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Start with principles. Sanders said ethical use begins with AI principles and humans choosing use cases.
- Don't paste company data into public tools. Pirzada warned that prompts may be saved and used for training.
- Ground the model. Sharma said good data foundations, domain control and traceability reduce hallucinations.
- Test and pilot. Sanders recommended pilot groups and human review before wide rollout.
- Pick use cases for value. Patel said the test is significant productivity or ROI and real adoption.
At COLLIDE 2023 in Atlanta, a panel of data and AI leaders weighed the business opportunities and the risks of generative AI, from bias and hallucinations to data privacy. Selma Dogic, Senior Manager of the Analytics COE at Carter's, moderated. Panelists were Meet Patel, Global Senior Director of Advanced Analytics at Kellogg's (the company the panelist described as newly spun off as Kellanova); Taj Pirzada, Director and Head of AI, North America, at Nike; Sean Sanders, Vice President and Global Head of Data Science at Randstad Enterprise; and Navin Sharma, VP of Product at Stardog.
In the roughly 39-minute session, the panelists said they spoke for themselves and not their employers. They described what they were building at the time, in the first conference AI Connect had held in person since the pandemic, and their views reflect the state of generative AI in late 2023.
Where are the most promising generative AI use cases?
Sanders said Randstad Enterprise was looking at generative AI as an interface to existing search-and-match tools, and as a way to help employers and job seekers present themselves accurately in job ads and resumes.
Patel said that six months earlier the concern had been privacy and regulation, but teams could now "safely" augment many use cases. His focus was operational efficiency in sales and marketing, such as turning dashboard data into actionable insights and speeding content creation and monitoring.
Sharma described a different set of uses seen by a knowledge-graph vendor. He cited a soda manufacturer working with Accenture to capture the tribal knowledge of retiring warehouse workers, plus drug discovery researchers querying research papers and manufacturers querying supply chain data. English, he said, is becoming "the next programming language."
How should companies handle bias and ethics?
Pirzada said models learn the biases in their data and that nobody outside the model builder knows what is in a large model's training set. He noted that LLMs predict the next word in a sequence with some probability, so they can pass for human while feeding users junk, and organizations need checks and balances. He said he had not yet found an effective mitigation.
Sanders said it starts with AI principles and people deciding which use cases fit the organization. Evaluation is harder because inputs and outputs are less deterministic, so his team scrubs inputs, reviews usage logs and restricts how people use the tools. One struggle: staff who sign in to an outside tool instead of the in-house one.
Patel advised bringing the legal and privacy teams into an operating council for AI and planning for change management and adoption.
How do you reduce hallucinations?
Sharma said inference is what makes models powerful and also what produces hallucinations. His prescription was a good data foundation, domain-specific control over what the model draws on, and traceability so users can follow an answer back to its source. Smaller domain-specific models, he noted, were already being discussed.
Sanders said accuracy is not a solved problem and that testing matters as much as it did for traditional machine learning. He recommended a human in the loop and a pilot group that gives feedback before a wider rollout, and said teams should keep individuals in mind even when aggregate metrics look good.
Pirzada said explainable AI has been forgotten in the rush. He described measuring a model against a written "ground truth" paragraph, not a perfect metric, and warned that legal departments will not accept "it's built by this company and they are amazing" as an answer.
What are the privacy and security risks?
Pirzada was blunt: don't put organization data into open consumer tools. Anything typed in, he said, may be saved and used for training. He said his company had negotiated with OpenAI for six months without legal teams agreeing on terms that would keep employee use from feeding the wider model. He noted vendors such as Microsoft Azure were claiming data would stay inside their environment, which he described as a claim to be verified.
Sanders added that data belongs to the organization, the individual and the client, and that techniques such as k-anonymity and anonymization are harder when inputs are unstructured and training data is unknown. His team was still exploring and did not want to roll out quickly.
How should companies integrate generative AI?
Patel said the test is whether a use case can deliver significant productivity or return and whether people will adopt it, and that sustaining it means watching for model drift. Sharma's approach at Stardog was to augment models with domain knowledge graphs, have every employee submit queries to build an evaluation set and run an early access program.
Pirzada described rolling out GitHub Copilot first to a set of engineers with training, using LLMs to add comments to uncommented legacy code, and measuring efficiency carefully because learning a new tool can slow people at first.
On the next five years, panelists leaned toward productivity, collaboration and creativity for knowledge workers, with Patel predicting a shift from efficiency toward transformation. Pirzada said simply that LLMs "are here to stay" and urged caution.
Where to learn more
Executives who want to compare notes on generative AI governance can attend The AI Enterprise Conference 2027 on Tuesday, August 31, 2027, at Pier Sixty in New York City, or COLLIDE 2027 on Wednesday, October 13, 2027, at Sandy Springs Performing Arts Center in Atlanta. See /events/ai-enterprise-conference/ and /events/collide/. Registration is coming soon.
Frequently asked questions
How can companies reduce generative AI hallucinations?
Sharma said to build a good data foundation, control the domains a model draws on and keep traceability back to sources. Sanders added testing and a human in the loop.
What privacy risks come with public generative AI tools?
Pirzada said data entered into public tools may be saved and used to train models, so company data should not be posted in them.
How should organizations handle AI bias?
Panelists suggested AI principles, an operating council with legal and privacy teams, input scrubbing, usage log review and continued monitoring.
What generative AI use cases did the panel mention?
Search and match for hiring, sales and marketing efficiency, capturing tribal knowledge, drug discovery research and coding assistants.
This article is based on remarks by Meet Patel, Global Senior Director, Advanced Analytics, Kellogg's; Taj Pirzada, Director - Head of AI, North America, Nike; Sean Sanders, Vice President, Global Head of Data Science, Randstad Enterprise; Navin Sharma, VP of Product, Stardog; moderator Selma Dogic, Senior Manager, Analytics COE, Carter's at COLLIDE Data + AI Conference 2023 on October 3, 2023. More coverage: COLLIDE AI Conference.
Filed under: AI Governance & Trust, AI Use Cases & Applications, Security & Fraud
