AutoML and Generative AI: Opportunity, Black Boxes and Governance
In a 2024 AI Connect webinar sponsored by Kumo.ai, leaders from Nike, Citibank, Webster Bank and Pella discussed how generative AI could reshape AutoML. They covered synthetic data, democratizing machine learning, black-box concerns and model risk management.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- AutoML gets smarter. Pareek said generative AI moves AutoML from brute-force search toward more intelligent automation.
- Two black boxes. Kuklova said AutoML plus generative AI is hard to explain to bank regulators, so humans must stay involved.
- Synthetic data helps regulated teams. Citibank's Saha described training models on generated data with sensitive fields masked.
- Access needs guardrails. Panelists urged human-in-the-loop controls before business users run models on their own.
- Start with the problem. Saha said Citibank aligns technology to a business need rather than the reverse.
In a 2024 AI Connect webinar, "AutoML Evolved: The Evolution of Automated Machine Learning with Generative AI," data and AI leaders from Nike, Citibank, Webster Bank, Pella and Kumo.ai discussed how generative AI changes automated machine learning (AutoML) and why regulated industries are moving carefully. The session was hosted with sponsor Kumo.ai and moderated by Brian Mink, co-founder and president of AI Connect.
The panelists were Parth Pareek, Product Manager Lead at Kumo.ai; Taj Pizarda, Director, Head of AI, North America at Nike; Debu Saha, SVP, Architecture and Engineering at Citibank; Natalia Kuklova, Chief Enterprise Data Science Officer at Webster Bank; and Jacey Heuer, Sr., Head of AI, Data Science and Advanced Analytics at Pella.
What are AutoML and generative AI, and how do they relate?
Pizarda described AutoML as a response to a model-building cycle that used to take at least six to eight weeks from access to data. Given a clean dataset, inputs and a response variable, AutoML tries many models and parameter settings by brute force and returns the best by a chosen metric, in minutes or hours. Pizarda treated it as a reference, never as the end-all.
Saha listed six places AutoML helps across the model development life cycle: data processing, feature engineering, model selection, evaluation, tuning and deployment, with monitoring for accuracy drift in production. Generative AI adds a natural-language interface, code generation and richer feature generation, Saha said, along with automated documentation for compliance and synthetic data generation.
Pareek argued that generative AI lets AutoML finally deliver on its original promise, moving from brute-force automation to more intelligent automation of modeling.
Is the combination too much of a black box?
Kuklova was the sharpest skeptic. In banking, Kuklova said, AutoML models are already hard to explain to regulators, and generative AI adds a second opaque layer. She joked that the combination is a black box squared. The tools are powerful but imperfect, so humans must stay involved.
Pizarda added a practical point from experience in a public company: legal and privacy teams must sign off, and the approval process is a risk mitigation framework. "You shall do no harm," Pizarda said.
What will the next few years look like?
Pareek expected a more impatient future, with business teams that have tried ChatGPT expecting better models, faster, from data science teams. He was most excited about applying the Transformer approach behind generative AI to the private data in a company's warehouse, which is what Kumo does.
Saha expected AutoML to become more accessible and widely adopted, and to help with bias detection, compliance and auditing. Another panelist stressed that data quality remains fundamental, since neither tool is a "silver bullet" for missing or poor data, and said reasoning agents could eventually treat AutoML as one tool among several.
Pizarda looked further out: AutoML might one day build a custom generative model from a person's own documents.
What can generative AI do for regulated data today?
Saha described a proof of concept at Citibank. Because regulations stop the bank from exposing sensitive data to AI and ML models, the team used generative AI to create synthetic data from a subset of high-quality records, expanding it to a much larger set with sensitive fields masked. Models were trained and tuned on the synthetic data and then moved to production data. Saha said they had seen some good outcomes and hoped to go to production.
Pizarda recalled earlier text-heavy projects, such as summarizing a 72,000-page combined PDF of accounting memos, which took many weeks, and said that summarization is now far easier.
Should business users get these tools?
Kuklova again urged caution. AutoML for non-experts, Kuklova said, is "like giving a Ferrari to someone who's just starting to learn to drive." The most complex tasks, Kuklova said, should stay with experts.
Pareek called generative AI a double-edged sword. It widens access through a chat interface, but at best it predicts the next word and is not always right. He proposed keeping a human in the loop, setting guardrails and limiting access to specific use cases. Saha said Citibank begins with the business problem rather than the technology, comparing it to buying a fan you don't need: it doesn't matter how good the fan is.
How do you handle explainability and hallucinations?
Pizarda would rely on classic testing, including A/B and multivariate tests, with a trust-but-verify approach to anything a model explains. Pareek noted the irony that generative AI is meant to explain models while remaining hard to interpret itself.
At Citibank, Saha said, everything goes through established model risk management: teams upload documentation with each model so reviewers can assess its risk and tolerance. Kuklova said trust is the biggest challenge. Webster Bank's team manually compares results across models and humans, and when it compared Mistral and Llama 3 results it saw differences. "It's just another tool in the kit," Kuklova said.
Join the conversation in 2027
Data leaders weighing automation and governance can meet at COLLIDE 2027 on Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center in Atlanta. AI Connect also has 24 free live webinars on its 2027 schedule; see the webinar page. Registration is coming soon.
Frequently asked questions
What is AutoML?
Pizarda described it as tools that try many models and parameter settings on a dataset and return the best by a chosen metric, in minutes or hours instead of weeks.
How can generative AI improve AutoML?
Panelists cited natural-language interfaces, code and feature generation, automated documentation, and synthetic data generation.
Why are banks cautious about AutoML and generative AI?
Kuklova said models are hard to explain to regulators, and combining two opaque tools adds risk.
How does Citibank govern new models?
Saha said every model goes through model risk management, with documentation uploaded for reviewers to assess risk and tolerance before production.
This article is based on remarks by Parth Pareek, Product Manager Lead, Kumo.ai; Taj Pizarda, Director, Head of AI, North America, Nike; Debu Saha, SVP, Architecture and Engineering, Citibank; Natalia Kuklova, Chief Enterprise Data Science Officer, Webster Bank; Jacey Heuer, Sr., Head of AI, Data Science and Advanced Analytics, Pella; moderator Brian Mink, Co-Founder and President, AI Connect at AI Connect webinar, Generative AI for Data Executives Webinar Series (2024) on September 12, 2024. More coverage: Webinars.
Filed under: AI Use Cases & Applications, AI Governance & Trust, People, Culture & Leadership
