Semantic Layer and Knowledge Graphs for Million-Column Databases
At COLLIDE 2023 in Atlanta, RelationalAI's Nikolaos Vasiloglou explained why cloud data platforms with hundreds of millions of columns need a semantic layer. He described how ontologies, human review and knowledge graphs help language models answer questions.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Access is not discovery. Analysts may reach 100 million columns but still cannot find what they need.
- Shared definitions cut errors. A semantic layer fixes the meaning of terms like revenue and active customer.
- Most columns are derived. Vasiloglou said 99% are not core concepts, so ontologies can start small.
- Humans stay in the loop. Business experts seed the ontology; language models expand it under review.
- Retrieve before you query. Narrowing to relevant concepts keeps prompts small and lets cheaper models work.
At COLLIDE 2023 in Atlanta, Nikolaos Vasiloglou, PhD, Vice President of Research ML at RelationalAI, described what happens when an analyst gains access to a hundred million columns in a cloud data platform and cannot find the ones they need. In a short talk on answering questions over multi-million column databases with knowledge graphs, he argued that a semantic layer built on an ontology is the way to make that data usable, including by language models.
The talk was delivered in 2023, and the product details reflect RelationalAI's offering at that time.
Why is finding data now the problem?
Vasiloglou said the cloud revolution of 10 to 15 years ago was about storage and compute on demand. Today, he said, the conversation is about the data cloud, where companies put all their data in a platform such as Snowflake or BigQuery and build applications on top of it.
He drew a distinction between cloud hosted and cloud native. A cloud-hosted system was built to run on a single machine, however large, and is constrained by that machine. A cloud-native system uses on-demand compute and storage, so a workload needing thousands of cores can be run and shut down.
Once chief data officers can federate their databases into one place, he said, the next question is: "How do I find what I need?"
What does a semantic layer do?
Vasiloglou said that when BI tools, notebooks, feature engineering and data apps all read and write directly to the database, the result is noise and redundancy. One application may define revenue differently from another, and column names can be cryptic. "We all have to agree what the revenue is," he said, as well as what counts as a customer or an active customer.
He defined an ontology as a collection of concepts, relationships and business rules used by domain specialists. Every enterprise has one, he said, whether or not it is modeled, in people's heads, invoices and applications.
Benefits he listed included:
- Reuse and fewer errors: One shared definition reduces mistakes from misread columns.
- Lineage and sharing: It is easy to share inside and outside the organization.
- Compact queries: He cited a 10x to 100x reduction in code when queries traverse many concepts.
- Portability: A semantic layer can move between platforms, for example from Snowflake to BigQuery.
He pointed to dbt's success as a sign of demand and named tools such as LookML, Malloy and PRQL, alongside older semantic web standards such as SPARQL and RDF.
How do you build an ontology for a huge database?
Doing it manually is hard, Vasiloglou said, and it is not straightforward with language models either, which need a library of prompts. A hybrid approach with a human in the loop lowers the bar.
He said that at companies with hundreds of millions of columns, 99% of columns are not core concepts. They are derived from a few source applications, such as a website or customer service app. His process starts with business experts, who can produce a core ontology within a few weeks, after which language models do the heavy lifting under human review.
He showed a customer example in which about 18,000 columns were translated over the first 23 weeks, with a sudden jump once the team understood the approach. As benefits became visible, he said, the customer invested more people and time.
To evaluate an ontology, he suggested building an internal benchmark, such as a set of 1,000 questions, and measuring how many can be answered automatically.
How do knowledge graphs help language models answer questions?
Vasiloglou used a telecom example: find the cell tower that receives the most calls from customers living in a given area. A model can translate such a question into a query such as Datalog or SQL, he said, but as the ontology grows the query no longer fits in the prompt, and models make more mistakes as the context window fills.
His alternative works in stages. The model identifies the parts of the ontology relevant to the question, and the knowledge graph is then used to find the path through those concepts. He said this allows cheaper models that can run on premises, without dependence on a commercial API provider.
Where does RelationalAI fit?
Vasiloglou said RelationalAI positions itself as a "relational knowledge graph" and an AI co-processor for Snowflake, announced at the most recent Snowflake Summit. It runs inside Snowflake's security perimeter using Snowpark Container Services, he said, so data does not have to leave and a security officer does not need to vet another vendor. He said it supports graph analytics, business rules and optimization.
More on semantic layers and enterprise AI
AI Connect hosts ongoing coverage of data and AI strategy. The AI Enterprise Conference 2027 is Tuesday, August 31, 2027, at Pier Sixty in New York City; see the event page. The group also offers 24 free live webinars on its 2027 schedule at the webinars page.
Frequently asked questions
What is a semantic layer?
Vasiloglou described it as a shared ontology of concepts, relationships and business rules that all data consumers agree on.
Why do large databases need an ontology?
With hundreds of millions of columns, analysts can access data but not find it, and inconsistent definitions cause errors.
Can language models build an ontology?
He said it is not straightforward, but a human-in-the-loop approach with prompts makes it much faster than doing it manually.
How do knowledge graphs help LLMs answer questions?
They narrow the ontology to relevant concepts and find the path through them, so the prompt stays small.
This article is based on remarks by Nikolaos Vasiloglou, PhD, Vice President of Research ML, RelationalAI at COLLIDE Data + AI Conference 2023 on October 3, 2023. More coverage: COLLIDE AI Conference.
Filed under: Context, Semantics & RAG, Data & AI Infrastructure
