Semantic joins with AI, For Deeper Knowledge Discovery

Deeper knowledge discovery often requires matching data without access to common join keys. This includes combining data from disparate systems such as CRM and sales databases, matching customer data across different divisions within the same company or incorporating third-party data. Similarly, identifying and merging duplicate records within datasets poses the same fundamental challenge.
This problem, commonly known as record linkage or entity resolution, remains difficult to solve at scale. Traditional approaches from basic string similarity metrics to sophisticated ontology-based methods, struggle with modern data's scale, ambiguity, and diversity. They cannot reliably distinguish among synonyms, polysemy, and abbreviations, require lots of manual maintenance, and ultimately fail to handle the full "fuzziness" of real-world data.
This talk will explore how LLMs address these core limitations by replacing syntactic analysis and brittle rules with semantic comprehension. LLMs can adapt to new semantic matching tasks without requiring costly retraining, enabling users to define complex or domain-specific join criteria through natural language and examples. This substantially reduces manual effort and paves the way for more accurate, automated, and adaptable entity resolution—whether matching across systems or deduplicating within them.

Understand the limitations of traditional record linkage approaches (string metrics, ontologies, rule-based systems, ...) Understand how LLMs enable semantic comprehension for data matching at scale Apply LLM techniques for automated, adaptable entity resolution and deduplication