Scaling Conceptual Semantic Search to Terabytes

Scaling Conceptual Semantic Search to Terabytes

Explore techniques for scaling semantic search to enterprise-level performance in this 33-minute conference talk from #HLTCon 2018. Delve into methods for accelerating conceptual search from a few thousand comparisons per second to 500 million on commodity hardware. Examine the application of semantic search in an alerting platform, where incoming textual data is compared to pre-defined alerts to highlight crucial information for analysts. Cover topics including text vectors, indexing challenges, Python implementation, and real-world applications such as identifying children of immigrants and detecting cyber attacks. Gain insights into optimizing semantic search for terabyte-scale datasets and enhancing data analysis efficiency.