DuckDB: Vector Similarity Search Extension

The Vector Similarity Search Extension for DuckDB, known as vss, is an experimental feature that enhances the database's capabilities by introducing indexing support for accelerating vector similarity search queries. This extension leverages DuckDB's fixed-size ARRAY type to optimize queries involving distance metrics. Users can create HNSW indexes on tables with ARRAY columns using the CREATE INDEX statement with the USING HNSW clause. Different distance metrics like Euclidean distance and cosine similarity are supported, and users can customize index creation with various options like ef_construction and ef_search. However, there are limitations to consider, such as only supporting FLOAT vectors and requiring the index to fit in memory. Additionally, persistence of custom extension indexes is an experimental feature due to potential data loss risks during unexpected shutdowns. Users are advised to exercise caution when enabling experimental persistence and to avoid using it in production environments.