July 5th, 2024

DuckDB Community Extensions

The DuckDB team launched the DuckDB Community Extensions repository for easy extension installation. Users benefit from a simplified process, while developers can streamline publication tasks. Security measures include code vetting options.

Read original articleLink Icon
DuckDB Community Extensions

The DuckDB team has introduced the DuckDB Community Extensions repository to simplify the process of publishing and installing extensions. This repository allows users to easily install extensions using a specific syntax, avoiding the need for compilation and distribution by extension developers. DuckDB's design philosophy focuses on simplicity, leading to the creation of a powerful extension mechanism that supports new functionalities like file formats, compression methods, and network protocols. The Community Extensions repository enhances user experience by enabling easy discovery, installation, and maintenance of extensions directly from the DuckDB SQL prompt. For developers, the repository streamlines the publication process by handling tasks such as building extensions for various platforms and signing binaries. The repository already contains key extensions like crypto, h3, lindel, prql, scrooge, and shellfs. While DuckDB Labs and the DuckDB Foundation do not vet the code within community extensions, users can disable loading them with a configuration option for security. The DuckDB team looks forward to expanding the repository and encourages developers to contribute extensions via pull requests.

Related

Show HN: Eidos – Offline alternative to Notion

Show HN: Eidos – Offline alternative to Notion

The Eidos project on GitHub offers a personal data management framework as a Progressive Web App with AI features. Customizable with extensions and scripting, it leverages sqlite-wasm technology for chromium-based browsers.

What Happens When You Put a Database in the Browser?

What Happens When You Put a Database in the Browser?

WebAssembly (Wasm) enhances browser capabilities, enabling high-performance apps like DuckDB for ad-hoc queries and Python environments. DuckDB Wasm boosts performance in interfaces like lakeFS, Evidence, and Count. MotherDuck enables local querying, emphasizing efficient data processing.

Show HN: Qq: like jq, but can transcode between many formats

Show HN: Qq: like jq, but can transcode between many formats

The GitHub repository hosts `qq`, a tool using `jq` query syntax and `gojq` for configuration format transcoding. It offers interactive query building, multiple format support, and encoding performance focus. Installation options include source or releases. Contributions welcome.

Show HN: DuffMan – Fuzzer for Postman Collection

Show HN: DuffMan – Fuzzer for Postman Collection

The GitHub URL provides details about DuffMan, a Go-based diagnostic tool for fuzzing API nodes. It parses Postman collections, conducts fuzz testing, and offers various installation methods and usage instructions. Inquiries for assistance are encouraged.

DuckDB: Vector Similarity Search Extension

DuckDB: Vector Similarity Search Extension

The vss extension in DuckDB enhances vector similarity search with HNSW indexing for ARRAY columns. Users can optimize queries with distance metrics but should be cautious due to limitations and experimental features.

Link Icon 8 comments
By @xnx - 5 months
Very cool. The shellfs extension (https://github.com/rustyconover/duckdb-shellfs-extension) that allows shell commands to be used for input and output will make DuckDB even more useful as a command line analysis tool. I'm not sure how I'll use it yet, but I'm betting I can streamline some multi-step data processes.
By @netcraft - 5 months
>DuckDB Labs and the DuckDB Foundation do not vet the code within community extensions and, therefore, cannot guarantee that DuckDB community extensions are safe to use. The loading of community extensions can be explicitly disabled with the following one-way configuration option:

So we should think of this like NPM.

Still, very cool and very useful. Would love a way from inside of duckdb directly to query the extensions available from community.

By @ec109685 - 5 months
If anyone is curious, web assembly is also supported: https://duckdb.org/2023/12/18/duckdb-extensions-in-wasm.html
By @9cb14c1ec0 - 5 months
> What happens behind the scenes is that DuckDB downloads an extension binary

The baser part of me wonders how hard it would be to compromise that supply chain.

By @shubhamjain - 5 months
Honest question, how feasible it would be for DuckDB to release a non-columnar version of their DB (or at least make DuckDB a decent choice for a typical web app)? I don't know any other DB that makes installing extensions this easy. The rate at which they're shipping awesome features makes me wonder if they could eventually become a great generic database.

I know, I know, this could just as easily be a double-edged sword. A database should prioritize stability above everything else, but there is no reason why we shouldn't expect them to reach there.

By @gigatexal - 5 months
This is the coolest thing! I’m very excited to see what we will have next. Hah maybe an extension that imbeds vim and then I’ll never leave DuckDb lol
By @victor106 - 5 months
Does duckdb (natively or through extensions) support Delta Tables?