Sq.io: jq for databases and much more

I love JQ. But ... I'd never considered its query language to be particularly admirable. If I want to ask questions of some databases, I don't understand why I'd choose JQ's XPATH-like language to do it.

Sometimes I wonder if it wouldn't be more efficient for people to just learn SQL instead of trying to build tools or layers on top of it that introduce more complexities and are harder to search for.

This is interesting. I wonder if there is anything that does the opposite - takes JSON input and allows you to query it with SQL syntax (which would be more appealing to an old-timer like me)

To be honest, JQ is handy but it's so hard to maintain. I found myself not able to fully read other's JQ related script

Better would be the reverse. SQL queries over json: octosql.

Related is Google’s pipe syntax for SQL https://research.google/pubs/sql-has-problems-we-can-fix-the...

More tools are always great! Even if it doesn't become the mainstream, it's always great to see people explore new ways of dealing with databases!

Though I respect and applaud the effort that went into creating this and successfully releasing it, It has fewer features than duckdb supports at the moment.

Duckdb supports both Postgres, Mysql, SQLite and many other extensions.

Postgres: https://duckdb.org/docs/extensions/postgres

MySQL: https://duckdb.org/docs/extensions/mysql

SQLite: https://duckdb.org/docs/extensions/sqlite

You can try this yourself.

1. Clone this repo and create a postgres container with sample data: https://github.com/TemaDobryyR/simple-postgres-container

2. Install duckdb if you haven't and if you have just access it on the console: https://duckdb.org/docs/installation/index?version=stable&en...

3. Load the postgres extension: INSTALL postgres;LOAD postgres;

4. Connect to the postgres database: ATTACH 'dbname=postgres user=postgres host=127.0.0.1 password=postgres' AS db (TYPE POSTGRES, READ_ONLY);

5. SHOW ALL TABLES;

6. select * from db.public.transactions limit 10;

Trying to access SQL data without using SQL only gets you so far and you can just use basic sql interface for that.

If "sq" is pronounced "seek", then is "jq" pronounced "jeek"?

This is neat but I'm not really seeing anything I can't do with standard SQL and CLI tools like psql. Seems like you'd learn more reusable things using standard SQL too.

The demo appears too stateful for me. The real power of `jq` is its reliability and the ability to reason about its behavior, which stateful tools inherently lack.

First there was shell scripting, then grep, then sed, then awk, later Perl... well, now there's 'sq'!

Looks like an absolutely great (and necessary!) utility, which will automate many future workflows and dataflows, save countless hours of time collectively for many people en masse, and therefore change the world (allow more people to get more done in less time!) much like Unix, shell scripting, grep, sed, awk and Perl gave the world...

Congratulations on writing what no doubt will become one of the major Unix/Windows/MacOS/Other OS/Linux shell scripting commands in the future, if it isn't already!

Well done!

I love sq. It's handy for quickly performing simple operations on DBs and outputting that as CSV or JSON. Though my one wish is that the sq query language (SLQ) supported substring matching like SQL's `... LIKE "SOME_STRING%"`. Though you can just invoke SQL manually with `sq sql`

Wow what an expensive domain name.

It still seems to me a better solution to these sorts of problems is to use a better shell like nushell, that has richer datatypes, and so you can use the same tool to manipulate files, processes, json, csv, databases and more.

Although jq query style is not absolutely pleasant I see many examples where this tool can be used such as data transformation, import/export and linux pipelines that need access to databases.

I really like the idea of https://github.com/dinedal/textql, which uses SQL to interact with file-based data stores. However, I don't understand why sq does the opposite—using a new DSL to access a database that already has a widely-adopted and easy-to-use language: good old SQL.

It is great, I installed it, only thing I'd suggest, probably minor, is to also extract the commands to install from the bash script, and put them in the `Install` section directly, I don't run .sh script, especially if they need privileges, so I went through the bash script to take the commands for debian, they're there, probably could also be outside for other kind of people

For anyone else wondering; it's written in Go, and it keeps state inside its config file, for example sources (like a db connection string).

I was reading about something like json output coming to Postgres one day (https://www.postgresql.org/message-id/flat/ZYBdnGW0gKxXL5I_@...). Also the `.wrangle | .data` wraps on an iPhone 13 mini.

Even without a JSON column in Postgres, this is pretty trivial: SELECT jsonb_pretty(to_jsonb(employees)) FROM employees;

> sq is pronounced like seek. Its query language, SLQ, is pronounced like sleek

As a person who is apart from the tech scene, and lurks in the tech space out of interest, I appreciate this guidance. For the longest time I didn’t know nginx was pronounced Engine-X; I called it N-jinx.

Sorry but I am pronouncing that as 'ess-cue` and there is nothing anyone can do about it.

Looks kinda neat for when I don't want or need anything more than bash for a script.

Dang, I wish I had this while I still had SQL databases.

I love the idea of pushing JQ and other DSLs close to the database. I've written jq extensions for SQLite [0] and Postgres [1], but my approach involves basically embedding=pushing the jq compiler into the db. So you can do `select jq(json, jqprogram)` as an alternative to jsonpath.

Trying to understand: Is the main purpose of this to use jq-syntax for cataloging-like functionality and/or cross-query? I mean it's quite a few lines of code, but you inspect the database catalogs and offer a layer on top of that? I mean, how much data is actually leaving the database?

[0] https://github.com/Florents-Tselai/liteJQ [1] https://github.com/Florents-Tselai/pgJQ

Wow, very cool. I was looking for something like this

TSV support might be nice for Zeek logs

Not to be confused with the gpg alternative from sequoia-pgp also called sq : https://sequoia-pgp.org/

At some point, why not package Python into a single executable, and symbolic link applications and modules into it for Unixy-ness?

Another POV is all the developers I know who thrive the most and have found the most success: they rate aesthetic concerns the lowest when looking at their tools. That is to say that the packaging or aesthetic coherence in some broader philosophy matters less than other factors.