Show HN: Qq: like jq, but can transcode between many formats

I think there may already exist a jq alternative for every letter of English alphabet. Why not just make a tool that specializes at transcoding csv/json/xml/protobuf/etc, and leave querying to any of dozens of these utils, that already exist? So that I can write qq input.xml -o json | jq '.' | qq -f json -o hcl? It's not like the intention is to bring something new with regards to querying lanuage, the "real stuff" is just an heuristic transcoder of popular data serialization formats.

Readers may be interested to Dasel as well https://daseldocs.tomwright.me/supported-file-formats

This is so promising!

I presume like pandoc, the goal is to use an internal model to reduce the M*N complexity of format transcoding to M+N.

But when I look at the tests for this (and Dasel and rq also mentioned here), I see very little. jq seems to have much more. Perhaps I'm missing them.

Which leads to some side/related comments...

A great project would be to build a transcoded test suite that all these tools could run against. It wouldn't be hard to generate.

Also, while go is a great language, I think pandoc gets a lot of benefit from implementing the internal model in haskell. Pandoc has a much harder problem because there's a lot more variability and less structure in the formats, so it should be doable for formats.

Honest question, yq pretty much does this, are you solving for something yq doesn't?

As an aside, chatgpt is very good at generating jq queries from an example input and text version of the query.

This is still under a lot of change, expect more formats to be supported along with some general improvements to interactive mode and autocomplete.

I have been working on a project in a similar vein: rq[0]. Mine started out as an attempt to make a jq-like frontend for the Rego[1] language. However, I do find myself using it to simply convert from one format to another, and for pretty printing, quite often.

The interactive mode that qq has is really slick. I didn't torture test it, but it worked pretty smoothly with a quick test.

I see that the XML support is implemented using the same library as rq. This definitely has some problems, because the data model of XML does not map very cleanly onto JSON. In particular, I recall that I had problems using mxj to marshal arrays, and had to use this[2] ugly hack instead. qq seems to have done this a little more cleanly[3]. I may just have to lift this particular trick.

I definitely found CSV output to be challenging. Converting arbitrary JSON style data into something table shaped is pretty tricky. This[4] is my attempt. I have found it works well enough, though it isn't perfect.

I can also see that qq hasn't yet run into the byte order marker problem with CSV inputs yet. Might want to check out spkg/bom[5].

One final breadcrumb I'll drop - I drew a lot of inspiration for rq's input parsers from conftest[6]. That may be a good resource to find more formats and see specific usage examples of them.

Thanks for sharing! It's really interesting to see some of the convergent evolution between rq and qq.

0 - https://git.sr.ht/~charles/rq

1 - https://www.openpolicyagent.org/docs/latest/policy-language/

2 - https://git.sr.ht/~charles/rq/tree/c67df633c0438763956ff8646...

3 - https://github.com/JFryy/qq/blob/2f750f04def47bec9be100b7c89...

4 - https://git.sr.ht/~charles/rq/tree/c67df633c0438763956ff8646...

5 - github.com/spkg/bom

6 - https://github.com/open-policy-agent/conftest