Format json with JQ_COLORS

[thaliaarchi]

This PR adds support for colors via JQ_COLORS and enabling/disabling with NO_COLORS, --color-output/-C, and --monochrome-output/-M. The formatting matches jq (not gojq).

I've tested this and it has identical behavior to jq. If we want to add tests, it would need a new harness, because --run-tests uses value-based equality, not formatted textual equality.

I also removed the tojson/1 intrinsic, because neither jq nor gojq define it.

If jqlang/jq#3034 is merged, I'll update _tojson here to match.

[thaliaarchi]

Commit 231277f moves _tojson without changing it, after the previous commits modify it. If you're reviewing with a diff viewer that doesn't detect text moves (like GitHub), then you can review it easier in three chunks: before (master...808b031), move (231277f), and after (231277f...d7cea3c)

[wader]

fq lacks JQ_COLORS support atm (has it's own color config with more stuff) maybe can steal some of this code :)

[thaliaarchi]

You're welcome to use it for that :). It should represent the C parsing and formatting exactly.

From my tests, it appears the gojq ignores JQ_COLORS and it formats differently. Maybe it would be worth coordinating a decision with gojq? I figure you probably inherited that from using gojq as a library.

I submitted jqlang/jq#3034, which fixes some usually-invisible inconsistencies with colors in jq. I think that should be merged regardless, but in addition it might worth bringing jq more into line with gojq. In particular, I don't really like how in jq, {, :, ,, and } in objects get the object color and [, ,, and ] in arrays get the array color. I think : and , punctuation should be colored uniformly across objects and arrays. Personally, I don't see much use in coloring braces and brackets, but many people like coloring them by nesting depth in VS Code, so I can see why this simplistic differentiation could be somewhat useful.

[wader]

You're welcome to use it for that :). It should represent the C parsing and formatting exactly.

👍

From my tests, it appears the gojq ignores JQ_COLORS and it formats differently. Maybe it would be worth coordinating a decision with gojq? I figure you probably inherited that from using gojq as a library.

Kind-of, fq uses a modified version of cli/encoder.go from gojq https://github.com/wader/fq/blob/master/internal/colorjson/encoder.go that makes the options non-global and adds some things needed by fq (how to format binary strings etc)

I noticed now that gojq seems to only read GOJQ_COLORS, was thinking if i added support in fq i would probably read FQ_COLORS and fallback to JQ_COLORS? hmm

I submitted jqlang/jq#3034, which fixes some usually-invisible inconsistencies with colors in jq. I think that should be merged regardless, but in addition it might worth bringing jq more into line with gojq. In particular, I don't really like how in jq, {, :, ,, and } in objects get the object color and [, ,, and ] in arrays get the array color. I think : and , punctuation should be colored uniformly across objects and arrays. Personally, I don't see much use in coloring braces and brackets, but many people like coloring them by nesting depth in VS Code, so I can see why this simplistic differentiation could be somewhat useful.

I see, hmm haven't really thought about it much so no string opinion, but making them more consistent make sense i think. Let's see what the other maintainers think

[wader]

Commit 231277f moves _tojson without changing it, after the previous commits modify it. If you're reviewing with a diff viewer that doesn't detect text moves (like GitHub), then you can review it easier in three chunks: before (master...808b031), move (231277f), and after (231277f...d7cea3c)

👍

[thaliaarchi]

Thanks for the review! I'll go through your comments in the morning.

Shortly after your review, I implemented the raw output options, which I've now pushed. They touch the _tojson filter after eval, so are fairly related to this PR. I'd prefer to have it reviewed here, but I could split it from this PR and rebase it onto master once this lands, if you'd like.

This reduces the missing features needed for wsjq even more!

[wader]

Thanks for the review! I'll go through your comments in the morning.

No stress!

Shortly after your review, I implemented the raw output options, which I've now pushed. They touch the _tojson filter after eval, so are fairly related to this PR. I'd prefer to have it reviewed here, but I could split it from this PR and rebase it onto master once this lands, if you'd like.

Either way works for me so let's keep it here

This reduces the missing features needed for wsjq even more!

🥳

[thaliaarchi]

Since jqlang/jq#3034 looks like it'll be merged, I've pushed a commit to match it.

[wader]

I tried to rerun this, still 307, but i did notice that something seems strange with the output when running tests. Have a look at make test guessing it's the switch to --join-output that require some change?

[thaliaarchi]

I just changed ./jqjq to replace --join-output with --raw-output for --run-tests.

[thaliaarchi]

I get 308 on jq.test from jq origin/main.

[wader]

Huh made sure i had master also now, still 307, weird, some locale thing? anyways, that's for another time

[wader]

BTW talking about ANSI output, maybe https://github.com/wader/ansisvg is interesting? think my main use of it has been producing colorful output from jq and fq for presentations :)

[wader]

So ready to merge?

[thaliaarchi]

Yep!

[wader]

Great work! and so happy someone else wants to help out with this weird thing :)

[wader]

Hmm noticed --help looks weird, maybe the last minute fix for test output messed up? wonder if there is a neat we to have output tests

[thaliaarchi]

Your ansisvg project looks useful. I've used asciinema for that kind of thing before, but this would be usable outside the browser when it's not animated.

It reminds me of a tiny project I did, terminal-palette. It computes the nearest color in a palette for RGB colors, by using the perceptual distance between colors in the CIELAB color space (logic borrowed from james-oldfield/nearest-colour). I used it to make fallback color schemes for my Tetris game, programmed in Whitespace. It uses the official piece colors when the terminal supports 24-bit color, or uses similar colors from the 256 colors if that's supported, or otherwise uses the basic 16 colors.

[wader]

Neat! think i will have to have a closer look at the whitespace language

[wader]

Getter a bit late here, feel tree to merge a fix for help if you look into it. Otherwise will fix or review tomorrow

[thaliaarchi]

Good catch for --help. I had picked --join-output as the unconditional output format (except for tests), because it gives the most flexibility to jqjq. jqjq can insert LF or NUL as appropriate, according to its own --raw-output/--raw-output0/--join-output options. That means, though, that it needs to be more conscious of how everything is formatted.

In this case, the fix is to add \n to each line of help. I'd go further to even concatenate it into one string. I have a fix. Can I just push it to master?

[wader]

Nice, yeap push is fine too!