Use another composition operator

[davidanthoff]

Right now we use the + operator both for layered plots and for sub specs like repeat, facet etc. That is kind of confusing. I think generally we should use two operators for this, one for layering, and a different one for sub specs.

One idea would be to use & for layering, and + for sub specs. The benefit is that & has higher precedence, and I think that is what we want for layering. So something like

@vlplot(repeat=...) + p1 & p2 & p3

Would repeat a sub plot that has three layers.

One thing that I don't like about + is that normally folks probably assume it is symmetric, but in our use case it is not. I guess one question is whether there is some alternative infix operator that more clearly is non symmetric that we could use. Bonus if we would find one that has lower precedence than +, so that we could use + for layering :)

@tkf and @mcmcgrath13, I'd be very interested to hear your thoughts on this!

[tkf]

I guess one question is whether there is some alternative infix operator that more clearly is non symmetric that we could use.

Why do you want to use infix operators? I think it makes sense to use infix operators only when you have associative or commutative operations. (So, for layering I think it makes sense.)

Any reason to not use something like vlplot(spec::VLSpec, repeat=[:A, :B, :C]) and @vlplot(spec, facet={row=:A})?

[davidanthoff]

Why do you want to use infix operators?

I'm trying to get rid of as many nesting levels as I can, I think they make things tricky in interactive code.

But I agree, using something like + for this is weird... Maybe >? So something like this could work:

@vlplot(repeat=...) > p1 + p2 + p3

Essentially this would read as "apply the repeat operator on that layered plot I created via the + signs".

Any reason to not use something like vlplot(spec::VLSpec, repeat=[:A, :B, :C]) and @vlplot(spec, facet={row=:A})?

I think we should also support that! Should it be vlplot(spec=my_other_spec, repeat=[:A, :B, :C]) and @vlplot(spec=my_other_spec, facet={row=:A}), though?

[tkf]

I think they make things tricky in interactive code

Isn't @vlplot(p1 & p2 & p3, repeat=...) as easy to type as @vlplot(repeat=...) + p1 & p2 & p3?

But if it's just a shortcut for interactive use, I guess I don't mind.

Maybe >?

I am a bit against using > or any comparison operator for this syntax as Julia assumes it to return a Bool. This becomes problematic for chaining more than three specs (though I'm not sure when you need it)

julia> Meta.@lower a < b < c
:($(Expr(:thunk, CodeInfo(
    @ none within `top-level scope'
1 ─ %1 = a < b
└──      goto #3 if not %1
2 ─ %3 = b < c
└──      return %3
3 ─      return false
))))

julia> Meta.show_sexpr(:(a < b < c))
(:comparison, :a, :<, :b, :<, :c)
julia> Meta.show_sexpr(:(a < b))
(:call, :<, :a, :b)

FYI it looks like .. has a lower precedence

julia> Meta.show_sexpr(:(a .. b + c + d))
(:call, :.., :a, (:call, :+, :b, :c, :d))

though it is already used by https://github.com/JuliaMath/IntervalSets.jl and https://github.com/ChrisRackauckas/EllipsisNotation.jl.

Should it be vlplot(spec=my_other_spec, repeat=[:A, :B, :C]) and @vlplot(spec=my_other_spec, facet={row=:A}), though?

Yes, supporting them makes sense. But I've been guessing the "philosophy" of VegaLite's shortcuts is to let people use the positional argument when there is a clear "primary" argument. In this case, spec sounds like a such primary argument. It's also trivial to add this support:

vlplot(spec::VLSpec; kwargs...) = vlplot(; spec = spec, kwargs...)

[davidanthoff]

Isn't @vlplot(p1 & p2 & p3, repeat=...) as easy to type as @vlplot(repeat=...) + p1 & p2 & p3?

So I'm mostly thinking about the interactive scenario where one doesn't want to assign things to intermediate variables. So the example would look more like:

@vlplot(@vlplot(...) & @vlplot(...) & @vlplot(), repeat=...)`

vs

@vlplot(repeat=...) + @vlplot(...) & @vlplot(...) & @vlplot(...)

and now we got rid of one level of parenthesis :) I think these deeply nested parenthesis things make interactive use tricky, so many of the additions that I made in VegaLite.jl are meant to get rid of deep nesting.

I am a bit against using > or any comparison operator for this syntax as Julia assumes it to return a Bool.

Yep, didn't know that, and that seems an important argument... I'm actually quite surprised by this, that essentially means that < is control flow like &&? That seems weird...

Yes, I think we should consider .. already taken, plus it also doesn't have any visual notion of direction. I'll keep looking :)

Yes, supporting them makes sense. But I've been guessing the "philosophy" of VegaLite's shortcuts is to let people use the positional argument when there is a clear "primary" argument.

I think mostly I've considered these shortcuts as a way to make super common scenarios easier. I almost feel that this composition stuff is obscure enough that maybe having a little bit more verbose syntax is ok?

[tkf]

and now we got rid of one level of parenthesis

You can get rid of the parentheses when the operation is associative and that is why you don't need to write parentheses for associative binary operators. Using binary operator to get rid of parentheses sounds like backward to me. OK, I'm pretty much influenced by this Dijkstra's notes on syntax for this topic and I understand this view is rather extreme. (e.g., I know I can't defend /)

I can see that there is a tension between easy-to-write and rigor in the API. But I think it's much better to get the rigor part right and then put sugars on top of it.

[davidanthoff]

Dijkstra's note is interesting, but yeah, it does strike me as a bit extreme :)

But here is another reason against >: it has lower precedence than |>, which would really screw things up.

Maybe we should just use |>... Then the question becomes whether it would be df |> @vlplot(repeat=...) |> @vlplot() or the other way around df |> @vlplot() |> @vlplot(repeat...). I think I'm tending towards the latter: with the data we already have the convention that the thing on the left of |> gets put into the thing on the right, so that would be somewhat analogous?

I can see that there is a tension between easy-to-write and rigor in the API. But I think it's much better to get the rigor part right and then put sugars on top of it.

Yep, agreed! And on that note the current shipping version is a disaster ;) Where + plays all sorts of double roles. But maybe that is just @vlplot(spec={...}, repeat=...)? The one thing we should enable is @vlplot(spec=@vlplot(...), repeat=...), that seems really uncontroversial but doesn't work right now, I think.

[tkf]

Then the question becomes whether it would be df |> @vlplot(repeat=...) |> @vlplot() or the other way around df |> @vlplot() |> @vlplot(repeat...).

OK, that's interesting. If we say

1. @vlplot(...) always curries
2. @vlplot(spec::VLSpec, ...) is supported

then I think df |> @vlplot(...) |> @vlplot(repeat...) can be "derived" from that.

For consistency, maybe @vlplot(data::Any, ...) should then be supported. It would mean that @vlplot(:point, ...) works because of @vlplot(::Symbol, ...) "method" and the Any case tries to interpret the first argument as a data/iterator.

The one thing we should enable is @vlplot(spec=@vlplot(...), repeat=...), that seems really uncontroversial but doesn't work right now, I think.

Yeah, it would be great if it works.

[davidanthoff]

Ok, here is another idea:

• we use & for layering
• we use + for sub plots, and we don't care whether the parent or the child is specified first. The rule is that one of the two args to + must be an parent plot that is missing a child element, and that will become the parent, and the other plot the child. We'd probably need to go carefully through all possible combinations whether that actually works, though.
• we use |> for @vlmod only

[tkf]

By subplots you mean concat? Isn't [...] concise enough?

[davidanthoff]

No, I mean all composite views. They always have an outer spec, and then some children (either one child or a multiple in the case of layers and concats).

Here are some examples how you would use this:

# Concat
@vlplot(title="Some title for the entire plot") + [p1 p2]
[p1 p2] + @vlplot(title="Some title for the entire plot")

# Layer
@vlplot(title="Some title for the entire plot") + p1 & p2
p1 & p2 + @vlplot(title="Some title for the entire plot")

# Facet
@vlplot(facet={...}) + p1 # Here p1 will be faceted, again you could change the order

# Repeat
@vlplot(repeat={...}) + p1 # Here p1 will be repeated

[tkf]

Can't they be supported by ... |> @vlmod(...)?

[haberdashPI]

Right now the composition operators feel a bit too "magical": I don't understand the precise transformations they perform and they do not feel very composable. From a transparency perspective, I would prefer that there be be two levels of composition: one that matches very closely to the vegalite spec, and uses nested forms, as suggested by @tkf above, because that is how vegalite has been organized. Then, if I prefer the operator-like version, I can use that as a short-hand for these longer-form nested statements.

I think this follows the principle laid out in the VegaLite.jl documentation early on: it says that you can pretty naturally translate any spec in vegalite to the syntax used in VegaLite.jl and there are some useful shortcuts provided by VegaLite.jl to make things easier. From what I understand, this actually isn't currently true when it comes to the composition operators, because of the assumption made around spec's and layers.

I agree that the operator for adding parents to children should be directional, and I agree it should be different from the operation to compose children together. I think being able to change the order of parent and child would be confusing and unnecessary.

Where parent is some @vlplot spec and a, b, and c are children, I could see either of the following also working:

• parent => [a,b,c]
• parent(a,b,c) (make parent objects callable)

I also think I like @tkf's proposal of using @vlmod but I don't know that I fully understand the nature of @vlmod.