feature request: support gonb

[HaveF]

Thanks for this repo!

could it possible support gonb?

for now, error:

jupytext.formats.JupytextFormatError: Extension '.go' is not a notebook extension. Please use one of '.ipynb', '.md', '.markdown', '.Rmd', '.py', '.coco', '.R', '.r', '.jl', '.cpp', '.ss', '.clj', '.scm', '.sh', '.ps1', '.q', '.m', '.wolfram', '.pro', '.js', '.ts', '.scala', '.rs', '.robot', '.resource', '.cs', '.fsx', '.fs', '.sos', '.java', '.groovy', '.sage', '.ml', '.hs', '.tcl', '.mac', '.gp', '.do', '.sas', '.xsh', '.qmd', '.myst', '.mystnb', '.mnb', '.auto'.

[mwouts]

Hi @HaveF , thanks for reaching out! Yep sure it would be a great addition.

For that I would need at least a short example notebook, or ideally a full pull request. The process for adding a new language is documented here: https://jupytext.readthedocs.io/en/latest/contributing.html#add-support-for-another-language.

Thanks!

[HaveF]

@mwouts Thanks for the reference!
The process for adding a new language seems easy, let me try it!

[HaveF]

I believe I can't fix it by myself. @mwouts

added _SCRIPT_EXTENSIONS in jupytext/languages.py:

    ".go": {"language": "go", "comment": "//"},

simple hello_world_gonb.ipynb, I have zipped it for upload to this issue.

hello_world_gonb.ipynb.zip

some error:

E                       jupytext.compare.NotebookDifference: Cell content differ on code cell #4:
E                       --- expected
E                       +++ actual
E                       @@ -1,2 +1 @@
E                       -%%
E                        fmt.Printf("Hello World!")

the sample notebook is copied from gonb/examples/tutorial.ipynb at main · janpfeifer/gonb.

%% is special mark for it.

[mwouts]

Hi @HaveF , can you tell me a bit more about the %% marker in go notebooks? Also, I'd like to give you the authorship of the commit in the PR, would you like to open a PR yourself?

[HaveF]

Hi, @mwouts , Thanks for helping me out. I believe your PR is fine. I can help to test.

%% is not part of Go, but it is a shortcuts of GoNB. Anything after a "%%" will be wrapped inside a func main() { ... }

like

%%
fmt.Printf("Hello World!")

It will converted to this behind the scenes

func main() {
    fmt.Printf("Hello World!")
}

It can appears at any part of a cell, not just beginning of a cell, like:

var startValue = float32(1)

%%
startValue = incr(startValue)
fmt.Printf("current startValue=%f\n", startValue)

Do you see the error I mentioned before? Or, you can help to change in #1250, then I test it

[HaveF]

Other special notation in GoNB:

%% --who=world can pass flags to main func

import (
    "flag"
    "fmt"
)

var flagWho = flag.String("who", "", "Your name!")

%% --who=world
fmt.Printf("Hello %s!\n", *flagWho)

%args also can pass flags

%args --who=Wally

func main() {
    flag.Parse()
    fmt.Printf("Where is %s?", *flagWho)
}

!* notation run the command in special building dir

!*pwd ; ls -l

%test for test the current cell

import "github.com/stretchr/testify/require"

func TestIncr(t *testing.T) {
    require.Equal(t, 2, incr(1))    
}

func BenchmarkIncr(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = incr(i)
    }
}

%test

other GoNB command

%track

%goworkfix

[mwouts]

Thank you @HaveF - that's very helpful! Well, I'm afraid this is going to be a little tricky...

Can I ask you one more question? How do you think that this notebook cell should be represented in the go:percent format?

%%
fmt.Printf("Hello World!")

Do you think that this might work for you? (is this a valid go file?)

// %%
%%
fmt.Printf("Hello World!")

[HaveF]

@mwouts Thanks for your help.
Yes, It is a bit tricky, because GoNB introduces shortcuts that are not originally part of Go.

// %%
%%
fmt.Printf("Hello World!")

Although this is not a valid go file, I think if someone uses GoNB, he should understand the meaning of %%.

• Or for greater clarity, we just call the jupytext file as *.gonb? Then just let the user configure gonb's syntax highlighting?
• Or just name the files as *.go, let users be responsible for understanding some special syntax of GoNB. This allows users to configure less content(I prefer this)

---

Hi, @janpfeifer (author of GoNB), jupytext's author(Marc Wouts) is help us to support GoNB in jupytext. We are now facing a problem with the correspondence between jupyter notebook and jupytext. Can you help us see if the above syntax correspondence is the most ideal? Or you have other ideas? Thanks!

[janpfeifer]

hi @HaveF , @mwouts

Thanks for following up on this, Jupytext looks awesome.

I was reading through Jupytext readthedocs page and I'm not sure I got all the details right -- for the py:percent format, the # %% marks the start of a code cell. What marks the end of the cell ? Is it until the next # %% tag ?

A few meta comments, from what I understood (I may be wrong):

1. The input for Jupytext could never be really proper Go, because either it contains statements in the global scope (which is not valid in Go) or it would have various definitions of func main(). So we can expect better IDEs to get confused.
2. Because of (1), I agree calling the suffix .gonb, since anything we come up with will be kernel dependent. And it serves to remind the user that there is some small special syntax to consider -- with the added pain of configuring IDE's syntax highlighting (one can always create a symlink from the .gonb file to a .go file)
3. About gonb special commands: what about prefixing them with //gonb: in Jupytext files ? (edited) Where Jupytext would remove any line prefixed with //gonb: and pass the rest of the line as is to GoNB I could change gonb to also accept special commands prefixed with //gonb:. This is somewhat aligned to special //go: special comments, but it clearly points out it is gonb related. It's lots of typing ... but usually they don't make most of the code.. Btw, %help lists all of the special commands in gonb with a description.
4. When using Jupytext + Go, one can always use various func main() {...} (one per cell) and never need to use any gonb special command. More typing ... but should be fine ?

Any thoughts?

Example of what a Jupytext+Gonb code would look like:

// %% [markdown]
// # Defining a $x^2$ function
// It returns x*x.

// %%
func Square(x float64) float64 {
  return x * x
}

// %% [markdown]
// # Examples
//
// ## Example A: $x = 3$

// %%
var x = flag.Float64("x", 0.0, "Value of x")
//gonb:%% -x=3
fmt.Printf("Square(%g)=%g\n", *x, Square(*x))

// %% [markdown]
// ## Example B: $x = 4$

// %%
//gonb::%% -x=4
fmt.Printf("Square(%g)=%g\n", *x, Square(*x))

This seems unambiguous, trivial to parse ... but not very pretty on the eyes 😄 (too many %%).

I would suggest an alias for the //gonb:%% special command, something like //main or something else ? Any suggestions ? Another option is to start with this verbose but full support format, and create the aliases later, after some use we may have a better ?

Btw, if (1) becomes a serious usability issue (IDE complaining of various func main), I could create a %main myFunc special command in gonb that would interpret myFunc to be main. So a cell would look like this in Jupytext:

// %% [markdown]
// ## Example C: $x=5$:

// %%
func exampleC() {
  fmt.Printf("Square(%g)=%g\n", *x, Square(*x))
}
//gonb:%main exampleC -x=5

[janpfeifer]

I just went for a hike and was thinking more about it: one of the benefits of the Jupytext file were named .go and were proper Go, is that it can be compiled and tested with normal Go (without Jupyter). It could even be code shared between a proper Go program (or test), and read as Jupyter Notebook (or kept paired with a .ipynb).

So I like even more the idea of having a //gonb:<special gonb commands>, but having the //gonb: prefix removed (in this case by Jupytext, if possible) when converted/synced to a notebook. And adding the special %main <func_name> args... special command to gonb.

The above example could look like a file squared.go:

package square

// %% [markdown]
// # Defining a $x^2$ function
// It returns x*x.

// %%
func Square(x float64) float64 {
  return x * x
}

// %% [markdown]
// # Examples
//
// ## Example A: $x = 3$

// %%
var x = flag.Float64("x", 0.0, "Value of x")

func example() {
  fmt.Printf("Square(%g)=%g\n", *x, Square(*x))
}
//gonb:%main example -x=3

// %% [markdown]
// ## Example B: $x = 4$

// %%
//gonb:%main example -x=4

This way this compiles in Go, and we can even create on the side a file square_test.go that tests that the results of example() are what one wants, or create a separate main.go and offer the same functionality from the command line.

Any thoughts ?

[HaveF]

@janpfeifer ❤️ Thanks for your constructive comments.

• If you can support some tweaks, //gonb: seems to be the simplest, and has a certain scalability solution. I don't know if it is easy to change in jupytext. Let's wait @mwouts 's thoughts 💭
• I don't know if it is necessary to make the files converted by jupytext more like real go. After all, this part of the work is already done by gonb (such as adding necessary imports, etc.). My original use scenario is actually, when jupyter lab is not so convenient to edit, I can switch to a modern editor to edit, even use AI to modify directly in the modern editor, and then reload the changes in jupyter lab.

[mwouts]

Hi @HaveF , @janpfeifer , thank you for your useful inputs! That's very interesting. And thank you for your kind comments too!

To follow-up on the above,

1. I do have a preference too for generating .go files rather than .gonb (users can still use a custom prefix, e.g. export nb.go:percent if they want to get e.g. a .nb.go extension)
2. I like the idea of mapping %% (followed by nothing, or by a space - e.g. that would exclude the usual magic commands) in notebooks to //gonb:%% in .go files, and back. That's simple enough.

I'll ping you when I get a development version that you can try to see how it works on real notebooks.

[mwouts]

I might have something that works with the percent and markdown formats (not yet with the "light" format):

HATCH_BUILD_HOOKS_ENABLE=true pip install git+https://github.com/mwouts/jupytext.git@support_gonb

(remove HATCH_BUILD_HOOKS_ENABLE=true if you don't have nodejs and if you don't need the Jupytext Menu).

Let me know how it works for you. Thanks!