Skip to content

v.net: Add JSON output support#7088

Open
tangelll wants to merge 16 commits intoOSGeo:mainfrom
tangelll:vnet-json
Open

v.net: Add JSON output support#7088
tangelll wants to merge 16 commits intoOSGeo:mainfrom
tangelll:vnet-json

Conversation

@tangelll
Copy link

@tangelll tangelll commented Feb 14, 2026

This PR adds JSON output support to the v.net module using the gjson/parson library.

Description

This PR introduces JSON output support for the v.net module, specifically for the operation=nodes task. This enhancement allows users to programmatically capture network statistics (such as new node counts) for use in automated workflows and testing pipelines.

Changes

  • Core Logic: Integrated G_JSON support in main.c to handle structured output.
  • CLI Arguments: Added a new format parameter to choose between plain (default) and json.
  • Documentation: Updated v.net.md with usage examples and guidelines for JSON output.
  • Testing: Added an automated Python test case in testsuite/test_v_net.py to ensure output consistency.

How to Test

To verify the implementation, you can use the North Carolina sample dataset.

  1. Manual Verification:
# Copy sample data
g.copy vector=streets_wake,my_streets

# Run the module with JSON format for nreport
testsuite % v.net input=my_streets_ready operation=nreport nlayer=2 format=json

#Run the module with JSON format for report
v.net input=streets output=test_vnet operation=report  format=json 

Expected JSON Output :

[
{
"line_cat": 49599,
"start_node_cat": -1,
"end_node_cat": -1
},
...
]

  1. Automated Test Suite: Run the newly added test case:
python3 vector/v.net/testsuite/test_v_net.py

Copying features...
100%
Building topology for vector map test_vnet@PERMANENT...
Registering primitives...
40000
Copying attributes...
Building topology for vector map test_vnet@PERMANENT...
Registering primitives...
50000
v.net complete. 323 lines (network arcs) written to output.

.
Copying features...
100%
Building topology for vector map test_vnet@PERMANENT...
Registering primitives...
40000
Copying attributes...
Building topology for vector map test_vnet@PERMANENT...
Registering primitives...
50000
v.net complete. 156 lines (network arcs) written to output.

.
Copying attributes...
Building topology for vector map test_vnet@PERMANENT...
Registering primitives...
90000
v.net complete. 41813 new points (nodes) written to output.

.v.net complete.
.v.net complete.
.OK

@github-actions github-actions bot added vector Related to vector data processing Python Related code is in Python C Related code is in C module docs markdown Related to markdown, markdown files tests Related to Test Suite labels Feb 14, 2026
Copy link
Contributor

@petrasovaa petrasovaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use pre-commit.

struct boxlist *List;
G_JSON_Value *root_value = NULL;
G_JSON_Array *root_array = NULL;
if (format && strcmp(format, "json") == 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the other tools with json support, they use enums.

]
```

### Produce Report in JSON format
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be consistent with the other examples, use nc_spm_08_grass7 dataset. Consider moving the json example more towards the top where the example is already using report.

@tangelll
Copy link
Author

I added a check for G_OPT_F_FORMAT in args.c because it was causing a segmentation fault and UI warnings on my environment when the description was missing.

@github-actions github-actions bot added the CI Continuous integration label Feb 19, 2026
Comment on lines +13 to +15
opt->format->key = "format";
if (!opt->format->description)
opt->format->description = _("Output format");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should not be any need to add these.

Suggested change
opt->format->key = "format";
if (!opt->format->description)
opt->format->description = _("Output format");

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"After removing those parts and running make, the build gets stuck at the UI description generation step with the following warnings: 'Missing option key' and 'Description for option <?> missing'.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested it and it works fine for me. Look at other usages of G_OPT_F_FORMAT.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you'd need to clean before make, like make clean, make libsclean, or make distclean. With the last one, you'd need to run configure again

nnodes, x, y, z, cat_line);
}
fprintf(stdout, "%d %d %d\n", cat_line, cat_node[0], cat_node[1]);
if (root_array) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be more readable to test the format enum value than root_array

@tangelll
Copy link
Author

Hi @petrasovaa , just wanted to check if you had a chance to look at this PR when you have a moment. Thanks!

G_JSON_Value *lines_val =
root_arr ? G_json_value_init_array() : NULL;

if (root_arr) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My previous comment about testing the enum instead of root_array applies here as well and check other places.

LIBES = $(VECTORLIB) $(GISLIB) $(DBMILIB)
DEPENDENCIES = $(VECTORDEP) $(GISDEP) $(DBMIDEP)
LIBES = $(VECTORLIB) $(GISLIB) $(DBMILIB) $(PARSONLIB)
DEPENDENCIES = $(VECTORDEP) $(GISDEP) $(DBMIDEP) $(PARSONDEP)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other Makefiles don't add PARSONDEP, I am honestly not sure about this, but to keep it consistent, I would remove this.

@petrasovaa
Copy link
Contributor

Was it closed by mistake?

@echoix
Copy link
Member

echoix commented Feb 26, 2026

Reviewing is hard when force merged are made after there was a first review. Not only we Connor just read where we left off, but comments posted are on commits (and lines) that don't exist anymore, and are now kind of "detached" from the PR changed files view (and commits view)

@echoix
Copy link
Member

echoix commented Feb 26, 2026

The commit 72dd6b2 seems affected by a bad rebase, its tricky when we force-push.

@tangelll
Copy link
Author

The commit 72dd6b2 seems affected by a bad rebase, its tricky when we force-push.

Hi!, yes you are right I opened a clean new branch and I will pull request it from there today.. thank you!.

@echoix
Copy link
Member

echoix commented Feb 26, 2026

We didn't review more since, you might as well fix up the same branch, so we still have the context of what we already discussed (instead of having it lost in a new PR)

@echoix echoix removed CI Continuous integration vector Related to vector data processing Python Related code is in Python C Related code is in C translation Message translation related libraries module docs markdown Related to markdown, markdown files general tests Related to Test Suite CMake labels Feb 27, 2026
@github-actions github-actions bot added vector Related to vector data processing Python Related code is in Python C Related code is in C module docs markdown Related to markdown, markdown files tests Related to Test Suite labels Feb 27, 2026
@github-actions github-actions bot added the CMake label Feb 27, 2026
@tangelll
Copy link
Author

@echoix I've verified the changes locally with super-linter. Could you please approve the workflows to run the CI tests?

"""Verify that v.net report produces valid and accurate JSON output"""
vnet_module = SimpleModule(
"v.net",
input="streets_net",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this found? It doesn't seem to be in the test dataset, or created just for the test, as the same tests fail on all platforms

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, streets_net was part of my local North Carolina sample dataset and I mistakenly assumed it was available in the test environment. I will update the test to either use a standard test dataset.Sorry for the oversight!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you double check which of the tests take long? In the error logs, these 6 tests in the file took 69 seconds to run on Ubuntu, which is quite long in my opinion. Maybe they were already long, but take a look to see if it is the newly added tests.

About 38.4sec on macOS (the Apple silicon chips are really fast, so it's normal, about 2x as fast as the Linux runners usually).

66 sec for the Linux CMake

Windows didn't finish run it yet.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've optimized the tests to address the execution time concerns. The v.net network preparation steps are now moved to setUpClass, so they only run once for the entire test suite. My local tests show a significant reduction in total runtime. I've also ensured that existing tests and the tearDown logic remain intact.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the before and after (approx), and what was the other tests to compare to?

Copy link
Author

@tangelll tangelll Mar 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized I was preparing the heavy streets dataset twice (once for each new test). I have now moved the network preparation to setUpClass so it only runs once. This fixed the slowdown. Before it was 38 sec. in my local mac now it is 28 sec. in my local mac.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C Related code is in C CMake docs markdown Related to markdown, markdown files module Python Related code is in Python tests Related to Test Suite vector Related to vector data processing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants