Continue publishing under 2-clause BSD license

[ymd-stella]

Check out the work required to continue publishing under the BSD license.

#35 (comment)

I think it would be good to clarify that the similarity between OpenVSLAM and ORB_SLAM2 is based on the ORB_SLAM2 paper

Tasks:

• (moved) Clarify the functional correspondence between the ORB_SLAM2 code and OpenVSLAM code and evaluate the similarities #239
• Clarify the scope of contribution for each paper and treat it appropriately in README.md (for moral issue)
  • ORB_SLAM/ORB_SLAM2
  • ProSLAM
  • UcoSLAM

[ymd-stella]

Extracted similar modules between ORB_SLAM2 and OpenVSLAM.

• data modules
  • Frame - data::frame
  • KeyFrame - data::keyframe
  • MapPoint - data::landmark
  • Map - data::map_database
  • KeyFrameDatabase - data::bow_database, module::loop_detector, module::relocalizer
• visualization modules
  • Viewer - pangolin_viewer
  • MapDrawer, FrameDrawer - publish::*
• other modules
  • System - system
  • Initializer - solve::fundamental_solver, solve::homography_solver, solve::essential_solver, initialize::*
  • Tracking - tracking_module, module::frame_tracker, module::keyframe_inserter, module::local_map_updater
  • LocalMapping - mapping_module, module::local_map_cleaner, module::two_view_triangulator
  • LoopClosing - global_optimization_module, module::loop_bundle_adjuster
  • Optimizer - optimize::*
  • ORBextractor - feature::*
  • ORBmatcher - match::*
  • PnPsolver - solve::pnp_solver
  • Sim3Solver - solve::sim3_solver

Based on these correspondences, similarity should be assessed by making comparisons at a appropriately abstract level. It is important to note that their creative expression may be limited by ideas. (Please refer Merger doctrine.)

(As a side note, the structure of the module and the choice of libraries to use are clearly ideas rather than creative expressions. Therefore, I think there is no problem with similarities.)

This is too abstract task description for others to work with, so I'll give an example.
The following is the part of the Initializer that achieves the same function.

• https://github.com/OpenVSLAM-Community/openvslam/blob/b6c40f671f4fa226b933df47dd1b466dbfdab2ee/src/openvslam/solve/homography_solver.cc#L13-L76
• https://github.com/raulmur/ORB_SLAM2/blob/f2e6f51cdc8d067655d90a78c06261378e07e8f3/src/Initializer.cc#L124-L172

I think this is what @masahi mentioned in xdspacelab/openvslam#12 (comment).
As for the fact that the initialization order of the variables is exactly the same, I don't think it is acceptable if you look at it with strict evaluation criteria.

@SteveMacenski I would like to ask your opinion on whether the above similarity is acceptable or not, and if not, how it should be handled to continue publishing under 2-clause BSD license .

[SteveMacenski]

There's significant changes in the project architecture / filesystem and "although written from scratch" (from that link) tells me its on the right side of that line. There's better structured code for things like the camera models and such. Tons of optimizations and better written / faster ways of doing things ORB-SLAM2 completely lacks and uses OpenCV for.

The things you link to are similar to a degree. Finding a homography is an incredibly common thing to do and there's only so many ways to do it. How many formulations are there to find a homography given more or less the same sets of information? Within research code, there are just a few ways people that do this alot just do it naturally because they're all being educated by the same textbooks and graduate course slides / openCV demo code.

I look at those 2 sets of code and see completely insufficient similarities to count as derivative work. They use completely different underlying libraries (openCV vs something they made themselves) but only the same math. I agree with kensakurada's comment on the last comment in that thread. normalize is a very common function name in all of computer vision programming.

I think the main thing you're just seeing here is that they employ the same math, which if I were to implement this would look similar to both of these. But OpenVSLAM's is clearly different (different libraries, where this is happening in the codebase, optimizations and design decisions made, etc).

In my opinion, this is all above board. They've done things their own unique way based on the same ideas. Noting that 2 codebases are similar which employ similar mathematical and theoretical formulations is only natural, but the design intent across them and how they go about solving these problems based on their own requirements are very different.

[masahi]

I don't know the current code, but when I was looking at it, the similarity was way more than coincidence. That one can only tell if you spend a lot of time on ORB_slam codebase. At that time I also wondered publishing openvslam as bsd while the original code was GPL is a good idea (in terms of morality etc).

[ymd-stella]

@masahi I think there are similarities in terms of structure and algorithms, but I think that is because it is based on the methodology of the ORB_SLAM2 paper.
However I think the similarity at the code level is low. (It would appear that the only similarities are in the few areas where there is less freedom of expression.)
Just to confirm, is the similarity you are pointing out the former(structure/algorithm level) or the latter(code level)?

[masahi]

It was almost two years ago so my memory is not clear, but I would say it is code level. I didn't read the paper because it was not out yet, see xdspacelab/openvslam#12 (comment), so the only way to understand this work was by reading the code. I had code-level understanding of ORB_slam (I had to debug their code because it was full of bugs), so reading openvslam code it was obvious to me that what it does is basically the same as ORB_slam. My impression back then was this is "ORB_Slam2 rewritten from scratch in a clean way", that's it.

It is probably ok from a legal pov, since this is a from scratch implementation and non-tech people don't understand the code (and hence such ambiguous statement from the termination release). But seeing this code published without acknowledgement to ORB_slam, with BSD license, was enough to convince me that this was not something I wanted to use (for moral reason).

[ymd-stella]

I will describe my opinion from a moral point of view. I take "for moral reason" to mean because it goes against the will of the ORB_SLAM2 developers.

Let's consider the will of the ORB_SLAM2 developers in choosing the GPL. I think the purpose of GPL is to encourage people to contribute to the community by making the code public. In other words, the goal is to make ORB_SLAM2 a better Visual SLAM, or for the ORB_SLAM2 community to get a better Visual SLAM.

Since the repository of ORB_SLAM2 has been abandoned for maintenance, their actions show that the intention of the developers of ORB_SLAM2 is not the former now. On the other hand, it is a contribution to the ORB_SLAM2 community that OpenVSLAM continues to be developed as Visual SLAM that includes the features of ORB_SLAM2. Therefore, I believe it does not against the will of ORB_SLAM2 developers.

In order to develop OpenVSLAM that inherits the idea of ORB_SLAM2, I believe it is beneficial that OpenVSLAM is licensed under the BSD license. I think that using the GPL will lead to shrinking the community by narrowing the real scope of application.

(Of course, we need to make sure there are no legal issues.)

[masahi]

I do agree that bsd would be beneficial to the community. What I don't like, as a person who is passionate about open source software, is the dishonesty and lack of respect for ORB_slam from the openvslam team, refusing to admit that this is ORB_slam derivative work. In the termination release, they even described ORB_slam as "another SLAM software that happens to be similar to openvslam", when ORB_slam was released years before openvslam. Let me remind you that the concern around the similarity between the two was raised from both outside (see xdspacelab/openvslam#12) and inside (see the termination release) of their org. Do you still believe that openvslam is genuinely novel and worth completely different licensing than ORB_slam?

Probably the best way forward is to contact ORB_slam authors and ask if they are ok with openvslam being BSD. The reason they chose GPL and not more liberal ones is probably to prevent commercial use of their software and making money. They also have a new system and it is still GPL https://github.com/UZ-SLAMLab/ORB_SLAM3. If the reason for GPL was simply due to their dependencies being GPL, then there is a good chance they are no problem openvslam being BSD (assuming it has no GPL deps).

[ymd-stella]

Do you still believe that openvslam is genuinely novel and worth completely different licensing than ORB_slam?

License protects creative expression, not ideas; Ideas are protected by patents. Licensing does not impose any restrictions on the use of ideas. Even if we implement a new library based on the idea of ORB_SLAM2 without changing any of its functionality, we can still publish it under our own license. (As long as we don't steal the creative expression in the ORB_SLAM2 code.)

You may be confused because ORB_SLAM2 is a complex system. Maybe it's better to replace it with a simple library that you know well and do some thought experiments. (In addition, the fact that some of the code is suspected to have actually been stolen may have contributed to the confusion.)

[masahi]

Sure I get your point, indeed what I said above was not precise. So idea being the same is no problem wrt licensing, and whether or not it is novel doesn't matter. What you are concerned about, and the reason the code was taken down, is code similarity, correct?

I don't think code was stolen or copy-pasted as is. They reimplemented the same idea differently using the original code as reference, but failed to acknowledge the original work. You should also note the last sentence in the termination release: "If you are considering and using the downloaded OpenVSLAM as a derivative work based on ORB-SLAM2 under the GPLv3 license, there may be no issues on this matter.". So they admit that if you continue BSD, there could be issues.

[ymd-stella]

but failed to acknowledge the original work.

Could you please tell me more about it?
It seems to be listed as a reference in README.md, but does that mean it's not enough?

You should also note the last sentence in the termination release: "If you are considering and using the downloaded OpenVSLAM as a derivative work based on ORB-SLAM2 under the GPLv3 license, there may be no issues on this matter.". So they admit that if you continue BSD, there could be issues.

Rather than saying that continuing to use BSD will cause problems, the claim seems to be that using GPLv3 will eliminate any doubt of problems. I think those are distinctly different claims.

[masahi]

Could you please tell me more about it?

I think having ORB_slam listed as one of references doesn't count as an acknowledgement, since unlike other systems in the reference, openvslam code is based entirely on ORB_slam. I don't know the code of other systems, but given that the termination release only mentions ORB_slam, I think we can assume that other systems are not concerned. So I'd expect a special treatment for ORB_slam, rather than merely listing it as one of prior works. Also when xdspacelab/openvslam#12 was opened, there was no reference to prior work whatsoever. From these points, I'm 100% sure that the authors deliberately tried to hide the fact their system is ORB derivative. That's what I mean by "they failed to acknowledge the original work". Hope this makes sense.

continuing to use BSD will cause problems

I agree this is a different statement, what I said is there could be issues. I think this is a reasonable interpretation of that statement.

[ymd-stella]

I don't know the code of other systems, but given that the termination release only mentions ORB_slam, I think we can assume that other systems are not concerned. So I'd expect a special treatment for ORB_slam, rather than merely listing it as one of prior works.

For example, is it sufficient to include it in the overview in README.md?

From these points, I'm 100% sure that the authors deliberately tried to hide the fact their system is ORB derivative.

I think this is an overstatement. I checked the history, and in fact, other references such as ProSLAM and UcoSLAM were added at the same time. They are also clearly listed as references in the paper. Please be careful about what you say so as not to defame the authors.

[masahi]

For example, is it sufficient to include it in the overview in README.md?

Yes, in my opinion, the readme should clearly state that this is an ORB derivative work. But I'm not an openvslam user, so it is up to you.

I checked the history, and in fact, other references such as ProSLAM and UcoSLAM were added at the same time. They are also clearly listed as references in the paper.

I don't understand how your statements above relate to my claim of the authors hiding the fact that their system is ORB derivative? I know these references are added at the same time. Treating ORB just like any other prior works is the very reason of my suspicion.

My confidence is also based on my experience and knowledge of ORB_slam codebase, and openvslam authors refusing to admit that their work is ORB derivative. I don't think authors acted on a good faith. But again, I'm not a user, just wanted to give my perspective.

[ymd-stella]

Thanks for your comments.
I just realized that the paper was not available at the time of xdspacelab/openvslam#12 (comment).

In conclusion, it seems that the heart of the problem that @masahi pointed out is simply a lack of description. For each of the papers that OpenVSLAM refers to, it needs to be made clear which part of the software is involved. I believe that if the contribution of the papers is clearly shown, the moral problem will be solved.
@masahi states that he does not know about ProSLAM or UcoSLAM, and then says that ORB_SLAM2 should be given special treatment because of its large contribution. This opinion is probably correct, but it is impossible for someone who does not know about ProSLAM or UcoSLAM to judge whether the contribution is large or not to the whole. Therefore, it seems that we need to first clarify where ProSLAM and UcoSLAM are contributing in OpenVSLAM.

(Note that whether or not it is possible to use a BSD license is different from the moral issue above. The similarity of the code should continue to be checked separately.)

[masahi]

Sounds good, thanks. I agree with your point on licensing and moral issues being separate topics.

ORB_SLAM2 should be given special treatment because of its large contribution

A supporting evidence to pick up ORB_slam, among other works, is the fact there are more than one person, who:

• Have read both ORB_slam and openvslam code
• And raised concerns around code similarity

This includes me, the person who sent xdspacelab/openvslam#12 (see his comment), and internal people at authors' org. This didn't happened for other systems. If you do the same exercise, I believe there is a good chance you'd reach the same conclusion.

Another evidence is only ORB_slam was mentioned in the termination release.

These facts shouldn't be a coincidence. This is the reason I believe openvslam is ORB_slam2 derivative, and hence it should be given special treatment in the acknowledgement.

So I highly recommend taking a look at ORB_slam code base in detail. As one data point, I spent about a month trying to make the output of ORB_slam deterministic. I found a number of bugs in the process. So I believe I know what I am talking about here.