Enhancement Proposal - Request for Comment

[h4rm0n1c]

I thought this might make for a bit of nerdgasm, I've had some insights about the macintosh toolbox and VNC.

This is going to sound completely bonkers, but there's a good chance this client could be turned into a pixel accurate remote X session forwarded/RDP level of quality, I'm not joking.

I guess you could call this an RFC for MiniVNC.

Here's a onepager summary:

https://github.com/h4rm0n1c/macdevdocandexampleblob/blob/main/technical_breakthroughs/possibly_insane/MINIVNC_ENHANCEMENT_ONEPAGER.md

And here's a larger more detailed proposal if this takes anybody's interest:

https://github.com/h4rm0n1c/macdevdocandexampleblob/blob/main/technical_breakthroughs/possibly_insane/MINIVNC_ENHANCEMENT_PROPOSAL.md

Full Disclosure: this proposal was my idea and I only could have come up with it through actual exposure to apple's extremely detailed developer documentation, but I think it's fair to disclose that I was assisted by the latest ChatGPT 5.2 Extended Thinking Model, loaded up with 6 years worth of apple developer CDs raw text content from the late 80s to mid 1990s, this content was painstakingly extracted from 12 CDs that best captured key points in the platform's development over the span of time I considered to be most relevant to scope for Classic Mac OS.

I am glad to attempt a fork to see if I can prove the concept at some point and see what I can do with this if you think this is out of scope/does not match the project's goals or is seen as fruit of a poisoned tree due to my use of Ai.
(I try to at least respect people's boundaries as I know some are not fond of Ai)

I humbly submit this Request For Comment for your consideration.

[marciot]

Thank you for your interest in MiniVNC, I have read through your proposal, and I would like to point out that these ideas are not new. They were implemented in ChromiVNC. You can read about the toolbox patches to optimize change detection, for example. So what you propose, essentially, is to make MiniVNC into ChromiVNC.

My goal in designing MiniVNC was to make something that would run on early B&W Macs where the primary design constraint is keeping the code size small. Recall that early Macs had anywhere from 128K to 1MB of RAM that was shared between the operating system and the applications. Apple chose to put the toolbox in ROM because there is precious little RAM for executable code.

MiniVNC cuts into that budget, so it necessarily needs to be as small as possible and this means implementing the simplest, smallest VNC server with the minimum amount of executable code. Currently, the minimum memory resident portion of MiniVNC about 25K and this was accomplished by writing the minimum viable VNC server, not the most performant one.

It may be possible to get some performance gains by implementing the enhancements in your proposal, but every improvement will come at the expense of increasing that code size. Patching routines, for example, isn't free. It requires at least a dozen or so bytes of assembly code per patch. This can add up pretty fast.

Second, some of the anticipated performance gains are correct for later machines but are false for early Macs. Consider CopyBits and ScrollRect acceleration. This optimization makes sense when the underlying operating system and applications employ smooth window dragging and smooth scrolling, features that were not in widespread use until the late 1990, when CPU finally had enough raw power to make it possible. A machine of that era is probably capable of using ChromiVNC which already has such optimizations.

For older machines, the situation is that CopyBits is extremely slow on a 68000 CPU, especially on the horizontal axis, where bits need to be shifted within a byte, so "scrolling" will give you about 1 FPS on a good day, far too slow to be practical. If you look at early Macs, you will see windows are dragged as an outline and scrolling in apps is typically done by page or half-page increments, so optimizing CopyBits or ScrollRect in VNC would not really give much benefit.

So really, the question to ask is whether there is some machine out there that really needs something that is half-way between MiniVNC and ChromiVNC, and only then, does it make sense considering whether to make such a thing. If you decide the answer is yes, then you really ought to ask whether it makes sense to add features to MiniVNC to make it more suitable for later machines, or whether it makes more sense to port ChromiVNC to earlier machines. Of course, if you are willing to devote the time to it, either answer is acceptable!

[h4rm0n1c]

I was not aware of ChromiVNC because the only references left to it are on Web Archive, if I had known I probably would have tried this out.

Considering it's not apparently been touched for 20 years, maybe my efforts would be better spent there on a potential fork...

Thanks for the info, I guess we can just consider this a path explored and links to other vnc implementations on 68k macs, a learning experience, I just had a brainwave and had to share it in case it was useful. independent arrival at the same idea from first principles I guess...

Edit: Interesting, so the story has a fold, a third party vncpatches68k is required and... oh... the sources were uploaded to macintosh repository in 2021, that's cool.

Hmm... sounds like I have a target for a possible project that might benefit from some love and experiments. I wonder if I can get it up and running on something like Retro68...

[marciot]

I've been researching this for a few years, so probably the best I can do for you is to give you a brain dump on the research I have done, the dead ends I encountered, the leads I have explored, and how I would do it differently if did it all again.

First off, regarding ChromiVNC. I first came across it on the Internet Archive, but since then both the base program as well as the patches have been uploaded to Macintosh Garden (1, 2) and Macintosh Repository (3,4). It does not appear to me as if either site has archived the sources for ChromiVNC, so I would consider it a priority to do so. The sources are available through the Internet Archive 5 but ought to be uploaded to the Macintosh Garden and the Macintosh Repository.

The reason I chose not to use ChromiVNC was that it required OpenTransport, which does not run on earlier Macs. MacTCP can be used on any Mac going all the way back to the Lisa, so I believe investigating whether ChromiVNC can be back-ported to MacTCP would be a good line of investigation.

On a historical note, OpenTransport was cutting-edge when it was developed in 1995. It is based on System V STREAMS, which at the time was thought to be the natural successor to Berkeley Sockets. History has proven this to be incorrect, as Berkely Sockets remain the standard to this day. I consider this a reason the retro community ought to embrace MacTCP and consider OpenTransport a dead-end.

Despite history and compatibility being in its favor, the callback-API used in MacTCP is absolutely horrendous to work with. I did my best in MiniVNC, but it forced me into bad programming practices, such as overusing global variables as an alternative to stack-based variables that would not be retained across callbacks. I saw from writing MiniVNC how much I longed for threads and how much better it would be to interact with MacTCP through synchronous routines that would block a thread whenever data was unavailable -- this would eliminate the asynchronous callback hell in favor of an easier sequential model of programming.

Here again I did some research. Apple introduced their own threading API called the Thread Manager in System 7, but this is not very backwards compatible to older Macs. I then came across Ari Halberstadt's very excellent ThreadLib 6. It is a wholly self-contained thread library that does not rely on any Apple code at all and is said to be faster than Apple's own implementation. I was able to build the sample app and it worked fine on my Mac Plus running System 6.

This is as far as I got with my experimentation, but it did give me ideas of how I would have approached things differently were I to start from scratch. I would make an easy-to-use wrapper around MacTCP, similar to the one I used in MiniVNC, but integrate it with Ari's threading library so that it presented a synchronous interface to MacTCP. I would release an open-source library that would allow a developer to build modern MacTCP apps on retro computers going all the way back to the Lisa, using multiple threads for handling simultaneous connections and using a simple sequential programing style.

Then I would write a modern web server that ran under System 6 and was WebSockets capable. I would integrate MiniVNC into that and have it present a web page with an embedded VNC client written in WebAssembly. I would trim down MiniVNC to support only the VNC encodings that give the best performance on vintage Macs . I would make sure the client supported indexed color as this is required for B&W Macs and is very desirable for all the other Macs. Doing color conversions on the 68000 is very inefficient and the fact that most VNC encodings uses 32-bit color is a huge bottleneck. Currently RealVNC is the only client that supports indexed color natively, but it has its own limitations, such as using a proprietary undocumented protocol for file transfers. Having my own a web-based client would allow me to solve these issues nicely.

[h4rm0n1c]

Thanks for the notes, half of this I'd already started to grasp but it's useful having it all in one place like this, I did find the sources very quickly.

I'm super aggressive in abstract structure type planning for my software, I like wrappers and clever uses of pointers to make for lots of reusable toolbox style code, in fact, you'll find stuff like keepalive helpers for networking in my MinGW based win32 app where you just pass it a pointer to the socket and away it goes, I have experience with microcontroller based development as well so I'm super resource aware once I grasp the tools for monitoring such things, its a funny coincidence you mention network stacks and wrappers, I've already begun work on a dual network stack app as a side effect of wanting to write an app for System 7 that has more to do with Speech Manager, I'm making a Mac equivalent of my windows app just as a sort of proof of concept and as a cool little thing people can play with.

I get stuff working then I break it out into reusable source files, you can see my MacVox68 repo, it's full of UI related helpers already, I really need to structure my headers and some files in my source tree with a new layer of abstraction next once I'm done with the UI (I'm currently on disability healing from a rather awful long term condition I have nothing but time)

There's a good chance I'm going to at some point, pull down the chromi and wrapper sources, make a repo, I see value in getting such a thing up on github and compiling on a modern tool chain like Retro68 if at all possible, gives me a good chance to familiarize myself more with the particularities of VNC.

That stuff about threadlib is gonna come in handy later for other projects, thanks... :)

Maybe I use the third party wrapper as a sort of template for updating the build in wrapper as well, I've got nothing but time and disability payments while I heal and that's going to be a months to years long thing.

you make an excellent point about dedicated custom clients being a potential option. and it sparked some additional thoughts...

VNC only became a standard of sorts by getting started and putting something out, who's to say a dedicated client is the wrong path at this point? maybe the VNC protocol and clients, are the problem? maybe we need an intermediary proxy app that you run on your workstation, it could be as simple as a console or headless app that can take a new inhouse format and then translates back and forth to VNC, now there's an idea... a VNC arbitrator that runs on a modern system that does the hard VNC work for classic macs, and you could put all the annoying bits in there and use modern hardware features to deal with VNC itself... then you can get away with all kinds of weird custom encodings and algos that don't have to fit VNC's model for server... then the on device server component becomes a stub that does what it needs to efficiently push the minimal data over the network, this also makes it easier to start fresh with a MacTCP/OpenTransport wrapper based dual stack solution that allows us to have compile targets for a server stub for '000 to '040.

summed up, it just occurred to me that it's possible to remove VNC's design as a constraint entirely by arbitrating VNC from an intermediary component...

I'm liking the sound of this, it'd be a lot of work, but like I said, I have time.