Version 5.0.0 proposal

[josepot]

As much as I love Reselect, I hold the crazy idea that Reselect creates a leaky abstraction by exposing its 2 lowest level API functions (createSelectorCreator and defaultMemoize).

I think that if it didn't do that, then it would be possible to solve the problem of "shared-selectors" in a way that it would be almost transparent for the developer. I discuss that idea at length in this post. Also, I have created this library just to make sure that what I'm about to suggest is not totally crazy.

The API that I would like to suggest for version 5 would be something like this:

• createSelector: same signature as today
• createStructuredSelector: like the current one, but removing its second parameter
• createKeySelector: an enhancer that lets the library know that the selector that is being enhanced returns an special type of result: the identifier of the instance(s) that consume the selector.

With that API it would be possible to solve the problem of shared selectors, just like redux-views does.

Just in case that it is not obvious: I have not created Redux-Views to "compete" with Reselect. I did it just to show that it is possible to implement the API that I'm suggesting. If Reselect decides to go down this road (or a similar one) I would love to help and I would immediately deprecate Redux-Views, of course.

[theKashey]

👍from a low-level API perspective, your idea is perfect. The only moment I don't quite get - .use nobody is calling(except .use itself) and how you are going to clean memory.

I've also spiked a few ways to make reselect more stable:

• kashe - a WeakMap based memoization. It just stores result inside the argument itself.
  It's possible to have a unique pair state -> selector. For example, I would have a unique user record from a first selector, I could use as a key for WeakMap, which is just a local variable inside createSelector.

const getUserSummary = createSelector(
  getUser,
  ....
  doSomethingWithItAll
)
// ---- something like
WeakMaps[getUserSummary].set(getUser(state, id), result); //

This approach hovewer does not play well with simple selectors like

const getUsersInGroup = (users, group) => users.filter(user => user.group === group)

as long as you cannot use group(string) as a key for a WeakMap, and still able to hold only one result. But it might be not a problem.

• a year ago I've written another library - memoize-state, which also could handle only one, last, result. In the same time, I wanted to have proper per-instance memoization, so I've used redux mapStateToProps factory to separately wrap mapStateToProps for every component. In the same time, I still wanted to have shared cache for non-per-instance selectors. So I've wrapped it twice - the first level is per-instance memoization, with the second level shared across instances.

• later I've used the same idea in kashe inbox function - it "commands" internal cache to use another "weakmap" storage, "forking" realty. As a result - you might have per instance memoization(cache is bound to the "instance") from one side, and shared one from another side.

const getUsersInGroup = (users, group) => users.filter(user => user.group === group)
const memoizedgetUsersInGroup = kashe(getUsersInGroup))

const mapStateToProps = () => {
   const perInstance = kashe(memoizedgetUsersInGroup);
   return (state,props) => ({ user: perInstance(state, props.group) });
};

Unfortunately this still requires some engeneering mindset. Fortunately could be done automatically from redux side (needs some injections)

• the third approach is to utilize hooks. Not hooks, but idea behind hooks - "stack".

const createSelector = () => {
   return () => {
      if(!GLOBAL_RESELECT_CACHE) { 
        // first selectors reads a shared variable from a fiber.
        GLOBAL_RESELECT_CACHE = useRef();
      }
      try{
      pushSelector();
      .... everything as usual, all cache is in GLOBAL_RESELECT_CACHE     
      } finally {
      if(!popSelector()) {
        // redux could read this variable, and inject it before running mapStateToProps by itself to mitigate `useRef` usage.
        LAST_GLOBAL_RESELECT_CACHE = GLOBAL_RESELECT_CACHE;

        // last selector cleans state
        GLOBAL_RESELECT_CACHE = null;
      }
      }
   }
}

Maintaing "stack" manually would allow to escape from hooks rules, and from hooks itself. Just store result in two places - and call it a day.

The solution could be memory efficient, with automatic per-instance cache cleanup with instance end of life. Still could use "standard" reselect underneath to share shareable data.

Drawback - you might call the same selector with different props even within one component, and this approach would not help.

Overall:

• cache should be stored in weakmaps
• per instance memoization should be easy and automatic
• API to start(createRef) or set(globals) cache dimension should be exposed
• it shall not be bound to React

I am gonna try to spike useReselect to try the idea in a wild.

This all mean only one - keep moving forward!

[theKashey]

https://github.com/theKashey/useReselect

[josepot]

Hi @theKashey ,

Thanks a lot for your feedback.

Before I share my opinions on your attempts to "make reselect more stable", let me explain a few things about what I'm proposing that I have not explained very well. Sorry about that.

The only moment I don't quite get - .use nobody is calling(except .use itself) and how you are going to clean memory

There is this section inside the Redux-Views README that talks about that.

Basically, what it says is that -as of today- if you want to have "automatic" cache invalidation, you have 2 options:

1. Use a custom connect HOC, rather than directly using the one provided by React-Recux, something more or less like this:

const customConnect = (selector, ...rest) => {
  const { keySelector, use } = selector || {}
  return Base => {
    const Component = connect(selector, ...rest)(Base);
    return props => {
      const key = useMemo(
        () => keySelector ? keySelector(null, props) : undefined,
        [keySelector, props]
      );
      useEffect(() => use && use(key), [use, key])
      return <Component {...props} />;
    }
  }
}

2. Use React-Redux-Lean instead of using React-Redux (which BTW: it has hooks already)

I'm currently working on a much better version of React-Redux-Lean, which should improve its performance dramatically... We will see.

However, if Reselect decided to accept my version 5 proposal, we should probably convince React-Redux to account for "usable" selectors, and I can see how that could be problematic/polemic. IMO React-Redux already set a similar precedent by accepting factory-selectors... But I guess that it doesn't really compare.

👍from a low-level API perspective, your idea is perfect

I was hopping that my idea was perfect from a high-level API perspective 😅.

Let me explain why I say this using a real-life example: The other day at work one of my co-workers (@voliva) was having problems with a React-Redux App because a form was super sluggish. We debugged that together and we found that the issue was that all the instances of a list component were being re-rendered every time that the user pressed a key on a form-field (and their selectors were being re-computed for "no reason"). You can already imagine what the problem was, right? They hadn't used factory-selectors for creating the "shared-selectors" of the items of that list. For me that was the perfect chance to ask for a favour: Could you please remove reselect, add redux-views and use webpack to force Reselect to resolve to Redux-Views? Also, please change the definition of one selector so that it uses createKeySelector. My coworker tried that and... voila! Everything worked amazingly well! The problem was gone and the rest of the App was fully functional.

I asked @voliva not to use that in production (yet). However, it made me happy to see that the API that I'm proposing for version 5 is compatible with the way that most devs use Reselect, and that at the same time it solves the "shared-selectors" problem.

---

I was going to let you know what my thoughts are on your different spikes of how "to make reselect more stable", but then I realised that that is a separate conversation and that it is a bit out of topic in here. In this issue I would like to discuss whether this new API that I'm proposing makes sense or not, and if there are any other draw-backs apart from the ones that I'm already pointing out.

Please, do not get me wrong, I want to share what my thoughts are on your spikes, for sure. However, since you are not proposing any of those as a possible next version major version of Reselect, I think that it would be better to have that conversation in a different issue 🙂.

That is why I have created the following issue in Redux-Views, so that we can continue that conversation there. Could you please copy-paste in that issue, from the line:

I've also spiked a few ways to make reselect more stable:

Onward, so that we can continue that conversation in there? Thanks!

[faceyspacey]

I wouldn’t be so concerned about contributing back to reselect. I would be happy to use your product.

If you are the most motivated in this space as of 2019, build it under your name and collect credit which will help you get jobs and clients, which in turn will reinforce your motivation to maintain and evolve it. Having successful repos under your name can make you hundreds of thousands of dollars and change your life. My recommendation is to not martyr yourself.

The sentiment is nice, but isn’t necessary beyond getting the community to look at your solution. Fork React-Redux if you have to (which I see you have). Fork Redux itself. Preserving the old for so long has just resulted in decay. Redux is in great need of a makeover; it’s inevitable. There’s so many places we never took these tools—mainly because people get the wrong idea that there’s nothing left to do and it’s “done.”

What you are working on is a key piece. Be bold and confident in what you intuit to be the way forward.

If, on the other hand, you’re not motivated to carry the flag for very long and just wanna pass off your discoveries, that’s another story. But don’t be afraid to replace the status quo—that’s what software is about and what is currently lacking in the stagnant React landscape.

With hooks it might not be clear, but the pace of innovation has greatly slowed in the space compared to a few years ago...And innovation trumps fatigue, to say the least.

React, with hooks and suspense, is making an attempt to replace Redux. There will need to be a Redux renaissance for it to survive. And I think it should survive because there are things we can’t do with hooks/suspense, and more importantly latent capabilities that have yet to evolve in the Redux path.

What you are working on is a key pillar in the Redux renaissance.

Ps. I just read your medium article. It was excellent by the way. Write more. A lot more and post them everywhere if you don’t want to be a tree that falls in the forrest that nobody hears. The core of what your doing here revolves around promotion. From experience, don’t expect the powers that be to give a shit or help you. That’s clear by how few likes you got on your amazing launch article on medium from a month ago. You deserve a lot more. That’s a sign of how shitty the React game is and how little influencers ultimately know. Don’t depend on them. You gotta do your own promotion, and an endless amount of it. The developers that know the most work on real projects, often ones they own or are technical leads in, and have little time for the misleading echo chamber of react tweets and articles. Next suggestion: build fuller tools worthy of your time promoting it. Your fork of React-Redux is an example of this, whereas your fork of reselect isn’t. In other words, build a tool big enough that it’s a primary pillar to how people build their apps, rather than a hidden implementation detail. Something with a significant api surface and many ways to use it will bring you tons of paying clients in need of your wisdom. There’s disproportionate gains for open source developers that build a comprehensive system vs those that make minor fixes. Your work is actually pretty far along the spectrum toward being a contender people need to take seriously. You are doing real innovation. Unfortunately obvious innovation that needed to be done long ago. But thankfully you are finally doing it...I make those last comments just to indicate that the ecosystem is totally (and always) up for grabs for those that have the time and vision to enhance it.

[theKashey]

I was hopping that my idea was perfect from a high-level API perspective 😅.

And then please let me retell your story in other words: He walks into the factory, takes a look at the Big Machine, grabs a sledge hammer, and whacks the machine once whereupon the machine starts right up. Graybeard leaves and the company is making money again. The next day Manager receives a bill from Graybeard for $5,000. Hammer: $5. Knowing where to hit the machine with hammer: $4995 old story

That's why I called it a low level API - it's a hammer. You worth the rest 4995$. The high-level API should or handle keys automatically, or help you build right selectors, ie hight right spots to hammer.

The current one is a low-level, and, honestly, is a best fit for reselect from API look-n-feel prospective.

[ellbee]

Hey @josepot, thanks for this. I think it is worth serious consideration. Sharing selectors is definitely an area where Reselect doesn't push the user to "fall into the pit of success". I consider defaultMemoize and createSelectorCreator to be escape hatches but I have no idea how widely used they are in practice. This library get 1,200,000(!) downloads a week so if we go this route we will have to think about how to make the upgrade as painless as possible.

[josepot]

Hey @ellbee, thanks for considering this proposal!

I totally agree. If Reselect decided to go down this road, then we should act with caution. I think that a sensible approach would be to move defaultMemoize and createSelectorCreator to a different repo (reselect-old-internals or something like that) and then provide a codemod to make the migration to v5 as painless as possible for everyone.

I think that most Reselect users don't use those 2 functions, so I hope that most devs won't need to use that codemod. Although, I have used createSelectorCreator myself 😅, so maybe I'm wrong...

Also, I think that it would be very interesting to know how ppl have been using those 2 functions... Because IMO hiding those 2 functions would open the door to further improvements on Reselect's API.

For instance:

• A createBranchSelector or a createSwitchSelector or something similar could be pretty cool... For those cases when the output of a selector determines which dependency should be evaluated. The same functionality can be accomplished today using createSelector and then doing that branching in the compute function. However, that implies that all the dependencies will be evaluated when there is no need for that. Providing a declarative API it's a win/win: it reduces boilerplate for the developers, it makes their code more declarative and it allows the library to better understand how things are being used, which is what makes perf improvements possible.

• A propSelector could be very interesting too. So that Reselect can differentiate between those selectors that actually perform computations and the ones that just act like lenses on the state... IMO that could open the door to some pretty cool perf improvements, but it would take me too long to explain this, so I will leave it here 🙈

Anyways, I'm getting ahead of myself. However, I do think that moving those 2 functions outside of Reselect and asking feedback to the users that have been leveraging them could lead to further improvements on Reselect's API, which would also make Redux (and others) apps even more performant.

---

cc @faceyspacey and @theKashey sorry that I responded to @ellbee before I responded to you. I have been pretty sick these last 2 days and today I'm starting to feel a bit better. I will respond to you before EOD today, for sure!

[markerikson]

FWIW, a quick way to get a napkin math estimate of usage is to do a code search on Github:

https://github.com/search?l=JavaScript&q=reselect+defaultmemoize&type=Code

[josepot]

FWIW, a quick way to get a napkin math estimate of usage is to do a code search on Github:

Wow, thanks @markerikson! That is a great idea. Let's # of usages:

GitHub search results of reselect + defaultMemoize: ~800

GitHub search results of reselect + createSelectorCreator: ~750

GitHub search results of reselect + createSelector: ~60,000

So, according to that "napkin math", it seems that ~98% of Reselect users are not using those 2 low-level API functions.

Let's keep in mind that what I'm proposing is a major version update. So, I think that it should be "ok" to remove those 2 from Reselect. Especially if we keep them available in another repo and we provide some migration notes and tools (i.e codemod).

[markerikson]

On the flip side, do these have to be removed? Could the new stuff be added and the existing APIs be left in place?

[josepot]

On the flip side, do these have to be removed? Could the new stuff be added and the existing APIs be left in place?

While that is technically possible, I don't think that's a good idea. Because it wouldn't be possible to build the "standard" createSelector just using createSelectorCreator anymore and defaultMemoize would no longer be true to its name either.

Also, if in the future we wanted to further improve Reselect's API, that kind of enhancer would be a limiting factor... Lastly, it would make the composition of the selectors created with a custom createSelectorCreator with the "standard" selector-creator of Reselect quite challenging.

I mean, if what you are suggesting is to create an intermediate minor version update on version 4 where we still keep those ones, before v5 is released... I honestly think that would be unnecessary, but I guess that would be reasonable.

However, if we go for a major version update, then I think that the right thing to do would be to hide those 2 from the public API.

(I realize that I keep using the first person plural and I'm not part of this org, I hope that you don't find that annoying 😬)

[josepot]

Not to mention the fact that the current implementation of Reselect is basically those 2... Since the new API can't be implemented using those 2 functions, that would mean approximately doubling the parsing size of Reselect. So, having a tone of unused code.

But I would like to insist on the importance of the main reason, which is that perf optimizations are implementational details, and they should be hidden from the API. We should not expose a defaultMemoize function or its enhancer, because the way that Reselect chooses to memoize things should be hidden from the API (it's an implementational detail).

[mpenney99]

Just to offer another point of view to the discussion!

For me, one of the best thing about reselect is its simplicity. I think this proposition is a nice idea, but maybe it belongs more as a separate library.

That said though, I agree that caching by key is a really common problem that could be solved in reselect itself. As an alternative, how about something along the lines of:

const getWidget = createKeyedSelector((state, id) => id, (id) => {
		return createSelector(getDashboard, (d) => d.widgets[id2]);
	));

getWidget(state, id);

multiple keys:

 const getWidget = createKeyedSelector(
		(s, { id1 }) => id1,
		(s, { id2 }) => id2,
		(id1, id2) => {
			return createSelector((s) => s.dashboards[id1], (d) => d.widgets[id2]);
		));
	
getWidget(state, { id1, id2 });