Big list performance with React

Question

I am in the process of implementing a filterable list with React. The structure of the list is as shown in the image below.

PREMISE

Answer 1

Like I mentioned in my comment, I doubt that users need all those 10000 results in the browser at once.

What if you page through the results, and always just show a list of 10 results.

I've created an example using this technique, without using any other library like Redux. Currently only with keyboard navigation, but could easily be extended to work on scrolling as well.

The example exists of 3 components, the container application, a search component and a list component. Almost all the logic has been moved to the container component.

The gist lies in keeping track of the start and the selected result, and shifting those on keyboard interaction.

nextResult: function() {
  var selected = this.state.selected + 1
  var start = this.state.start
  if(selected >= start + this.props.limit) {
    ++start
  }
  if(selected + start < this.state.results.length) {
    this.setState({selected: selected, start: start})
  }
},

prevResult: function() {
  var selected = this.state.selected - 1
  var start = this.state.start
  if(selected < start) {
    --start
  }
  if(selected + start >= 0) {
    this.setState({selected: selected, start: start})
  }
},

While simply passing all the files through a filter:

updateResults: function() {
  var results = this.props.files.filter(function(file){
    return file.file.indexOf(this.state.query) > -1
  }, this)

  this.setState({
    results: results
  });
},

And slicing the results based on start and limit in the render method:

render: function() {
  var files = this.state.results.slice(this.state.start, this.state.start + this.props.limit)
  return (
    <div>
      <Search onSearch={this.onSearch} onKeyDown={this.onKeyDown} />
      <List files={files} selected={this.state.selected - this.state.start} />
    </div>
  )
}

Fiddle containing a full working example: https://jsfiddle.net/koenpunt/hm1xnpqk/

Answer 2

My experience with a very similar problem is that react really suffers if there are more than 100-200 or so components in the DOM at once. Even if you are super careful (by setting up all your keys and/or implementing a shouldComponentUpdate method) to only to change one or two components on a re-render, you're still going to be in a world of hurt.

The slow part of react at the moment is when it compares the difference between the virtual DOM and the real DOM. If you have thousands of components but only update a couple, it doesn't matter, react still has a massive difference operation to do between the DOMs.

When I write pages now I try to design them to minimise the number of components, one way to do this when rendering large lists of components is to... well... not render large lists of components.

What I mean is: only render the components you can currently see, render more as you scroll down, you're user isn't likely to scroll down through thousands of components any way.... I hope.

A great library for doing this is:

https://www.npmjs.com/package/react-infinite-scroll

With a great how-to here:

http://www.reactexamples.com/react-infinite-scroll/

I'm afraid it doesn't remove components that are off the top of the page though, so if you scroll for long enough you're performance issues will start to reemerge.

I know it isn't good practice to provide a link as answer, but the examples they provide are going to explain how to use this library much better than I can here. Hopefully I have explained why big lists are bad, but also a work around.

Answer 3

Try filter before loading into the React component and only show a reasonable amount of items in the component and load more on demand. Nobody can view that many items at one time.

I don't think you are, but don't use indexes as keys.

To find out the real reason why the development and production versions are different you could try profiling your code.

Load your page, start recording, perform a change, stop recording and then check out the timings. See here for instructions for performance profiling in Chrome.

Answer 4

Check out React Virtualized Select, it's designed to address this issue and performs impressively in my experience. From the description:

HOC that uses react-virtualized and react-select to display large lists of options in a drop-down

https://github.com/bvaughn/react-virtualized-select

Answer 5

For anyone struggling with this problem I have written a component react-big-list that handles lists to up to 1 million of records.

On top of that it comes with some fancy extra features like:

• Sorting
• Caching
• Custom filtering
• ...

We are using it in production in quite some apps and it works great.

Answer 6

First of all, the difference between the development and production version of React is huge because in production there are many bypassed sanity checks (such as prop types verification).

Then, I think you should reconsider using Redux because it would be extremely helpful here for what you need (or any kind of flux implementation). You should definitively take a look at this presentation : Big List High Performance React & Redux.

But before diving into redux, you need to made some ajustements to your React code by splitting your components into smaller components because shouldComponentUpdate will totally bypass the rendering of children, so it's a huge gain.

When you have more granular components, you can handle the state with redux and react-redux to better organize the data flow.

I was recently facing a similar issue when I needed to render one thousand rows and be able to modify each row by editing its content. This mini app displays a list of concerts with potential duplicates concerts and I need to chose for each potential duplicate if I want to mark the potential duplicate as an original concert (not a duplicate) by checking the checkbox, and, if necessary, edit the name of the concert. If I do nothing for a particular potential duplicate item, it will be considered duplicate and will be deleted.

Here is what it looks like :

There are basically 4 mains components (there is only one row here but it's for the sake of the example) :

Here is the full code (working CodePen : Huge List with React & Redux) using redux, react-redux, immutable, reselect and recompose:

const initialState = Immutable.fromJS({ /* See codepen, this is a HUGE list */ })

const types = {
    CONCERTS_DEDUP_NAME_CHANGED: 'diggger/concertsDeduplication/CONCERTS_DEDUP_NAME_CHANGED',
    CONCERTS_DEDUP_CONCERT_TOGGLED: 'diggger/concertsDeduplication/CONCERTS_DEDUP_CONCERT_TOGGLED',
};

const changeName = (pk, name) => ({
    type: types.CONCERTS_DEDUP_NAME_CHANGED,
    pk,
    name
});

const toggleConcert = (pk, toggled) => ({
    type: types.CONCERTS_DEDUP_CONCERT_TOGGLED,
    pk,
    toggled
});


const reducer = (state = initialState, action = {}) => {
    switch (action.type) {
        case types.CONCERTS_DEDUP_NAME_CHANGED:
            return state
                .updateIn(['names', String(action.pk)], () => action.name)
                .set('_state', 'not_saved');
        case types.CONCERTS_DEDUP_CONCERT_TOGGLED:
            return state
                .updateIn(['concerts', String(action.pk)], () => action.toggled)
                .set('_state', 'not_saved');
        default:
            return state;
    }
};

/* configureStore */
const store = Redux.createStore(
    reducer,
    initialState
);

/* SELECTORS */

const getDuplicatesGroups = (state) => state.get('duplicatesGroups');

const getDuplicateGroup = (state, name) => state.getIn(['duplicatesGroups', name]);

const getConcerts = (state) => state.get('concerts');

const getNames = (state) => state.get('names');

const getConcertName = (state, pk) => getNames(state).get(String(pk));

const isConcertOriginal = (state, pk) => getConcerts(state).get(String(pk));

const getGroupNames = reselect.createSelector(
    getDuplicatesGroups,
    (duplicates) => duplicates.flip().toList()
);

const makeGetConcertName = () => reselect.createSelector(
    getConcertName,
    (name) => name
);

const makeIsConcertOriginal = () => reselect.createSelector(
    isConcertOriginal,
    (original) => original
);

const makeGetDuplicateGroup = () => reselect.createSelector(
    getDuplicateGroup,
    (duplicates) => duplicates
);



/* COMPONENTS */

const DuplicatessTableRow = Recompose.onlyUpdateForKeys(['name'])(({ name }) => {
    return (
        <tr>
            <td>{name}</td>
            <DuplicatesRowColumn name={name}/>
        </tr>
    )
});

const PureToggle = Recompose.onlyUpdateForKeys(['toggled'])(({ toggled, ...otherProps }) => (
    <input type="checkbox" defaultChecked={toggled} {...otherProps}/>
));


/* CONTAINERS */

let DuplicatesTable = ({ groups }) => {

    return (
        <div>
            <table className="pure-table pure-table-bordered">
                <thead>
                    <tr>
                        <th>{'Concert'}</th>
                        <th>{'Duplicates'}</th>
                    </tr>
                </thead>
                <tbody>
                    {groups.map(name => (
                        <DuplicatesTableRow key={name} name={name} />
                    ))}
                </tbody>
            </table>
        </div>
    )

};

DuplicatesTable.propTypes = {
    groups: React.PropTypes.instanceOf(Immutable.List),
};

DuplicatesTable = ReactRedux.connect(
    (state) => ({
        groups: getGroupNames(state),
    })
)(DuplicatesTable);


let DuplicatesRowColumn = ({ duplicates }) => (
    <td>
        <ul>
            {duplicates.map(d => (
                <DuplicateItem
                    key={d}
                    pk={d}/>
            ))}
        </ul>
    </td>
);

DuplicatessRowColumn.propTypes = {
    duplicates: React.PropTypes.arrayOf(
        React.PropTypes.string
    )
};

const makeMapStateToProps1 = (_, { name }) => {
    const getDuplicateGroup = makeGetDuplicateGroup();
    return (state) => ({
        duplicates: getDuplicateGroup(state, name)
    });
};

DuplicatesRowColumn = ReactRedux.connect(makeMapStateToProps1)(DuplicatesRowColumn);


let DuplicateItem = ({ pk, name, toggled, onToggle, onNameChange }) => {
    return (
        <li>
            <table>
                <tbody>
                    <tr>
                        <td>{ toggled ? <input type="text" value={name} onChange={(e) => onNameChange(pk, e.target.value)}/> : name }</td>
                        <td>
                            <PureToggle toggled={toggled} onChange={(e) => onToggle(pk, e.target.checked)}/>
                        </td>
                    </tr>
                </tbody>
            </table>
        </li>
    )
}

const makeMapStateToProps2 = (_, { pk }) => {
    const getConcertName = makeGetConcertName();
    const isConcertOriginal = makeIsConcertOriginal();

    return (state) => ({
        name: getConcertName(state, pk),
        toggled: isConcertOriginal(state, pk)
    });
};

DuplicateItem = ReactRedux.connect(
    makeMapStateToProps2,
    (dispatch) => ({
        onNameChange(pk, name) {
            dispatch(changeName(pk, name));
        },
        onToggle(pk, toggled) {
            dispatch(toggleConcert(pk, toggled));
        }
    })
)(DuplicateItem);


const App = () => (
    <div style={{ maxWidth: '1200px', margin: 'auto' }}>
        <DuplicatesTable />
    </div>
)

ReactDOM.render(
    <ReactRedux.Provider store={store}>
        <App/>
    </ReactRedux.Provider>,
    document.getElementById('app')
);

Lessons learned by doing this mini app when working with huge dataset

• React components work best when they are kept small
• Reselect become very useful to avoid recomputation and keep the same reference object (when using immutable.js) given the same arguments.
• Create connected component for component that are the closest of the data they need to avoid having component only passing down props that they do not use
• Usage of fabric function to create mapDispatchToProps when you need only the initial prop given in ownProps is necessary to avoid useless re-rendering
• React & redux definitively rock together !