Selecting row ranges with Shift+Click slow with large tables (select style "os" and "multi+shift")

PajamaJam

I've noticed that selecting a range of rows using shift+click (when select style is set to "os" or "multi+shift") is very slow when working with a large table (say, ~50k rows). This is true even when using the deferRender option.

I understand that ~50k rows is a large table to render entirely on the client and I really should use serverSide rendering. That's definitely on the to-do list, but for now I'm working with an existing implementation and wondering if I can make shift+click row selections less laggy on large tables since they freeze the UI until the datatable selection code has run its course.

Is the select state of all rows calculated on each select event? This is what appears to be happening. My guess would have been that meta data for rows/cells/columns/etc. is stored in memory which would make getting/setting something like a selected:true bit a non-issue.

As always, thanks in advance for any help/insights offered.

allan

Server-side processing and row selection brings its own problems. You can't select rows on the client-side that don't exist there (i.e. you can only select the rows on the current page).

Can you link to the page showing the issue so I can profile it and see if I can make any performance improvements please?

Allan

PajamaJam

Hi again, Allan!

Here is a test page that demonstrates the issue:

http://live.datatables.net/yunitufo/2/edit?js,output

After digging a little deeper, it appears that performance is an issue when selecting a large number of rows regardless of the method (not just via shift+click as I originally mentioned). I've provided "Select All" and "Deselect All" buttons to demonstrate the issue.

As long as I have your eyes on this, any insights on how to best implement similar functionality when using serverSide:true? I'm concerned that, given the performance of "select all" and select-via-shift+click when everything is on the client, that we'll still have issues with this even after moving to server-side rendering.

Thanks!

allan

As long as I have your eyes on this, any insights on how to best implement similar functionality when using serverSide:true?

You'd need to implement some kind of session controller on the server-side that would keep track of what rows have been selected on the client-side, and then keep them in-sync. That's not something that I've tried yet myself.

Here is a test page that demonstrates the issue:
http://live.datatables.net/yunitufo/2/edit?js,output

Excellent - thank you! A profile shows that almost all of the time there is spent in the _unique function. If that can be optimised or potentially the need for it removed, then it would run a heck of a lot faster.

I will look into that.

Allan

PajamaJam

Interesting. Why would the row selection API require calling a _unique() method? I ask only to better understand so I could modify the source if necessary in the short-term (something I generally try to avoid). I don't want to solve one problem and introduce another.

Thank you again for your help, Allan. Very much appreciated as always!

allan

Selecting rows doesn't, however using rows() (which itself is a selector, just of a different type) does use it to make sure that any given row selector doesn't pick out the same row multiple times (consider for example: table.row( ':eq(0), :eq(0)' ) - you wouldn't do that obviously, but more complex selectors might result in something similar). As a sanity check it makes sure that each item is present only once.

This is where it is run - in the code common to all selectors (rows(), columns(), cells() etc).

I guess you could try modifying the code to remove that and just return out. That will let it run much faster.

Allan

PajamaJam

Good stuff, Allan. As always.

If I understand things correctly, the _unique() method is being called automatically each time the rows() method is called to prevent issues for developers using row selectors that *might* match one-or-more row(s) multiple times, yes? If this is the case, then it seems like a performance hit is incurred for all scenarios in order to avoid errors in a (relatively) unlikely scenario. Not the worst thing, but couldn't the "unlikely" scenario be handled by the developer using the unique() utility api call?

var rows = table.rows(':eq(0), :eq(0)').unique();

I'll make the change to the source code and see if that can't get me by or the short-term. The part I dread about doing so is that it's difficult manage these custom patches and keep up with library updates.

Thanks again Allan. Really appreciate the feedback.

allan

If I understand things correctly, the _unique() method is being called automatically each time the rows() method is called to prevent issues for developers using row selectors that *might* match one-or-more row(s) multiple times, yes?

Exactly that.

a (relatively) unlikely scenario.

You'd be surprised ;-)

However, I do agree. That is something that I think code and should be optimised. Interested to hear how you get on with it when you change the code to remove that line.

Allan

allan

A little update on this - I've just committed a change which significantly improves the _unique() method by checking if it is needed before running it. If not, then the performance improvement is massive (going from almost 1S to ~20mS on my test with 50k rows).

This will be in DataTables 1.10.14 which will ship next week and the nightly will rebuild with the change shortly.

Allan

BrianAtCuahsi

Friends,

Thanks for a terrific job on DataTables!! A quick question re: selecting rows in tables with many entries, if I may.

The _unique() method fix described in this forum topic is said to be available in DataTables v1.10.14.

I am running (minimized) DataTables v1.10.15 with Scroller v1.4.2 and Select v1.2.2. Unfortunately, I experience noticeably slow row selection in tables with c. 10K -15K entries.

How can I verify I have the _unique() method fix?

Best,

Brian

allan

I am running (minimized) DataTables v1.10.15

You do have the _unique() fix if you are using 1.10.15.

Perhaps there is something else going on, or a selector is being used which is not allowing DataTables to fall into the special case allowing it to bypass _unique. I'd need a test case so say either way.

Allan