Grid resize is slow

Alex.Zaikin

This is 5000 rows X 20 columns.
I already using minified css files. But it is taking around 5 sec to resize.
Whole web app is unresponsive during these 5 seconds of resizing.

Is there a way to show loading overlay onBeforeResize and hide it in OnResize event.
Any suggestions how to keep browser more responsive during the resizing time?

Also browser is freezing during Redraw time. Any help with that?

colin

This section of the FAQ should help, it discusses various techniques to improve performance,

Cheers,

Colin

Alex.Zaikin

Thanks Colin!
That's all we make to improve the performance.
We cannot do defer rendering though because scrolling speed is very important for us.
My question is more how to show loading overlay during grid resizing.
Because there is only one event OnResize which is fired when grid already resized. But we really need some hook to show overlay BEFORE resizing starts...

allan

Hi Alex,

Could you give us a few more details about the table please? For example, are you using scrollX or scrollY - those dramatically slow the drawing down due to the calculations that are required to keep the header and body columns in alignment. I'm going to guess that you do have at least one of them enabled and a lot of columns if it is taking as long as 5 seconds to calculate (wow!).

This is the point in the code that is calling the resize calculate function. It isn't really async enough to display a "Busy" message is part of the problem here - the browser will queue redraws, so even if we did show one, it wouldn't actually render until the calculation is done and the table reflowed anyway...

So our best option I think is to look at how we can speed up the redraw. As I say, if you are able to give me some more information about the table - the init object and information about the rows and columns, that would be great.

Allan

Alex.Zaikin

this.dtOptions = {
      data: this.inventories,      
      pageLength: 5000,
      deferRender: false,
      scroller: false,
      scrollY:'70vh',
      scrollX: false,
      scrollCollapse: true,
      rowId: 'ticketGroupID',
      order: [[1, 'asc']],
      paging: false,
      info: false,
      searching: false,
      processing: true,
      responsive: false,
      ordering: false,
      language: {'emptyTable': 'No results found' },     
      fixedColumns: false,          
      select: {
            style:'os'
      },
      rowCallback: ( row, data, index ) => {
        // workaround for empty/corrupted rows
        if (data.prodID === -1) {
            $(row).hide();
        }       

      },
      rowGroup: {

            startRender: ( rows, group)=> {
              const dt = rows.data()[0];
              if(dt.prodID < 0) return;

              const groupName = 'group-' + group;
              const rowNodes = rows.nodes();
              rowNodes.to$().addClass(groupName);             
               
              return `<div class="groupingTable">
                        <div style='width:50px;text-align: left;padding-left:18px;margin: 2px 0px 2px 0px'><input type="checkbox" class="group-checkbox" data-group-name="${groupName}"></div>
                        <div class="groupingCell" style='width:350px'>${dt.primaryEventName} ${dt.secondaryEventName ? '/' + dt.secondaryEventName: ''}</div>
                        <div class="groupingCell" style='width:${this.maxVenueNameMaxWidth}'>${dt.venueName}</div>
                        <div class="groupingCell">${this.getFormattedDate(dt.eventDateTime)}</div>                        
                      </div>`;


            },
            dataSrc: 'prodID'
      },
      keys:{
        columns: ':not(:first-child)',
        editor: this._editor,
        
      },
      columnDefs: [ 
          {
            orderable: false,
            width: '10px',
            checkboxes: {
              selectRow: true
            },
            targets:0
          },
          {
            targets: [ 1,2,3,4,5 ],
            visible: false
          },
           {
            targets: [ 6 ],
            width: '5px',
            orderable: false,
            'createdCell':  (td, cellData, rowData: TicketGroupWithEvent, row, col) => {
                $(td).html(this.getBroadcastComCompiled(rowData));
                $(td).mouseover((event) =>{
                    this.toggleBroadcastTooltip(rowData.ticketGroupID, true);
                });
                $(td).mouseout((event) =>{
                    this.toggleBroadcastTooltip(rowData.ticketGroupID, false);
                });
                $(td).on('click', (event) =>{
                    this.showBroadcastPopup(rowData.ticketGroupID);
                });
            }
           },
          {
            targets: [ 7 ],
            width: '5px',
            'createdCell':  (td, cellData, rowData, row, col) => {
                const deliveryType: DeliveryType = cellData;
                $(td).html(this.getDeliveryIconComCompiled(cellData));
            }
          },
          {
            targets: [ 8 ],
            width: '5px',
            maxwidth: '10px',
            'createdCell':  (td, cellData, rowData, row, col) => {
                const statusType = cellData;
                $(td).html(`<div class="icon">
                                 ${this.getStatusIconComCompiled(cellData)}
                            </div>`
                          );

            }
          },
          {
            targets: [9],
            width: '10px',
            orderable: false,
            autoWidth:false,
            'createdCell':  (td, cellData, rowData: TicketGroupWithEvent, row, col) => {                
                $(td).html(this.getPDF_BCIconComCompiled(rowData, true));
            }
          },
          {
            targets: [10],
            width: '10px',
            'createdCell':  (td, cellData, rowData, row, col) => {
                $(td).html(this.getPDF_BCIconComCompiled(rowData, false));
            }
          },
          {
            targets: [ 11 ],
            width: '15px',
            autoWidth:true
          },          
          {
            targets: [12,13 ],
            width: '10px'
          },
          {
            targets: [ 14 ],
            width: '25px'
          },
          {
            targets: [15,16],
            width: '10px',
            'createdCell':  (td, cellData, rowData, row, col) => {
                $(td).html(this.getCheckboxIconComCompiled(cellData));
            }
          },
          {
            targets: [17],
            width: '15px',            
            'createdCell':  (td, cellData, rowData:TicketGroupWithEvent, row, col) => {                
                if(cellData != null){
                  $(td).html(`<div>` + this.currencyPipe.transform(cellData) + `</div>`);
                }
                // In Case TicketGroup is Cancelled or Expired a respective cell is NOT editable
                if(rowData.statusTypeId === TicketGroupStatus.Available){
                  $(td).addClass('editable');      
                  $(td).attr('id','priceCell');                                
                }
              
            }          
          },
          {
            targets: [18,19],
            width: '15px',
            'createdCell':  (td, cellData, rowData, row, col) => {
                $(td).html(this.currencyPipe.transform(cellData));
            }          
          },
           {
            targets: [20],            
            'createdCell':  (td, cellData, rowData, row, col) => {
                 $(td).addClass('lastCell100'); 
            }          
          }
          ],
         
      
      columns: [
      { 
        data: 'prodID',
        defaultContent:'',
        title: '',
        searchable: false,
        orderable: false,

      },
      {
         title: 'Prod ID',
         data: 'prodID'
      },
       {
        title: 'Team/Performer',
        data: 'primaryEventName'
      }, {
        title: 'Opponent/Performer 2',
        data: 'secondaryEventName'       
      }, {
        title: 'Venue',
        data: 'venueName'
      },
       {
        title: 'Event Date',
        data: 'eventDateTime'
      },

      { data: 'userShNetworks', title: 'Exc' },
      { data: 'deliveryTypeId', title: '' },
      { data: 'status', title: '' },
      { data: 'isPdf', title: 'PDF' },
      { data: 'isBC', title: 'BC' },
      { data: 'section', title: 'Section' },
      { data: 'row', title: 'Row' },
      { data: 'quantity', title: 'Quantity' },
      { data: 'seats', title: 'Seats'},     
      { data: 'maskSeats', title: 'MS' },
      { data: 'isConsignment', title: 'CN' },
      { data: 'price', title: 'Market' },
      { data: 'cost', title: 'Cost' },
      { data: 'face', title: 'Face' },
      {data: null,
        defaultContent: '',
        title: '',
        searchable: false,
        orderable: false}
      ],

Alex.Zaikin

This is our table definition with columns

Alex.Zaikin

So how to make vertical scrolling without scrollY?

allan

Hi Alex,

Perfect - thank you. This:

pageLength: 5000,
paging: false

combined with the scrolling is what is killing the performance (note that pageLength is redundant if you are disabling paging).

In order to draw the table it needs to layout the entire table, take the column widths from the table body and then apply them to the header to align the columns. As an experiment, try setting the pageLength to be 10 and paging to be true - I expect the performance to be significantly better. Does it?

So how to make vertical scrolling without scrollY?

If you want the table in a container, then scrollY is how it is done - but what you could do is try using Scroller. That makes use of paging in the background, but visually the user sees a full scrolling grid.

If you could use it without a scrolling container - i.e. just let the page scroll, then you could use FixedHeader to give a somewhat similar effect.

Regards,
Allan

Alex.Zaikin

Thanks for the reply!
We tried Scroller - the performance when scrolling is degraded.
So the scrolling performance is most important. That's why we cannot use virtual rendering.

Alex.Zaikin

Just tried once again with a Scroller.
When i drug scroll bar down with mouse the grid is jumping and CPU is high...

My options like that:
paging: true,
pageLength: 10,
deferRender: false,
scroller: true,
scrollY:'70vh',
scrollX: false,
scrollCollapse: true,

Any chance to improve it?

allan

If there were a way to improve scroller performance that I was aware of, it would be implemented already .

deferRender should be enabled I would say - it will spread the load out.

How was the resize performance with Scroller enable?

Are you able to create a test case on http://live.datatables.net (or JSFiddle, or whatever) based on your configuration and load it up with faked data? That will let me look closely at where the time is being spent in processing things.

Interesting that you say performance is poor with Scroller - normally it really improves things!

Allan

Alex.Zaikin

Resizing in improving but the Scrollingf itself is very poor. If you drag the scrollerbar fast up and down...I could send you a video, so you better see what i mean.
Please let me know how to attach video file and send it to you.

Alex.Zaikin

IT is part of angular solution, so it will take time to create a test excerpt. I can send you the whole file with the grid and a logic around it. And JSON file with data. So you could recreate the test case by yourself.

Or i could send you Chrome Performance profile data, so you could see what caused the issue.
Please let me know what better work for you.

allan

Could you e-mail me the video please? That might give me a clue as to what we should be looking for.

Is Scroller slow for you in this example? I wonder if there is some interaction with Angular. I'm afraid I don't know Angular at all, so I wouldn't be able to recreate a test case with it.

Allan

Alex.Zaikin

https://drive.google.com/file/d/1cP0TY-N2fjiXVNZikUC5wYy8HmoeSBCx/view?usp=sharing

Alex.Zaikin

Please open this html in your browser and resize it. It will take 5-7 sec...
Any suggestions how to improve Style Recalc or shorten the styles?

Thanks!

Alex.Zaikin

https://drive.google.com/open?id=1EkVDct2g0_UIATDZ6ZapORhBW0gVIGr-

This is folder with following files:
1) IntialLoading Chrome Profiler;
2) Scrolling with Defer Rendering and Scrolling Extention: Video, Profiler;
3) Html full generated file to test Resizing issue and Chrome Profiler for 5K resizing;

Thanks for helping out!

allan

Hi Alex,

Thanks for the files! To confirm - the TVMockData.html file there is also showing performance problems for you? I've just tried it on my machine and it actually appears to work quite well. I'm not noticing any jumping around from it. That said - I don't think its actually running any code?

Could you send me scripts.js from your profile please? There is a function ma @ localhost:4666/scripts.js:2517 which I'm particularly interested in.

Thanks,
Allan

Alex.Zaikin

TVMockData.html is for showing resizing slowness. How many seconds it takes to resize on your machine?

Alex.Zaikin

OK I added scripts,js to the same Google Folder. Please download it.
BTW: in line 2517 there is a scrolling function from SpryMedia

allan

Super - thanks. Because it was a combined file with minimized code, there is no way without the file to know what the actual source is for the function without the file. Looking at it I can see that it is this function in DataTables core.

That is the function that is called to align the columns in the header and body tables when scrolling is enabled. The majority of the time is being taken by the get scroll top an get scroll position. Those are required to perform the column alignment calculations unfortunately - getting that information forces the browser to render the layout - which is always a really slow process (Javascript is fast on its own, but as soon as you mess around with the DOM, the performance nose dives).

I don't see that in the HTMl file as it looks like that is a "Save as..." file? So it isn't actually running any Javascript. It is just a snap shot and the scrolling is instant. None of the Javascript interactions on the page (filtering, paging, etc) are working.

The realignment of the columns drawing in the No Scroller JSON shows that it takes 450mS to complete. Interestingly after that there is then a AG Grid function which takes 650mS. Combine the two together and that's a long time. Are you able to disable the AG Grid one?

With the DraggingScroll JSON profile I see it is taking a bit over 4 seconds to complete the same function! 2 seconds of that is the calculation for the width offset, and 2 seconds is the calculation for the scroll position.

So that is where all the time is going and unfortunately there isn't much I can do about it - we need that information to be able to calculate the column widths / positions.

Two options to make this performant I think:

1. Don't use scrolling. Your performance issue would immediately disappear if you used paging instead.
2. Use table-layout: fixed in your CSS for the table, which would force the browser to use the columns widths you give for the header and footer, meaning DataTables doesn't have to perform its width calculations. We don't have any optimisation code in for that at the moment, but it would be possible to add. The big downside is that you have to specify width for every column in the table.

Allan

Alex.Zaikin

Thanks Allian!
The table around 2000 to 5000 records is the core requirement for this table.
So scrolling is a must, paging not suitable.

In that static html file - you are correct. This is a snapshot of our table. But what i want to know why resizing is slow in it? i(f you change size of a browser window) and than restore the browser is hanging for 5-7 seconds. For layout recalc. Is it possible to do something with it?

Alex.Zaikin

FYI: I tried table-layout: fixed. No any difference.
In which case should it help? Loading time, Resizing time or scrolling with Virtual Scroller?

allan

It is slow even without the Javascript simply due to the recalculation the browser has to do with the alignment of 80488 cells in the table body alone:

Because you are asking them all to be displayed at the same time, the browser needs to lay them all out. That is why paging would offer a massive performance improvement here, since then it is only layout out 10 (or so) at a time.

Scroller should help since it makes use of the paging built into DataTables. So if you must display this in a scrolling grid, rather than a paged one, then that is the way forward I would suggest.

table-layout: fixed isn't implemented in DataTables core as I noted above.

As an experiment to see if it helps performance in your table, you could modify the _fnScrollDraw function that is in DataTables (just your local copy of the code would do, to add the following at the top of it:

if (settings.__fixedLayout) {
  return;
}
var layout = $(settings.nTable).css('table-layout');
settings.__fixedLayout = layout === 'fixed';

if (settings.__fixedLayout) {
  return;
}

it isn't optimal of course, but it would stop the column alignment calculations being performed by DataTables.

Allan

Alex.Zaikin

https://drive.google.com/file/d/1lHKIdtmcHMA-JjaOl5kGaCX4uVs7DxRt/view?usp=sharing

Please take a look to the new profiler.
Still grid is freezing when loading next chunks. _fnScrollDraw is changed according to your suggestions. and CSS added for table-layout.

As you could se from the profiler style recalculation pikes are there to prevent table from smooth scrolling...

Any idea how to improve it?

allan

I'm getting:

Malformed timeline data: Unknown JSON format

From that one in Chrome I'm afraid.

However, if we are still stalling out in the browser's calculations and _fnScrollDraw is exiting almost immediately, then I think we're in the territory of hitting the browser's limits, since it is going to have to reflow the page.

Are you using Scroller with that profile? That should significantly reduce the number of cells on the page, thus making the reflow calculation faster in the browser.

Allan

Alex.Zaikin

Yes. I am sending you the new one.
Scroller + 5K rows + changed _fnScrollDraw and fixedLayout.
Most freezing in Style Recalc...

https://drive.google.com/file/d/1gyq81jfD2FHXigDlsd3IDzepXhIjAl13/view?usp=sharing

allan

Thanks you. Its showing ~ 5 seconds for the reflow, of which 4.3 seconds is the style calculation and reflow the browser is doing. It also indicates those values aren't being used by DataTables, since there is no additional processing on them, so those are just the limits that the browser is having based on the number of rows in the table.

Could you do:

$('#tableId tbody tr').length;

in your browser's console please (changing the tableId as required). Scroller should be having it show only 30-50 records and I'd be astonished if the browser is struggling so badly with that few records (unless you are using a Pentium 2 or something... ).

Allan

Alex.Zaikin

I have 215 in this $('#dataTable tbody tr').length;

Alex.Zaikin

I changed
scroller: {
displayBuffer: 20,
boundaryScale: 0.50
},

So now it shows around 50... Much better but not enough. Scrolling still not smooth, kind of jumpy. Attached the new Perf Profiler.

https://drive.google.com/file/d/1rHG6huEYmUuVrTdNNp5cyem_p4h0nPfV/view?usp=sharing

Alex.Zaikin

FYI After browser resizing and scrolling a little bit. Rows starting growing to 600. Don't know the reason. Maybe bug...
But profiler i sent you is for 45 Rows still