[data grid] Improve Server-side support with new data source

[cherniavskii]

Summary 💡

The server-side data source will be implemented in a step-by-step manner, the issues it will be covered in are mentioned in the list below.

• A TypeScript interface with clear requirements of what the user has to provide for the data grid to have the necessary data. (Handled in [DataGridPro] Add data source interface and basic documentation #10543)
  • Start with a Collection interface where you provide the interface for the data collection that could be used by Toolpad in their server-side function. The goal is to have the configuration of the data source in a single place where the users can translate the pagination, filter, sorting models, etc. to whatever you have on the server (HTTP request, DB query, or something else)
  • It should handle the fetching first, and we can extend the interface with updating functions later.
• [data grid] Implement server-side data source in tree data #3377
• [data grid] Implement server-side data source in row grouping #10859
• [data grid] Implement server-side data source in row lazy loading #10857
• [data grid] Implement server-side data source in row infinite loading #10858
• [data grid] Implement server-side data source with aggregation #10860
• [data grid] Support edit functionality with server-side data source #13261
• [data grid] Implement server-side data source with nested data lazy loading #14527
• [data grid] Implement server-side data source with pivoting #10861

Motivation 🔦

To make the integration of the data grid with users' data easier. Today we have https://mui.com/x/react-data-grid/row-updates/#lazy-loading but it could be a lot better.

Examples 🌈

• https://marmelab.com/react-admin/DataProviders.html
• @Janpot has an idea for the createCollection() function in the Toolpad

Shaping page

https://www.notion.so/mui-org/Data-source-proposal-db58b8b749154420a407f231c5528c1d

User requests

• https://mui.zendesk.com/agent/tickets/15724

[cherniavskii]

cc @mui/toolpad

[cherniavskii]

We have recently explored supporting lazy loading for tree data and row grouping, and this seems to be extending the original description of the data source.

With the data source, we want to support at least these use cases:

• Row grouping: lazy-load children rows.
  Currently, row grouping only works on the client side. This means that all the rows have to be fetched in advance. This is not scalable for large datasets.
• Tree data: lazy-load children rows.
  Same as with row grouping, tree data only works on the client side. We have a demo allowing lazy load children rows, but it's not a part of the core package. We need to support this natively.
• Hooking up server-side pagination, sorting, and filtering
  While it's possible to use server-side pagination, sorting, and filtering, each of these features has to be handled separately. We need to provide a way to hook up all of these features at once. It will also be beneficial for us if we have a centralized way of fetching data from the server (when not using client-side features).
  This effectively replaces paginationMode="server", sortingMode="server" and filterMode="server".

It’s preferable for the data source to be optional in the use cases that are currently supported by the data grid (like server filtering, sorting, pagination, etc.).

Benchmarks:

• https://www.ag-grid.com/javascript-data-grid/server-side-model/
• https://js.devexpress.com/Documentation/ApiReference/Data_Layer/CustomStore/
• https://www.telerik.com/kendo-react-ui/components/grid/data-operations/odata-server-operations/
• https://blazor.syncfusion.com/demos/datagrid/remote-data?theme=bootstrap4
• https://www.infragistics.com/products/ignite-ui-angular/angular/components/grid/remote-data-operations

[Janpot]

Just for cross-reference: We now have an implementation in Toolpad which supports pagination, filtering, sorting and all CRUD operations.