Caching and anticipatory loading of scenes

[letsfindaway]

Loading a scene until it is visible on the screen can take a non-negligible amount of time. Limiting factors are

• Data transfer of large files, especially when they are located on a network drive.
• CPU usage, especially loading of scenes with many strokes (handwriting) or rendering of complex PDF documents.

To reduce the time necessary to switch between scenes it is useful

• to load scenes in advances if there is some probability that they might be selected next,
• and to cache already loaded scenes and PDF renderings.

This article analyzes the loading process, describes the current status, and investigates the options to create a smooth user experience.

Loading of scenes

An OpenBoard scene (or page) is stored on disk as follows:

• The main scene description is contained in an SVG file. The SVG represents the strokes and objects on the scene.
• Objects may reference other files, e.g. images, media content, PDF files or web widgets.
• There is also a small preview image of the scene, which is used to display a page preview in document mode.

When loading a scene, the SVG elements are processed one by one, and the corresponding graphical items on the scene are created. If necessary, referenced files are also accessed, even if they are often not completely loaded in this stage:

• If an image is loaded, then indeed the complete image must be read to create the graphical item.
• If audio or video content is referenced, that content is not loaded until the audio or video playback starts. This means it is not loaded during scene creation.
• If a PDF document is referenced, then only the header and catalog is loaded, which contains a list of all pages and their relative location within the document. The data is only read when the PDF page is rendered.
• If a web widget is loaded, then the corresponding web page and web view are created and the loading process is started, but continues in the background.

Background loading

To load a scene in the background without affecting user experience the tasks may be executed in another thread. There is however one limitation: Creation of the graphical items of the scene cannot be performed in another thread due to internal Qt restrictions (see this comment). We can however

• we can execute the reading of a file (SVG, PDF, image) in another thread, and
• create the graphical items one by one on the main thread when this thread is idle.

Caching

Creation of graphical items for a scene and rendering of PDF pages are the most time consuming tasks. Therefore OpenBoard has caching features for both of them.

• Scenes are cached in UBSceneCache. This cache covers scenes from several documents. It is managed by a LRU policy with a limited number of scenes.
• Rendered PDF pages are cached in XPDFRenderer. There is one renderer per PDF document. Here all rendered pages from that PDF document are cached, probably in several resolutions. The cache is only cleared if no more scene references this PDF document and the renderer is destroyed.

None of the caches check for a memory limit.

Current status

After merging the "lazy loading" PR, the situation is as follows:

• The SVG files are read on the main thread. Before merging this PR, this was done in a separate thread, but the returned data was dropped and read again in the main thread. Therefore this code was removed.
• The graphical items are created one by one in small steps on the main thread in the background.
• When creating an image item, the data is read on the main thread during such a step.
• The scene cache is limited to a number of scenes.
• The PDF render cache is not limited and lives as long as the document is referenced by any scene.
• The previous, next and next but one page is loaded in advance.

Room for improvement

I see the following places which could be improved:

• Read the SVG file in another thread (reactivate the previous code), but use the resulting data.
• Split the step of creating an image item into pieces:
  • Read the SVG element and locate the referenced image file (main thread).
  • Load the image data (separate thread).
  • Create the graphical image item and place it on the scene (main thread).
• Use a memory limit for both the scene cache and the PDF render cache, so that cached data is not dropped on a computer with reasonable amount of memory.
• Create a strategy for which pages to delete based on what users typically do.
• Create a strategy for which pages to load in advance based on what users typically do.
• Include the thumbnails in the caching strategy, so that they don't have to be recreated when switching documents.

Caching strategies (ideas)

Loading

The following non-exhaustive list of ideas might also be combined.

• Load neighbored pages.
• Load everything in the currently open document.
• Keep a persistent history of recently opened pages and load those, plus adjacent pages.
  • consider a page "opened" only if it is selected for at least a few seconds.
• Let the user preselect pages for loading in document mode.
• Let the user preselect documents for loading in document mode.
• Load pages whose thumbnails are currently visible in the thumbnail sidebar.
• Maintain a priority of pages to be loaded.
• When switching to a page, which is not already cached, display the thumbnail preview image while loading the actual content in the background. This avoids blocking of the UI when switching to a not loaded page.

Unloading

The only reason for unloading is memory. We should definitely avoid swapping.

• Use the ratio available / total as limit.
• Unload heaviest pages first.
• Unload least recently used pages first.
• Make some combination of these two by calculating a score based on age (how long not used) and size.
• When leaving a document, mark pages which have never been used and unload them first.
• When unloading a page, also clear the associated PDF rendering cache entry.
• Make some educated guess for memory usage.
  • For PDF renderings, use the QImage::sizeInBytes() to calculate memory usage.
  • For a graphics scene, use a heuristic formula based on the number of strokes and the size of images to calculate memory usage.
  • Update these numbers when leaving a page.

[letsfindaway]

Caching of documents

With PR Unified Thumbnails also caching of documents with their thumbnails is possible. This PR introduces a UBDocument class with an attached UBThumbnailScene, which contains the thumbnails for all pages of a document. Caching the instances of UBDocument enables fast switching between documents.

Loading of documents

A document is loaded when it is first accessed. There is no need for anticipatory loading, as the thumbnails are loaded in background and anticipatory loading will give no visual advantage.

Keep documents

Where and when does caching of document instances improve the user experience?

• Due to the background loading of thumbnails there is no big advantage in caching documents which have been selected in Document mode. When the user selects another document, then thumbnails are always shown starting with page 1. Loading of the first visible page of thumbnails is quite fast, perhaps except for extremely slow network drives.
• I can only imagine the situation that the user wants to jump to the last page of a large document immediately after selecting it. Then they would see the last loaded page instead of the actual last page of the document.
• For Board mode however I see an advantage for documents which have been opened in the current session and are now available as temporary favorites. Switching to another document by dragging a favorite to the board also selects the last visited page for that document. Here it is an advantage if the thumbnail for all pages are cached and immediately available.

Unloading documents

For unloading of documents we should also consider a strategy. My current ideas:

• Include the memory consumption of a document into the caching strategy for scenes and unload a document if it is not currently used and it is necessary to free memory.
• Keep document instances for all documents you have opened in Board mode, but drop them for those documents you have just examined in Document mode.

The second point seems to be an easy-to-implement strategy which does not negatively affect user experience. This is also what is implemented in the merged PR.