Making QGIS layer update from changed data source

Question

I'm trying to get layers to update automatically when their data source changes. I'm using R to write a shapefile with an attribute, and colouring according to that attribute in QGIS.

I want to write a new shapefile with different attribute values, and have the Qgis map colours update. Step 1 is triggering that process, step 2 is making the layer reload from the modified shapefile. Its step 2 I am worrying about here.

Other questions/mailing list chatter mentions using triggerRepaint on the layer - that doesn't work. Other suggestions include setCacheImage(None) and again that doesn't work. The layer does update eventually, but I really can't see the logic, and sometimes it happens by surprise after I've done nothing. Or maybe I did something two minutes ago.

The one reproducible way of getting it to update is to duplicate the layer from the legend menu - the duplicate always gets its data from the current shapefile, and the original layer updates itself too! So there must be some way of doing it.

I think it was working better in 2.8, but this is 2.10 so maybe there's a new bug somewhere.

Related, but doesn't work for me in 2.10:

How to automatically reload raster layers if source is changed in QGIS?

Other things I've tried:

• layer.dataProvider().dataChanged.emit() - worked once, then not again on the same layer

I think I've tracked down why duplicating the layer works - if I create a new throwaway layer based on the updated layer and then call .triggerRepaint() on the updated layer, it updates on the map canvas:

QgsVectorLayer( layer.source(), "layer copy", layer.providerType() )
layer.triggerRepaint()

If I use a different layer source it doesn't work, so it seems to be if you create a layer object based on the same layer source...

A quick test just now with a raster layer (from a GeoTIFF), and just calling rlayer.triggerRepaint() seems to reliably update the view of the raster in the map canvas.

Answer 1

This is related to the introduction of the OGR connection pool. [1]

Before QGIS 2.10, a file was reopened on every single access (e.g. repaint).

Since QGIS 2.10 the file handle is kept open and this means if a file is replaced the handle still points to the old file on Unix based systems.

QGIS 2.10: workaround

Unfortunately there is no API to nicely force QGIS to reopen the file in QGIS 2.10. As a workaround you can use an ugly hack:

layer.dataProvider().changeAttributeValues( { -1: { 0: 0 } } )
layer.triggerRepaint()

QGIS 2.12: solution

I just introduced a new method which will be available starting from QGIS 2.12:

layer.dataProvider().forceReload()
layer.triggerRepaint()

General approach

If you have the possibility to control how the file is being overwritten you can open the existing files with write permissions and change the content instead of replacing the files completely (delete/recreate) on disk.

[1] The connection pool was introduced to significantly speed up access to certain data sources.

Answer 2

If you pan or otherwise refresh the map it should update.

This article says that you can use the following in PyQGIS:

myLayer.triggerRepaint()

To refresh all layers following function can be used:

def refresh_layers(self):
    for layer in qgis.utils.iface.mapCanvas().layers():
         layer.triggerRepaint()

Answer 3

Reload currently selected layer

On current QGIS 3.16.16 I am able to successfully reload and redraw by selecting the raster layer, opening the Python console and using: iface.activeLayer().dataProvider().reloadData()

Reload all raster layers

To apply this to all raster layers you can instead use:

for layer in QgsProject.instance().mapLayers().values():
    if layer.type() == QgsMapLayerType.RasterLayer:
        layer.dataProvider().reloadData()

Auto-reload on file change

It is possible to automatically trigger this whenever the file changes. Adapting the suggestion at https://gis.stackexchange.com/a/58885/66716 to current QGIS 3.16.16, using Qt5 and the above methods and automatically taking the filename from the currently selected raster layer:

from PyQt5.QtCore import QFileSystemWatcher
rasterLayer = iface.activeLayer()
watcher = QFileSystemWatcher()
watcher.addPath(rasterLayer.dataProvider().dataSourceUri())
watcher.fileChanged.connect(rasterLayer.dataProvider().reloadData)

In my experience this filesystem watcher approach works well, with the QGIS display updating and providing immediate feedback as soon as my external application changes the file.

I was having no success with setCacheImage(None), triggerRepaint(), and other suggestions. QGIS seemed to only present fragments of the new data together with cached fragments of the previous version.

Extra technical background

There is a feature request for QGIS to add a context menu item to reload the data source, but no contributed implementation yet: https://github.com/qgis/QGIS/issues/31507

Here is an issue from late 2018 about redrawing raster layers after the source file changes: https://issues.qgis.org/issues/20536

It seems to describe the problems I was seeing, and links to a solution commit that adds a function QgsGdalProvider::reloadData(). These days (early 2023) it looks like this function is declared in src/core/providers/qgsdataprovider.h and implemented on each data provider type via functions like QgsGdalProvider::reloadProviderData().