Multiple threads and data sharing between them

Hello all: Thanks to Aquiles book and examples I’ve got a Python Qt5 app running, but there are problems. I have a class MainWindow and a class Threads. I know how to link them, but I want a thread to acquire data and “update” data in the MainWindow. I’ve looked at shared memory, but that’s not clear. I’ve also looked into QMutex. All of this seems a bit too complicated. I just want/need one thread to acquire new data, and another (the main thread with the GUI window) to be ‘updated’.

I can attach code (I think) if it will help, but the basic idea is hopefully here.
Thanks!
Dave

Hello Dave!
Welcome to the forum!

The idea of shared memory is that you can directly use the attributes of one object in another object. Let’s say that you have one class that acquires data from a device, something like this:

class Device:
    def scan(self):
        self.data = []
        for i in self.scan_range:
            self.driver.set_data(i)
            self.data.append(self.driver.read())

The code above is obviously not going to work, but it is just to prove the point. Now, let’s see what we can do in the main window:

class MainWindow(QMainWindow):
    def __init__(self, device, parent=None):
        super().__init__(parent=parent)
        self.device = device
        [...]
        self.thread = None
        self.timer = QTimer()
        self.timer.timeout.connect(self.update_plot)

    def make_scan(self):
        self.thread = Thread(target=self.device.scan)
        self.thread.start()
        self.timer.start(200) # Time here is in milliseconds

    def update_plot(self):
        self.plot.setData(self.device.data)

Let’s go step by step with the example above (which, again is not complete, but I hope you see the point). We have a window, and we have a self.device attribute define. This attribute is an instance of Device we defined at the beginning. We define self.thread and, more importantly, we define a QTimer.

Now, when we want to start a measurement, we have to run make_scan (you can connect the pressing of a button to it, for example). This method starts a new thread for the scan method of the device, and it also starts the timer with a 200 millisecond timeout. This means that every 200 milliseconds the method update_plot will be executed. Which brings us to the last portion of the code.

update_plot only does one thing: updates the plot. And to do that, it uses self.device.data. Now, bear in mind that data is changing while you acquire, and this is happening on a different thread. Using shared memory means that you don’t need to do anything extra to use that information, it is just available.

Hope the examples above point you in the right direction!

And also, consider that the same behavior can be achieved in different ways. For example, we defined the thread in the window because it is a quick solution. However, the thread could be defined in the Device. This has the advantage that you can use the same solution regardless of whether you are using this window, the command line, another user interface, etc.

There is also the possibility of using QThreads instead of Python threads. If you read Qt tutorials, some developers advise not mixing them. However, the documentation is very confusing regarding what is the proper way of defining new threads in Qt (not only in Python, but also the official docs). Therefore, I would not spend too much time trying to use them, just stick to the Python ones. QThreads have some advantages for bigger programs, but almost nothing that you can’t achieve with plain Python.

Thanks for the suggestions. I’ve changed my code to try to reflect it here:

from PyQt5 import QtGui, QtCore, QtWidgets, uic
from PyQt5.QtWidgets import *
import pyqtgraph as pg
import numpy as np
import threading
import readSoc
import sys

class MainWindow(QMainWindow):
    def __init__(self, parent=None):
        super(MainWindow, self).__init__(parent=parent)
        uic.loadUi('statusCAC.ui', self)
        self.thread = None       
        self.main_plot = pg.PlotWidget()
        layout = self.centralwidget.layout()
        layout.addWidget(self.main_plot)
        self.device = Device()
        self.timer = QtCore.QTimer()
        self.timer.timeout.connect(self.device.scan)
        self.startButton.clicked.connect(self.Scan)
    def Scan(self):
        print('Scan started...')
        self.thread = threading.Thread(target=self.Plot)
        self.thread.start()
        self.timer.start(500)
    def Plot(self):
        print('Update Plot...')
        self.main_plot.plotItem.plot(self.device.I)

class Device:
    def __init__(self):
        self.server = readSoc.ReadSoc('10.200.111.77', 19876)
        self.nBytes = 8192 * 4
        self.I, self.Q = np.zeros((0)), np.zeros((0))
    def scan(self):
        self.I, self.Q = self.server.ReadNext(self.nBytes, 'IQ')
        

 
if __name__ == '__main__':
    app = QtGui.QApplication(sys.argv)
    w = MainWindow()
    w.show()
    sys.exit(app.exec_())

However, I get this message (no matter what I do to try to fix it):

waiting for a connection <_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>
Scan started...
Update Plot...
QObject::startTimer: Timers cannot be started from another thread
QObject::startTimer: Timers cannot be started from another thread
QObject::startTimer: Timers cannot be started from another thread
QObject::startTimer: Timers cannot be started from another thread
QObject::startTimer: Timers cannot be started from another thread
QObject::startTimer: Timers cannot be started from another thread

I’d really love to get rid of the timer ultimately and just have the scan function ‘emit’ a signal that it’s done and have it continue. But I can’t get that to work either…

Best,
Dave

I didn’t use the formatting feature (sorry):
indent preformatted text by 4 spaces

from PyQt5 import QtGui, QtCore, QtWidgets, uic
from PyQt5.QtWidgets import *
import pyqtgraph as pg
import numpy as np
import threading
import readSoc
import sys

class MainWindow(QMainWindow):
    def __init__(self, parent=None):
        super(MainWindow, self).__init__(parent=parent)
        uic.loadUi('statusCAC.ui', self)
        self.thread = None       
        self.main_plot = pg.PlotWidget()
        layout = self.centralwidget.layout()
        layout.addWidget(self.main_plot)
        self.device = Device()
        self.timer = QtCore.QTimer()
        self.timer.timeout.connect(self.device.scan)
        self.startButton.clicked.connect(self.Scan)
    def Scan(self):
        print('Scan started...')
        self.thread = threading.Thread(target=self.Plot)
        self.thread.start()
        self.timer.start(500)
    def Plot(self):
        print('Update Plot...')
        self.main_plot.plotItem.plot(self.device.I)

class Device:
    def __init__(self):
        self.server = readSoc.ReadSoc('10.200.111.77', 19876)
        self.nBytes = 8192 * 4
        self.I, self.Q = np.zeros((0)), np.zeros((0))
    def scan(self):
        self.I, self.Q = self.server.ReadNext(self.nBytes, 'IQ')
        

 
if __name__ == '__main__':
    app = QtGui.QApplication(sys.argv)
    w = MainWindow()
    w.show()
    sys.exit(app.exec_())

Hello Dave,
I think you have mixed some things in your code. It does not necessarily mean it’s wrong, but it makes the logic of your program a bit confusing.

First, check that you are connecting the self.timer to self.device.scan. While the idea would be to connect the timer to the refreshing of the plot, so you can update the plot independently of how fast you are acquiring data.

Then, in the Scan method, you are creating a new thread for the Plot method. However, this method only runs once. If you would connect the timer to Plot, then the method would run periodically, updating the figure every 500ms, as you have set.

Then, in the Device, when you use the ReadNext, you are always overwriting self.I and self.Q, is this what you want, or do you want to accumulate data? Anyways, you understand your device better. However, what you can do, is that the scan runs an infinite loop until you stop it. For example:

def scan(self):
    print('Scan starting')
    self.keep_scanning = True
    while self.keep_scanning:
        self.I, self.Q = self.server.ReadNext(self.nBytes, 'IQ')
        sleep(self.delay)
    print('Scan ended')

(I’ve added a delay, because I’ve assumed you want to give your device a bit of time between one measurement and the other, but that is up to you).

If you have that, then your window can be updated like this (I’m not adding the complete code, please adapt it):

def __init__(self, parent=None):
    self.timer = QtCore.QTimer()
    self.timer.timeout.connect(self.Plot)
    self.startButton.clicked.connect(self.Scan)
    self.stopButton.clicked.connect(self.stop_scan)

def Scan(self):
    print('Scan started...')
    self.thread = threading.Thread(target=self.device.scan)
    self.thread.start()
    self.timer.start(500)

def stop_scan(self):
    self.device.keep_scanning = False

Beware that I have included a stopButton in the code, which will allow you to stop the scan by changing the value of device.keep_scanning.

In your Plot method, you are using plot, which will keep adding plots to the figure, instead of re-drawing the data. Is that what you want? You may also consider using setData instead, which will re-draw.

I believe, but I may be wrong, that the error you are seeing regarding the timer is related to using Thread(target=self.Plot). Since self.Plot is part of a QMainWindow, the entire window is sent to a different thread and then you end up having the timer badly defined (you have the same timer in two threads, and when you trigger from one, Qt complains, etc.). If you try with the changes from above, you wouldn’t see this error anymore.


Signals

Using signals in this context seems like a good idea, but is not trivial. To do so, you will have to make your Device a child of QObject, and this will allow you to use signals. However, this means that whatever you do with your code, you will ALWAYS need to have Qt installed, even if you are not using a graphic interface. Moreover, Qt works well with threads, but not with multi-processing. Therefore, it may become a bottleneck in the future (as always, depending on where your code is going).

Moreover, if you emit a signal every time data is acquired, you may end up in a situation in which you are refreshing a plot faster than what your screen can deliver (and your eye detect), and therefore I’ve always advised to decouple the refreshing of the plot and the acquisition of data.

Another option instead of using signals, which may be less elegant, but also works, is to have the device update the plot. This is, in my opinion, unorthodox, but it shows the power of shared memory):

def scan(self, plot):
    print('Scan starting')
    self.keep_scanning = True
    while self.keep_scanning:
        self.I, self.Q = self.server.ReadNext(self.nBytes, 'IQ')
        plot.plotItem.plot(self.I)
        sleep(self.delay)
    print('Scan ended')

And then in your window, you change the code, like this:

def Scan(self):
    print('Scan started...')
    self.thread = threading.Thread(target=self.device.scan, args=(self.main_plot, ))
    self.thread.start()

Now you see that the plot will be updated every time there is new data available in your device. Signals in Qt do exactly that, the only difference is that it is easier to develop with them than in the pattern I’ve just shown, but they are not faster in execution nor allow you to use the resources of your computer any better. But again, if you are generating data faster than what the screen can refresh, you will be wasting resources.


Hope I am pointing you in the right direction!