I have a tree which I populate with a file list from my computer on the click of a button. While the program is populating the tree the whole GUI hangs.

Is there a way to populate the tree in another thread so the tree gets populated as the files are found and not when everything has already been added? Or any other ideas?

This is what it looks like:

Screenshot of App

I am populating the tree with this code:

def traversePaths(self, path, parent):
        files = os.listdir(path)
        for myfile in files:
            if myfile[0] != ".":
                if os.path.isdir(path + os.sep + myfile):
                    current = self.filelist.append(parent,[self.dirIcon,self.dirIcon,self.dirOpenIcon,myfile,"DD",True,True,3])
                    self.traversePaths(path + os.sep + myfile, current)
                    current = self.filelist.append(parent,[self.fileIcon,self.dirIcon,self.dirOpenIcon,myfile,"AA",True,True,3])

which is executed on a button click:

def on_refreshbutton_clicked(self, button):
    self.traversePaths(self.path.get_filename(), None)

I don't think threading.Thread works because of the gtk thread for the gui and cannot find gi.repository.Gtk api's on the subject

Any ideas?


When asking programming questions, it is always best to provide a minimal working example and not just code snippets.

Without a working example to play with, I have the following suggestions:

  1. Use a thread to walk the directory tree and put your results in a second separate treemodel (not connected to a treeview). This means you have two treemodels, one connected (which gets rendered to the screen) and one not connected (which therefore does not get rendered)
  2. Add a timeout of a few seconds, which does calls a function that detaches and deletes the first treemodel, copies the second treemodel to a new one, which now serves as the "thread" treemodel and attaches the second tree model.

The reason why you see the GUI hanging is that the file walker takes very long and every time you add a file, a lot of overhead in Gtk is called. By adding a full treemodel, this overhead is only called one time. And by using a thread for the file walking, your GUI stays responsive.

As for the threading part in Gtk, you might want to look here: https://stackoverflow.com/questions/8120860/python-doing-some-work-on-background-with-gtk-gui

A couple of notes to your code:

  • Python has a build-in file walker, which might be faster but is certainly shorter than your code: os.walk
  • If you want to use your code instead, remember that Python has a build-in recursion limit. Depending on the size of your file system, you might want to replace the recursion with a trampoline like construct
| improve this answer | |
  • Thanks your answer was very helpful, I got it working, now I just need to get the thing stable – ender Aug 25 '12 at 1:34

Here's a complete example that updates tree concurrently using a single TreeStore:

import os
import threading
import time
from itertools import cycle

from gi.repository import GObject, Gtk
GObject.threads_init()  # all Gtk is in the main thread;
                        # only GObject.idle_add() is in the background thread

HEARTBEAT = 20   # Hz
CHUNKSIZE = 100  # how many items to process in a single idle_add() callback

def chunks(seq, chunksize):
    """Yield N items at a time from seq."""
    for i in xrange(0, len(seq), chunksize):
        yield seq[i:i + chunksize]

class TreeStore(Gtk.TreeStore):
    __gtype_name__ = 'TreeStore'

    def __init__(self, topdir, done_callback=None):
        Gtk.TreeStore.__init__(self, str)  # super() doesn't work here

        self.path2treeiter = {topdir: None}  # path -> treeiter
        self.topdir = topdir
        self.done_callback = done_callback
        self._cv = threading.Condition()

        t = threading.Thread(target=self._build_tree)
        t.daemon = True
        t.start()  # start background thread

    def _build_tree(self, _sentinel=object()):
        # executed in a background thread
        cv = self._cv
        p = self.path2treeiter
        for dirpath, dirs, files in os.walk(self.topdir):
            # wait until dirpath is appended to the tree
            while p.get(dirpath, _sentinel) is _sentinel:
            parent = p[dirpath]

            # populate tree store
            dirs[:] = sorted(d for d in dirs
                             if d[0] != '.')  # skip hidden dirs
            for chunk in chunks(dirs, CHUNKSIZE):
                GObject.idle_add(self._appenddir, chunk, parent, dirpath)

            for chunk in chunks(sorted(files), CHUNKSIZE):
                GObject.idle_add(self._appendfile, chunk, parent)

    def _appenddir(self, chunk, parent, dirpath):
        # executed in the main thread
        p = self.path2treeiter
        for d in chunk:
            p[os.path.join(dirpath, d)] = self.append(parent, [d])

    def _appendfile(self, chunk, parent):
        # executed in the main thread
        for f in chunk:
            self.append(parent, [f])

class Window(Gtk.Window):
    __gtype_name__ = 'Window'

    def __init__(self, topdir):
        super(Window, self).__init__(type=Gtk.WindowType.TOPLEVEL)
        self.__start_time = time.time()
        self.__title = 'GTK Tree MultiThreading Demo'
        self.set_default_size(640, 480)

        # create tree
        tree_store = TreeStore(topdir, self._on_tree_completed)
        tree_view = Gtk.TreeView()

        cell = Gtk.CellRendererText()
        tree_view.append_column(Gtk.TreeViewColumn(topdir, cell, text=0))
        scrolled_window = Gtk.ScrolledWindow()

        # update title to show that we are alive
        self._update_id = GObject.timeout_add(int(1e3 / HEARTBEAT),

    def _on_tree_completed(self):
        if self._update_id is None:
        # stop updates
        self._update_id = None
        self.set_title('%s %s %.1f' % (self.__title, ' (done)',
                                       time.time() - self.__start_time))

    def _update_title(self, _suff=cycle('/|\-')):
        self.set_title('%s %s %.1f' % (self.__title, next(_suff),
                                       time.time() - self.__start_time))
        return True  # continue updates

win = Window(topdir=os.path.expanduser('~'))
win.connect('delete-event', Gtk.main_quit)

As @xubuntix said each treestore.append() is expensive in this case. See whether it is acceptable for you.

| improve this answer | |
  • Sorry my reply took so long, your solutions looked perfect but did not have time to test it. Just finished implementing it and it works perfectly. Thanks a lot. It solved all the crashes and the program looks stable now. I'm just not used to using threads so needed the help – ender Oct 12 '12 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.