Handling *big* tables nicely

EDIT: implemented Carl’s suggestion from the next post

The problem:
I have a table component that I’m filling with lot’s of data and it locks the application for 10’s of seconds. If fact the spinner thingy in the top right that shows that a screen is loading is also locked. The screen doesn’t display because it’s waiting for the table to fill before it can be drawn.

The solution:
First I unbound the data property on the table. (It called a script to get the data).
Then I added a refresh button and unbound the other components on the screen that would cause the table to be refreshed. In this case I had a number of filters that the user could use to filter the data. Check boxes, text inputs, etc. Every time the user changed something the table would reload. Now the user sets up their filters and then clicks the refresh button.

For the mouseClicked event the button, I added this code:

import app app.history.waitCursor() system.util.invokeAsynchronous(app.history.get_SOE)
app.history.waitCursor() is:

[code]def waitCursor():
import app
app.history.changeCursor(‘Wait’)

def changeCursor(newcursor):
import system
soeWindow = system.gui.getWindow(‘SOELog’)
s = soeWindow.rootContainer.getComponent(‘SOEs’)
table = s.getComponent(‘SOETable’)
if newcursor == ‘Wait’:
table.cursorCode=3
else:
table.cursorCode=0
[/code]

This changes the cursor for the table so the user knows something is happening.
I don’t call changeCursor directly because invokeLater and invokeAsynchronous take the name of a function and no parameters are allowed. (In this case I could have but at one time this call was an invoke call)

app.history.get_SOE is:

[code]def get_SOE():
import app
import system
import re

...
#do major db stuff to collect and process gobs of data
...

soeWindow = system.gui.getWindow('SOELog')
table = soeWindow.rootContainer.getComponent('SOEs').getComponent('SOETable')
newDataset = system.dataset.toDataSet(columns, rowData)
def setData(table=table, newDataset=newDataset):
    table.data = newDataset
system.util.invokeLater(setData)
system.util.invokeLater(app.history.defaultCursor)

[/code]
These last few lines directly set the data property for the table.
When I first wrote this function, it was bound to the data property. The last line was:

    return system.dataset.toDataSet(columns, rowData)

Now that the table is no longer bound, I had it to change to what you see above.
Note the last line. This returns the cursor to the default:

def defaultCursor(): import app app.history.changeCursor('Default')

There was one more step. The table now loads only when the user clicks the Refresh button. I wanted the table to load when the screen was opened. To do this I added the following to the internalFrameOpened event for the window:

#change cursor to spinning and load data import app app.history.waitCursor() system.util.invokeAsynchronous(app.history.get_SOE)

Now I have a windows that opens instantly, fills up the table when the data has been processed, and lets the user know that something is happening.

Very nice! My only suggestion is to change the second-to-last line of the get_SOE script from this:

table.data = system.dataset.toDataSet(columns, rowData) to this:

newDataset = system.dataset.toDataSet(columns, rowData) def setData(table=table, newDataset=newDataset): table.data = newDataset system.util.invokeLater(setData)
This will ensure that the table’s setData() function is called on the Event Dispatch Thread (EDT) which will ensure you don’t get any painting goofs or deadlocks. Unlikely, but better safe than sorry. You can read more about the EDT here and here.

Thanks Carl,

As you can guess I played around with this a fair bit to get it to work. I wasn’t sure about that line, but it all seemed to work fine. I will make the suggest change.

That line #do major db stuff to collect and process gobs of data has undergone major changes this week. I’ll try and post that in a separate thread later.

I have been trying the same thing and have been getting a getComponent error in some lines. Can you tell me if the “SOE’s” is the group or a custom property?

I think it must be a Container

Well, I got it to work and it sure works fast!

I have Momentary buttons that move tags (rows) between these tables. I want the tables to refresh whenever these Momentary buttons are pressed.

It works fine when I put the script from the refresh button into both the “actionPerformed” and the “propertyChange” properties. But, when I close the window an error props up saying propertyChange script couldn’t load because the window is closed.

I tried to change the propertyChange code to act whenever the controlValue changes to 1 and although the error message doesn’t come up, it hampers the functionality.

Is there a better way for it to refresh data?

I’ve gotten that same error in a couple of other places. What’s happening (I think) is a threading issue. The GUI thread spawns a background thread to do a bunch of work. The GUI then closes the window (doesn’t matter why). Then the background thread completes but can’t find anyplace to return to.

What should happen (IMHO) is the background thread should silently die. What does happen is an exception is thrown.

Not sure if the dev team can change it but it would be nice if they did.

Could you provide more detail about the error you’re getting?

In particular, when the client first opens there are a number of screens in a tab control. The first screen loads. If the user clicks the tabs in quick succession the screens open and get swapped out by the next screen. If one of those screen calls a procedure that runs in the background for a “while”, the screen can be swapped out before it returns. And this error happens.

I think I have coded around it or redesigned it so that it doesn’t happen anymore because I haven’t seen it in a while.