Monday 27 May 2019

Watching A Folder

Been a while since I published some (hopefully) useful Xojo code, so this time I have quite an interesting one.

There are times when it is necessary to get your application to respond to changes in the file system. This is often used for programmers tools where a directory listing is used rather than storing each of the files in the project file, or if your application needs to process data sent via FTP. There are many other scenarios where this feature could be equally useful, but these are the only two I have had to address my self, so far.

In the .Net world, we have the FileSystemWatcher class that can accomplish this for us quite easily. Unfortunately in Xojo, we do not, so we need to roll our own.

The first thought behind this is to simply create a Timer that periodically checks the folder for changes.

OK. But there are some issues with this:
  1. How does the Timer event know if the folder has been altered?
  2. What happens with a particularly large folder? Just using the Time Action event would hijack the main thread and cause our application to become unresponsive when the checking is being done.
  3. What happens if a subfolder is amended?

There are other potential issues, but these are the main three.

My solution is to create a new class, that I will call FolderWatcher.

Class FolderWatcher

End Class

This class does not need to inherit anything, nor use any interfaces, so simply select the Class option from the Insert menu.

When creating a class that fires events, as this will, I like to set up the events definitions at the start, so right now selective Event Definition option from the Insert Menu and name the definition Action. This doesn’t need any parameters as the FolderItem will be retrievable from the class instance.

Talking of the FolderItem, we ought to set up that property now. From the Insert menu, click Property. Call this property mFolder, set the Type to FolderItem and its scope should be Private.



As I said before, we will need to retrieve the FolderItem during the Action event, so add a Computed Property, called Folder, keep this property Public so we can access it from outside the FolderWatcher instance. In the Get method of the Folder property, simply return the mFolder private property


.
I said before that using a Timer to check the folder would be inefficient and cause our application to lock up. However, we do need a Timer to trigger the file check and to see if the flag (which we will set up in a minute), to indicate the folder has changed, has been set.
So now we create the Timer property of the FolderWatcher. For simplicity sake, let us just call this mTimer and set it’s scope to Private.



We will also need two private flags within the class. So, set up two private Boolean properties (I’m sure you have worked out how to create properties by now) and call them mEnabled and mFolderChanged. Set both of the default values of these properties to False.
We only need to expose the mEnabled property to outside the class, so we do this with a computed Property called Enabled. This is also a boolean property. The Get method of this property simply returns the value of the mEnabled private property. However, the Set method sets the value of mEnabled, but also sets the Enabled property of the mTimer property.

  mEnabled = value
  mTimer.Enabled = value

There are just two more properties to add, so we may as well do that now, before we move on to the methods.

Add a private property called mThread. This is of type Thread and will be used to perform the check of the folder without interrupting the main thread.
Add another private property called mFolderDetails and set this to a String. This will store the structure of the folder for comparison during the execution of mThread.

Where have we got too so far?

By now, you should have a class that looks like this:



If you were to try to use this class right not, it would do nothing. Now we need to add the functionality with methods.

First up, the all important constructor.

Create the constructor like any other method. Select the Method option from the Insert menu.
Name this method Constructor and ensure its scope is set to Public. We also need to pass two parameters to the constructor: A folderItem, we will call Folder and a Boolean we will call AutoStart.
The constructor is where we will set everything up.
Firstly, check that we have actually passed a Folder, and not a file to the constructor. This is simply a case of testing the Directory property of the Folder variable. If this returns False, raise a runtime exception.
Now we have the preliminary check out of the way, we can continue with the set up:
Store the Folder in the classes mFolder property.
Create an instance of the Thread class and configure it. The only configuration we need to do here is set the event handler (which we will create in a moment).
Create an instance of the Timer class, and configure that also. This requires a little more configuration. We set the event handler (again, configured in a moment), the period between each trigger of the Timer Action event, the Mode property of the Time, and whether or not we want the Timer to start immediately.

We should end up with code like this in the constructor:

' This class only watches folders, not individual items, so if the
' item passed is not a folder/directory, raise an exception
If Not Folder.Directory Then Raise New RuntimeException

' Store the folder
mFolder = Folder

' Create a new Thread object and attach the Run handler
mThread = New Thread()
AddHandler mThread.Run, AddressOf ThreadAction

' Create the Timer object and add the Action handler
mTimer = New Timer()
AddHandler mTimer.Action, AddressOf TimerAction

' For now, set this Timer to fire every second
mTimer.Period = 1000
mTimer.Mode = Timer.ModeMultiple

' Enable the Timer if AutoStart os True
mTimer.Enabled = AutoStart

The first thing we need to address (no pun intended) is creating the event handlers for the Thread and the Timer. These are the key methods that allow this class to perform its desired task.

To create the event handler for the Thread, we simply create a new method, as we did for the constructor. This time, we will call the method ThreadAction, we set it to Private and have a single parameter called Sender of type Thread.



This method does not return a value.
The body of this method gets the structure of the folder and compares it to the stored structure. If there is a difference, then the mFolderChanged flag is set to True.

' When the Thread runs, get the string value of the folder structure
Dim NewDetails As String = GetFolderDetails(mFolder)

' If the structure has changed since the last check (or this is the first check)
' update the stored details and set the folder changed flag to true
If NewDetails <> mFolderDetails Then
mFolderDetails = NewDetails
mFolderChanged = True
End If

That is all there is to it. The main work of the thread is actually performed in the method GetFolderDetails, but we will get to that after setting up the event handler for the Timer.

To create the event handler for the Timer, we simply create a method in the class. As we can see in the AddHandler call for mTimer, we need to call this method TimerAction. So, that’s what we will do. Repeat the steps to create the ThreadAction method, but this time, call it TimerAction and have the parameter be of type Timer:



The body of the TimerAction method is just as simple as that of the ThreadAction:
Check the FolderWatcher is enabled, if not, exit the method immediately. If the mFolderChanged flag is set to True, reset it to False and then raise the Action event of our FolderWatcher class. Finally, if mThread is not already running, call the Run method of mThread:

' If the file watcher is not enabled, return from this handler
if Not mEnabled Then Return

' If the folder changed flag is set to true, 
' reset it to false and raise the FolderWatcher.Action event handler
If mFolderChanged Then
mFolderChanged = False
RaiseEvent Action()
End If

' If the thread not is running, call its Run method now
If mThread.State <> Thread.Running Then mThread.Run()

The GetFolderDetails method is the main work horse of the class, this is created as any other method, but this time, we pass a FolderItem as the only parameter and we return a string containing the structure of the folder.
As this is more complex that the other methods, we will go through this in closer detail:

Dim FolderDetails() As String ' An array of the folder contents
Dim Index As Integer = 0 ' The current index of the folder’s children
Dim YieldCount As Integer = 10 ' This is used to reduce the amount of yields

These three variables are initialised at the beginning. The first two should be self explanatory, however, the third will require some explanation.
When we execute a separate thread, we need to periodically yield control back to the main thread to prevent the application locking up. This is particularly important in long running, or potentially long running, threads. From my experience, the standard practice for this is to yield control at the end of each iteration of a loop. However, this can effectively cause the thread to be much slower as the ‘behind the scenes’ code to yield the thread builds up. To prevent this, I have found a yield counter is useful. This decrements down to zero on each iteration, when it hits zero, it yields the execution to the main thread. Depending on the complexity of your thread, you may want the yield counter to start at a higher value (say, 100), or lower, if the loop is particularly complex (possibly 2). 
We can now examine each child of the FolderItem passed to the method. This is done with a simple While loop, existing when the Index is equal to, or greater than the number of children.
 
' Get each child of the folder
While Index < Folder.Count

As this example is using the classic framework, the child items of a FolderItem are number from 1, not 0. So, we increment the Index at the start of the loop.

' Using the Classic Framework, so the FolderItem 
' children are in a 1-based array. Therefore,
' the increment is done at the beginning of the loop
Index = Index + 1
  
Get the name and modification date of the child and create a string based upon this details. In this example, I use the TotalSeconds property of the modification date. Please note, in a real world example, it may also be an idea to get the file size and add that to the string, but we won’t worry for this example.

' Get the name and modification date of the child FolderItem
' (NB: Should also be checking file size)
Dim ChildFolder As FolderItem = Folder.Item(Index)
Dim ChildName As String = ChildFolder.Name + " : " + _
Str(ChildFolder.ModificationDate.TotalSeconds)
  
If this child is a folder, we add a marker to the beginning of the string (in this case , I add “[D] “ and append the contents of that folder. TO get the contents of the folder, simply call this method recursively, passing the child as the FolderItem.

' If the FolderItem is a directory, then Add [D] 
' to the beginning of the name and append the
' structure of that folder by recursively calling this method
If ChildFolder.Directory Then ChildName = “[D] " + ChildName + _
GetFolderDetails(ChildFolder)

Add the final string for the child details to the array of strings that give us our structure .

' Add the details of the FolderItem to the array of details
FolderDetails.Append(ChildName)

Now we do the YieldCounter processing described earlier.

' Decrement the YieldCount
YieldCount = YieldCount - 1

' If the YieldCount is at zero, yield to the next thread 
' to prevent application lock up
If YieldCount = 0 Then
App.YieldToNextThread()
  
' Reset the YieldCount
YieldCount = 10
End If
Wend

Finally, we return the array of strings as a single string.

' Return the details as a single string
Return Join(FolderDetails,"|")


Now, to finish the class, we need to add a Destructor and two convenience methods.

The Destructor is required as shutting down your application with an active Timer or Thread may cause it to crash, and shutdown ungracefully. We want to avoid this for several reasons, including the fact that it doesn’t look very professional if your crashes every time your quit.

A Destructor is created in the same was as a Constructor, however, it does not, and cannot, take any parameters. To create it, just create a method called Destructor with a Public scope:


All this needs to do is call the Stop method of the FolderWatcher:

' Stop the FolderWatcher to prevent errors on application close
Stop()


However, we don’t have a Stop method, se we will create it now.
The Stop method is simply a convenience method that sets the Enabled property to False.
Create this by creating a public method called Stop, that takes no parameters, and give it the following body:

' Stop the FolderWatcher Timer
Enabled = False

To compliment this, we can create a Watch method that does the inverse, setting the Enabled parameter too True.
Create this by creating a public method called Watch, that takes no parameters, and give it the following body:

' Start the File Watcher
Enabled = True

And that is the FolderWatcher class, we should now have a class that looks like this:

Class FolderWatcher

Event Action()

Sub Constructor(Folder As FolderItem, AutoStart As Boolean = False) 
' This class only watches folders, not individual items, so if the
' item passed is not a folder/directory, raise an exception
If Not Folder.Directory Then Raise New RuntimeException
' Store the folder 
mFolder = Folder
' Create a new Thread object and attach the Run handler 
mThread = New Thread()
AddHandler mThread.Run, AddressOf ThreadAction

' Create the Timer object and add the Action handler 
mTimer = New Timer()
AddHandler mTimer.Action, AddressOf TimerAction

' For now, set this Timer to fire every second 
mTimer.Period = 1000
mTimer.Mode = Timer.ModeMultiple
' Enable the Timer if AutoStart is True 
Enabled = AutoStart

End Sub

Sub Destructor()

' Stop the FolderWatcher to prevent errors on application close 
Stop()

End Sub

Private Function GetFolderDetails(Folder As FolderItem) As String

Dim FolderDetails() As String ' An array of the folder contents
Dim Index As Integer = 0 ' The current index of the folderʼs children
Dim YieldCount As Integer = 10 ' This is used to reduce the amount of yields
' Get each child of the folder 
While Index < Folder.Count

' Using the Classic Framework, so the FolderItem
' children are in a 1-based array. Therefore,
' the increment is done at the beginning of the loop 
Index = Index + 1

' Get the name and modification date of the child FolderItem ' (NB: Should also be checking file size)
Dim ChildFolder As FolderItem = Folder.Item(Index)
Dim ChildName As String = ChildFolder.Name + " : " + _
Str(ChildFolder.ModificationDate.TotalSeconds)

' If the FolderItem is a directory, then Add [D]
' to the beginning of the name and append the
' structure of that folder by recursively calling this method
If ChildFolder.Directory Then ChildName = "[D] " + ChildName + _
GetFolderDetails(ChildFolder)

' Add the details of the FolderItem to the array of details 
FolderDetails.Append(ChildName)

' Decrement the YieldCount 
YieldCount = YieldCount - 1

' If the YieldCount is at zero, yield to the next thread 
' to prevent application lock up
If YieldCount = 0 Then
App.YieldToNextThread()

' Reset the YieldCount
YieldCount = 10
End If
Wend

' Return the details as a single string
Return Join(FolderDetails,"|")

End Function

Sub Stop()

' Stop the FolderWatcher Timer 
Enabled = False

End Sub

Private Sub ThreadAction(Sender As Thread)

' When the Thread runs, get the string value of the folder structure 
Dim NewDetails As String = GetFolderDetails(mFolder)

' If the structure has changed since the last check (or this is the first check) 
' update the stored details and set the folder changed flag to true
If NewDetails <> mFolderDetails Then
mFolderDetails = NewDetails
mFolderChanged = True 
End If

End Sub

Private Sub TimerAction(Sender As Timer)

' If the file watcher is not enabled, return from this handler 
If Not mEnabled Then Return
' If the folder changed flag is set to true, reset it to false and raise the FolderWatcher.Action event handler
If mFolderChanged Then 
mFolderChanged = False 
RaiseEvent Action()
End If

' If the thread not is running, call its Run method now
If mThread.State <> Thread.Running Then mThread.Run()

End Sub

Sub Watch()

' Start the File Watcher 
Enabled = True

End Sub

Enabled As Boolean
Get
return mEnabled
End Get
Set
mEnabled = value
mTimer.Enabled = value
if mThread.State = Thread.Running Then mThread.Kill()
End Set 
End Property

Folder As FolderItem
Get
Return mFolder
End Get 
End Property

mEnabled As Boolean = False 
mFolder As FolderItem 

mFolderChanged As Boolean = False 

mFolderDetails As String

mThread As Thread

mTimer As Timer 

End Class


So, after all that, how do we use the FolderWatcher class?

Add a FolderWatcher property to your application
Create an instance with the folder you wish to watch
Add an event handler
Respond to the event handler

Or in a little more detail, If we are adding the FolderWatcher to our main window to watch the Desktop folder:
Create a new property of type FolderWatcher
In the Open event of your Window, create an instance of FolderWatcher, passing the folder you wish to watch
Add an event handler to the Action event of the FolderWatcher
Call the Watch method of the FolderWatcher

mFolderWatcher = New FolderWatcher(SpecialFolder.Desktop)
AddHandler mFolderWatcher.Action, AddressOf FolderWatcherAction
mFolderWatcher.Watch()

I have provided a download to a small demonstration application that watches your desktop. Run it and then alter your files on the desktop and it should respond.



No comments:

Post a Comment