In the past weeks, I explained how we can start asynchronous Java threads to run in parallel to the main Matlab processing using Java, Dot-Net and C++ POSIX threads. Today I conclude the mini-series by examining two other alternatives, timer objects and process-spawning. As we shall see below, these are not “real” multi-threading alternatives, but they can indeed be important in certain use-cases.
Matlab timers
Multithreading helps application performance in two related but distinct ways:
- By allowing code to run in parallel, on different CPUs or cores
- By allowing code to run asynchronously, rather than in serial manner
C++, Java and .Net threads can improve performance by both of these manners. Matlab timers, on the other hand, only enable the second option, of running code asynchronously. The reason for this is that all M-code, including timer callback code, is executed by Matlab’s interpreter on a single processing thread (MT).
So, while a timer callback executes, no other M-code can run. This may seem on the face of it to be unhelpful. But in fact, the ability to schedule a Matlab processing task for later (non-serial) invocation, could be very handy, if we can time it so that the timer callback is triggered when the application is idle, for example, waiting for user input, following complex GUI update, or during late hours of the night.
I continue using last weeks’ example, where we compute some data, save it to file on a relatively slow USB/network disk, and then proceed with another calculation. The purpose of multi-threading would be to offload the I/O onto a separate thread, so that the Matlab computation can continue in parallel without needing to wait for the slow I/O. Here is an implementation of our asynchronous I/O example, this time using Matlab timers. First we define the timer’s callback function, using pure M-code (this is the Matlab equivalent of the run() method in the previous examples):
function timerCallback(hTimer,eventData,filename,data)
try
fid = fopen('F:\test.data','w');
fwrite(fid,data,'double');
fclose(fid);
catch
err = lasterror;
fprintf(2,'Error saving to file:\n%s\n',err.message);
end
endWe can now use this timer in Matlab, similarly to our previous examples:
data = rand(5e6,1); % pre-processing (5M elements, ~40MB)
timerFcn = {@timerCallback,'F:\test.data',data};
start(Timer('StartDelay',2, 'TimerFcn',timerFcn)); % start after 2sec
data = fft(data); % post-processing (timer I/O will run later!)The difference vs. our earlier examples is that the timer code is not run in parallel to the fft post-processing, but rather 2 seconds later, when MT is hopefully idle.
I often employ Matlab timers in GUIs: the initial GUI is presented to the user immediately, and then a timer is used to load data from I/O, something which could take long seconds. During this time the user can peruse the GUI, getting a feeling of improved responsiveness compared to a situation of having to wait all those long seconds for the GUI to initially load. This relates to the interesting topics of perceived performance and lazy/delayed evaluation. Matlab timers are certainly under-appreciated for their performance usefulness.
As a related usage, GUI callbacks should be designed to be as short as possible. The quicker the callback, the more responsive the GUI. Users may lose patience with long callback execution. Unfortunately, callbacks sometimes need to perform a lengthy computation or update (for example, updating an Excel file). In such a case, consider delegating the lengthy update task to a timer object that will execute asynchronously, and enable the synchronous callback to complete its work much quicker. This will ensure better GUI responsiveness without affecting the actual program logic. Here is a simple example (we may wish to specify some more timer properties – the snippet below is only meant for illustration):
% A utility function that performs a lengthy calculation/update
function utilityFcn()
% some lengthy calculation/update done here
end
% Regular callback function – takes a long time to complete
function myCallbackFcn(varagin)
% Call the utility function directly (synchronously)
utilityFcn();
end
% A better callback – completes much faster, using asynchronous timer
function myCallbackFcn(varagin)
% Start an asynchronous timer to perform the lengthy update
start(timer('StartDelay',0.5, 'TimerFcn',@utilityFcn));
endSimilarly, when plotting real-time data, we can employ a timer to periodically update the graph in near-real-time, enabling the main processing to work in near-parallel.
Matlab timers have an advantage over Java/C++/.Net multithreading in their synchronization with Matlab, since the M-code interpreter is single-threaded. We just need to handle cases where a timer callback might interrupt other M-code.
Matlab timers run pure M-code, so there is no need to know Java/C#/C++ or to use external compilers, and they are easy to set up and use. They can be very effective when tasks can be postponed asynchronously to when the MT is idle.
Spawning external processes
In some cases, it is impractical to create additional processing threads. For example, we might only have the processing element in executable binary format, or we might wish to use a separate memory space for the processing, to sandbox (isolate) it from the main application. In such cases, we can spawn heavyweight processes (as opposed to lightweight threads), either directly from within Matlab, or externally.
The simplest way to spawn an external process in Matlab is using the system function. This function accepts a string that will be evaluated in the OS prompt (shell), at Matlab’s current folder. By appending a ‘&’ character to the end of the string, we let Matlab return immediately, and the spawned process will run asynchronously (in parallel to Matlab); otherwise, Matlab will block until the spawned process ends (i.e., synchronous invocation of the process).
system('program arg1 arg2'); % blocking, synchronous
system('program arg1 arg2 &'); % non-blocking, asynchronousMatlab normally uses only a single core on a single CPU, except when using the Parallel Computing Toolbox or when doing some implicit parallelization of vectorized code. Therefore, on a quad-core dual-CPU machine, we would normally see Matlab’s CPU usage at only 1/(2*4)=12%. The simplest way to utilize the unused CPU cores without PCT is to spawn additional Matlab processes. This can be done using the system function, as above. The spawned Matlab sessions can be made to run specific commands or functions. For example:
system('matlab –r "for idx=1:100, doSomething(idx); end" &');
system(['matlab –r "processFile(' filename ');" &']);At this point, we may possibly wish to use processor affinity to ensure that each process runs on a separate CPU. Different OSes have different ways of doing this. For example, on Windows it can easily be done using Process Explorer’s context menu.
When Matlab spawns an external process, it passes to it the set of environment variables used in Matlab. This may be different than the set that is normally used when running the same process from the OS’s command prompt. This could lead to unexpected results, so care should be taken to update such environment variables in Matlab before spawning the process, if they could affect its outcome.
Once an asynchronous (non-blocking) process is started, Matlab does not provide a way to synchronize with it. We could of course employ external signals or the state or contents of some disk file, to let the Matlab process know that one or more of the spawned processes has ended. When multiple processes are spawned, we might wish to employ some sort of load balancing for optimal throughput.
We can use OS commands to check if a spawned processId is still running. This ID is not provided by system so we need to determine it right after spawning the process. On Unix systems (Linux and Mac), both of these can be done using a system call to the OS’s ps command; on Windows we can use the tasklist or wmic commands.
An alternative is to use Java’s built-in process synchronization mechanism, which enables more control over a spawned process. The idea is to spawn an external asynchronous process via Java, continue the Matlab processing, and later (if and when needed) wait for the external process to complete:
runtime = java.lang.Runtime.getRuntime();
process = runtime.exec('program arg1 arg2'); % non-blocking
% Continue Matlab processing in parallel to spawned processWhen we need to collect scalar results, we could use the process’ result code:
rc = process.waitFor(); % block Matlab until external program ends
rc = process.exitValue(); % fetch an ended process' return code
Or, if we need to abandon the work, we could stop the spawned process:
process.destroy(); % force-kill the process (rc will be 1)
While this mechanism enables synchronization of the Matlab and external process at the basic execution level, it does not enable synchronization of the data. Doing this between processes (that have independent memory spaces) is much harder (and slower) than it is between threads (that share their memory) or MEX. For inter-process data synchronization (known as IPC, or Inter-Process Communication), we can use shared memory, named pipes or data files. There are various mechanisms and libraries that enable this using C++ and Java that could be used in Matlab. Examples of memory sharing are Joshua Dillon’s sharedmatrix and Kevin Stone’s SharedMemory utilities, which use POSIX shared-memory and the Boost IPC library (SharedMemory is an improved version of sharedmatrix). Rice University’s TreadMarks library is another example of a shared-memory approach that has been used with Matlab, in the MATmarks package (whose current availability is unclear).
Named pipes can be used on Unix systems (Linux and Mac). In this case, the source process sends information to the pipe and the destination process reads from it. After setting up the pipe in the OS, it can be opened, updated and closed just like any other data file. Unfortunately, this mechanism is not generally used on Windows.
Matlab includes a dedicated doc-page showing how to synchronize inter-process data using disk files. An approach that combines memory sharing and files is use of memory-mapped files on R2008a or newer (memmapfile was buggy before then, so I suggest not using it on earlier releases).
Finally, we can use Matlab’s documented ability to serve as a COM/DCOM (automation) server to communicate with it from the external process via the COM interface. Data can be exchanged and Matlab functionality can be invoked by the process.
Followup – MEX functions in R2014a
A few weeks ago I reported that hundreds of internal MEX functions that were previously available and which were enormously useful for a variety of use-cases, have been removed in the R2014a pre-release. Now that the official R2014a has been released, I am happy to report that most of the more-important MEX functions have been restored in the official release (see details in the article addendum), perhaps in some part due to lobbying by yours truly and by others.
MathWorks should be commended for their meaningful dialog with users and for making the fixes in such a short turn-around before the official release, despite the fact that they belong to the undocumented netherworld. MathWorks may appear superficially to be like any other corporate monolith, but when you scratch the surface you discover that there are people there who really care about users, not just the corporate bottom line. I really like this aspect of their corporate culture. I wish all software developers were as receptive to user input as MathWorks is.
Related posts:
- Explicit multi-threading in Matlab – part 3 Matlab performance can be improved by employing POSIX threads in C/C++ code. ...
- Explicit multi-threading in Matlab – part 2 Matlab performance can be improved by employing .Net (C#, VB, F# or C++) threads. ...
- Explicit multi-threading in Matlab – part 1 Explicit multi-threading can be achieved in Matlab by a variety of simple means. ...
- Multi-line tooltips Multi-line tooltips are very easy to set up, once you know your way around a few undocumented hiccups....
