With 2012 behind us and a fresh 2013 ahead, it is time again for a short look at this website’s achievements so far, and plans for the future.

I started this blog nearly four years ago, thinking it would be nice to post about a few dozen undocumented features. I had no idea whether this material would be of any use to Matlab users, and I expected the website to be a niche blog at best. In fact, I was not sure that at a rate of one article per week I would have enough content for more than a year or two.

200 posts, 2000 reader comments, and 300K unique visitors later, an admission should now be made that I hugely underestimated the impact that this blog would make. In fact, my pipeline of future articles is now longer than ever (a full year long), and the readership growth appears to quickly approach the physical limit of total Matlab community size.

2012 in review

In 2012, I published 50 articles, including:

• a few posts about integrating 3rd-party Java libraries in Matlab (Waterloo, Groovy, JFreeChart, Java Collections)
• a mini-series about customizations to figure window menus (part1, part2, part3)
• a mini-series about customizations to the built-in Matlab profiler
• a few articles about customizing the Matlab desktop (editor, mlint failures, trapping warnings, Command Window bold text)
• many articles about Matlab’s built-in Java and its usages
• several articles on Matlab’s internal memory (leaks #1, leaks #2, profiling, in-place editing, internal format, preallocation, resizing, optimizations)
• a few articles on compiled (deployed Matlab applications (speed-up, splash window, privacy, disabling menu-items)
• quite a few articles about miscellaneous undocumented Matlab topics:
  • GUI (using spinners, uicontrol images, uitable headers, ScreenCapture utility)
  • graphics (Waterloo, JFreeChart, axes tick labels, scatter plot jitter, pinning annotations)
  • I/O (xlsread, xlswrite, urlread)
  • external interfaces (actxserver), etc.
• last but not least, a few posts dealing with performance, which is one of my favorite topics. In fact you will be reading quite a bit more on this in 2013. This is perhaps the place to announce that I am currently writing my second book on the subject of Matlab Performance Tuning. I hope and expect it to be published later this year, and I will of course inform when it is finally published.

2012 continued the trend in previous years of hosting articles by guest bloggers:

• Malcolm Lidierth wrote about integrating Waterloo and Groovy. Malcolm’s Waterloo mini-series quickly became a super-star in terms of readership, facebook likes, and user comments.
• Donn Shull wrote about extending Java classes using UDD. Donn’s series of UDD articles is one of the website’s most highly searched-for terms.
• Alexander Mering wrote about disabling menu items in deployed applications.
• Jim Hokanson wrote about extending the built-in urlread function.
• Peter Li wrote about the internals of Matlab’s memory and mex in-place editing.

Stats for nerds (and potential advertisers…)

Hopefully there was enough material and diversity in there to satisfy different audiences. Judging by the traffic on the site, this appears to be the case indeed. Interest in this website still grows steadily, continuing the trend from the past years. To date, 305,500 unique readers have read at least one article here (two on average), in over 560,000 different visits, reaching a new peak of over 7000 unique visits (plus over a thousand subscribers) per week just before the December holidays. These figures are nearly double those from last year. RSS and email subscription has also continued their steady growth pace. In the following graph, the traffic dips are due to the annual December holidays and the site-overhaul in 2011. The growth trend is quite evident:

Steady readership growth (click for details)

How significant are the absolute numbers?

Well, Matlab reportedly has “more than 1 million” users worldwide. So 305K readers represents about 30% of the total, as an upper bound. Granted, some of those visitors are innocent googlers who happened to stumble across the website and are not Matlab users. But I think we can safely assume that at least half of the visitors, who have visited the website multiple times, are actual Matlab users. Whichever way you look at it, this represents a significant portion of the total Matlab community. Somehow, despite having no investment in marketing, and (naturally) no active support from MathWorks, this website has reached wide-spread community attention. I can’t fully express how happy I am about this. I take it as a resounding community approval, so I intend to continue expanding the site with additional interesting content.

In the past 2 annual perspectives (2011, 2010) I have shown the readership breakup by country and by city. The relative trends that were presented then are still valid. So this time I will display the breakup of the top reader languages, as reported by their browsers. It seems that over two-thirds of you readers speak English (or at least, your browsers do), with German, French and Spanish coming far behind:

Steady readership growth (click for details)

Plans for 2013

In 2013 I plan to continue posting about undocumented aspects of Matlab. Specific plans include the much-overdue articles on uiinspect and its close relative checkClass (this is a promise from last year that I failed to deliver, sorry…). I also hope to cross out additional items in my TODO list. Two mini-series that I hope to get around to, are about Matlab-database connectivity and Matlab’s new toolstrip/ribbon.

There will also be more articles by guest bloggers. I am extremely pleased at the growing list of guest bloggers. If you have an idea for an article, please email me (altmany at gmail).

Finally, last year I started presenting Matlab training courses/seminars, a niche that I hope to expand in 2013. I expect that you will hear more about public events, and I will also continue custom training.

Happy 2013 everybody!

- Yair Altman

1. 2011 perspective & plans for 2012 2011 has seen a continued steady growth in readership of this website. This post takes an overview of past achievements and future plans. ...
2. 2010 perspective & plans for 2011 2009 and 2010 have seen steady growth in readership of this website. This post takes an overview of past achievements and future plans....
3. New York City visit, 21-24 May 2013 I will be visiting New York 21-24 May 2013 to speak at the Matlab Computational Finance Conference. ...
4. TODO List of future posts: Setting preferences programmatically Setting system-tray icons and messages Setting and customizing the status bar Database connectivity using JDBC Complex data structures: hashtables, queues, stacks, deques etc....

A few days ago, a client asked me to integrate an MRU (most-recently-used) file list in a Matlab GUI window. The naive approach was to add a new “Recent files” main menu, but this would look bad. Today I explain how to integrate the MRU list into the toolbar’s standard “Open File” button, as well as into the standard “File” main menu.

Customizing the standard “File” main menu

Note: this relies on earlier articles about customizing the figure menubar, so if you are not comfortable with this topic you might benefit from reading these earlier articles.

Customizing the standard “File” main menu is easy. First, let’s find the “File” main menu’s handle, and add an MRU sub-menu directly to it:

hFileMenu = findall(gcf, 'tag', 'figMenuFile');
hMruMenu = uimenu('Label','Recent files', 'Parent',hFileMenu);

Our new MRU menu item is created at the end (in this case, at the bottom of the “File” main menu list). Let’s move it upward, between “New” and “Open…”, by reordering hFileMenu‘s Children menu items:

hAllMenuItems = allchild(hFileMenu);
set(hFileMenu, 'Children',fliplr(hAllMenuItems([2:end-1,1,end])));  % place in 2nd position, just above the "Open" item

Now let’s fix the “Open…” menu item’s callback to point to our custom openFile() function (unfortunately, the “Open…” menu item has no tag, so we must rely on its label to get its handle):

hOpenMenu = findall(hFileMenu, 'Label', '&Open...');
set(hOpenMenu, 'Callback',@openFile);

Finally, let’s add the MRU list as a sub-menu of the new hMruMenu. I assume that we have a getMRU() function in our code, which returns a cell-array of filenames:

% Add the list of recent files, one item at a time
filenames = getMRU();
for fileIdx = 1 : length(filenames)
    uimenu(hMruMenu, 'Label',filenames{fileIdx}, 'Callback',{@openFile,filenames{fileIdx}});
end

Modified standard figure menu-bar

Clicking the main “Open…” menu item calls our openFile() function without the optional filename input argument, while selecting one of the MRU files will call the openFile() function with that specific filename as input. The openFile() function could be implemented something like this:

% Callback for the open-file functionality (toolbar and menubar)
function openFile(hObject,eventData,filename)
    % If no filename specified, ask for it from the user
    if nargin < 3
        filename = uigetfile({'*.csv','Data files (*.csv)'}, 'Open data file');
        if isempty(filename) || isequal(filename,0)
            return;
        end
    end
 
    % Open the selected file and read the data
    data = readDataFile(filename);
 
    % Update the display
    updateGUI(data)
end

We can take this idea even further by employing HTML formatting, as I have shown in my first article of the menubar mini-series:

HTML-rendered menu items

Customizing the standard toolbar’s “Open File” button

Note: this relies on earlier articles about customizing the figure toolbar, so if you are not comfortable with this topic you might benefit from reading these earlier articles.

The basic idea here is to replace the standard toolbar’s “Open File” pushbutton with a new uisplittool button that will contain the MRU list in its picker-menu.

The first step is to get the handle of the toolbar’s “Open File” button:

hOpen = findall(gcf, 'tooltipstring', 'Open File');
hOpen = findall(gcf, 'tag', 'Standard.FileOpen');  % Alternative

The second alternative is better for non-English Matlab installations where the tooltip text may be different, or in cases where we might have another GUI control with this specific tooltip. On the other hand, the 'Standard.FileOpen' tag may be different in different Matlab releases. So choose whichever option is best for your specific needs.

Assuming we have a valid (non-empty) hOpen handle, we get its properties data for later use:

open_data = get(hOpen);
hToolbar = open_data.Parent;

We have all the information we need, so we can now simply delete the existing toolbar open button, and create a new split-button with the properties data that we just got:

delete(hOpen);
hNewOpen = uisplittool('Parent',hToolbar, ...
                       'CData',open_data.CData, ...
                       'Tooltip',open_data.TooltipString, ...
                       'ClickedCallback',@openFile);

As with the menubar, the button is now created, but it appears on the toolbar's right edge. Let's move it to the far left. We could theoretically reorder hToolbar's Children as for the menubar above, but Matlab has an internal bug that causes some toolbar buttons to misbehave upon rearranging. Using Java solves this:

drawnow;  % this innocent drawnow is *very* important, otherwise Matlab might crash!
jToolbar = get(get(hToolbar,'JavaContainer'),'ComponentPeer');
jButtons = jToolbar.getComponents;
jToolbar.setComponentZOrder(jButtons(end),2);  % Move to the position between "New" and "Save"
jToolbar.revalidate;  % update the toolbar's appearance (drawnow will not do this)

Finally, let's add the list of recent files to the new split-button's picker menu:

% (Use the same list of filenames as for the menubar's MRU list)
jNewOpen = get(hNewOpen,'JavaContainer');
jNewOpenMenu = jNewOpen.getMenuComponent;
for fileIdx = 1 : length(filenames)
    jMenuItem = handle(jNewOpenMenu.add(filenames{fileIdx}),'CallbackProperties');
    set(jMenuItem,'ActionPerformedCallback',{@openFile,filenames{fileIdx}});
end

Modified standard figure toolbar

Clicking the main Open button calls our openFile() function without the optional filename input argument, while clicking the picker button (to the right of the main Open button) and selecting one of the files will call the openFile() function with that specific filename as input.

That's it. Quite painless in fact.

1. Customizing figure toolbar background Setting the figure toolbar's background color can easily be done using just a tiny bit of Java magic powder. This article explains how. ...
2. Modifying default toolbar/menubar actions The default Matlab figure toolbar and menu actions can easily be modified using simple pure-Matlab code. This article explains how....
3. Figure toolbar components Matlab's toolbars can be customized using a combination of undocumented Matlab and Java hacks. This article describes how to access existing toolbar icons and how to add non-button toolbar components....
4. Figure toolbar customizations Matlab's toolbars can be customized using a combination of undocumented Matlab and Java hacks. This article describes how to customize the Matlab figure toolbar....

After several years in which I have mentioned my uiinspect utility in posts, I figured it is high time to actually describe this utility in detail.

uiinspect in action (Java, HG, COM)

uiinspect, downloadable from the Matlab file Exchange, is a Matlab GUI utility that inspects the specified object and provides detailed information about its super-classes, methods, properties, static fields and callbacks in a unified Matlab window. uiinspect works on a very wide variety of inputs: Matlab/Java/Dot-Net class names and class objects; COM/DCOM objects, Handle Graphics handles etc.

In essence, uiinspect incorporates the information presented by the following built-in Matlab functions: inspect, get, and methodsview. uiinspect also presents additional aspects that are not available in any built-in Matlab function (for example, inheritance information, undocumented hidden properties, properties meta-data, grouping of callbacks by type).

uiinspect displays hidden properties and fields that are not normally displayed in Matlab (see my related getundoc utility). Property meta-data such as type, accessibility, visibility and default value are also displayed. Object properties and callbacks may be modified interactively within the uiinspect window.

Of over 40 utilities that I have so-far submitted to the File Exchange, uiinspect is one of my most complex (together with findjobj). It has undergone 24 revisions since its initial release in 2007. The latest revision has nearly 3000 source-code lines, of which 75% are code lines, 20% are comment lines and the rest are empty spacer lines. That’s a pretty complex utility to describe, and it relies on plenty of undocumented aspects, so today’s post will only highlight the important aspects. Readers are more than welcome to have a look at the source code for additional details. It is pretty-well documented, if I may say so myself.

Usage syntax

The basic syntax is:

hFig = uiinspect(handle)

Examples:

hFig = uiinspect(0);                         % the root handle
hFig = uiinspect(handle(0));
hFig = uiinspect(gcf);                       % current figure
hFig = uiinspect(handle(gcf));
uiinspect('java.lang.String');               % Java classname
uiinspect(java.lang.String('yes'));          % Java object
uiinspect(get(gcf,'JavaFrame'));             % Java object
uiinspect(classhandle(handle(gcf)));         % UDD class object
uiinspect(findprop(handle(gcf),'MenuBar'));  % UDD property
uiinspect(actxserver('Excel.Application'));  % COM object
uiinspect(Employee)                          % Matlab class object
uiinspect(?handle)                           % Matlab metaclass object
uiinspect('meta.class')                      % Matlab class name
uiinspect(System.DateTime.Now)               % Dot-Net object

uiinspect returns a handle to the created figure window. uiinspect opens a regular Matlab figure window which may be accessed via hFig (unlike Matlab’s methodsview and inspect functions which open Java frame that is not accessible from Matlab).

Unlike Matlab’s functions, multiple uiinspect windows can be opened simultaneously, for different objects. The differentiation is done based on figure title, which contains the inspected object’s name (if available) and its class – reinspecting the same object will reuse the existing figure window, but in all other cases a new figure window will be created.

Main panels

uiinspect includes the following information panels, that shall be described separately below:

• Class information (including superclasses, if applicable)
• Methods (for objects) or graphic handle hierarchy (for Handle Graphics)
• Callbacks (where applicable)
• Inspectable properties
• Other properties plus full meta-data (where applicable)

The panels are fully resizable. We can drag the divider lines up/down or left/right and the contents automatically adjust accordingly. Each panel has a minimal viewable width/height, and the dividers cannot be dragged to squeeze the panels below these minimums – they can only be minimized, which hides the relevant panel entirely. To minimize a panel, simply click the relevant small arrow mark on the divider. The opposite arrow mark next to it maximizes the panel, effectively minimizing the panel on the other side. Once minimized/maximized, the divider can be restored by simply clicking it once, or by dragging it (again, panel minimum sizes apply).

uiinspect only uses Java panels, so implementing the dividers required use of the simple JSplitPane. In a general case where we might wish to embed Matlab graphs in one of the side panels, we would need to employ a more sophisticated solution (see my UISplitPane utility).

Class information

The top-left panel displays a label with information about the object’s class and super-classes inheritance (where applicable).

The class name itself is hyper-linked to the class’s documentation: if this is a standard Java class, then to the official online javadoc for this class which opens up in Matlab’s internal web browser. In fact, since different Matlab releases use different JVM versions (1.3 through 1.6), the link points to the documentation page corresponding to the JVM version actually used.

If the class is non-Java, the hyperlink displays the class’s help section in Matlab’s Command Window / console. The panel’s tooltip displays the same information in a slightly different manner.

The hyperlink in the label is actually an optical illusion. In fact, the entire label is hyper-linked, and clicking any part of it will display the relevant documentation (a similar optical illusion is used to display the hyperlink at the footer of the utility window). The illusion is achieved using Matlab’s HTML formatting, where the part of the label string consisting of the class name is underlined. The cursor was dynamically modified to a pointed hand-finger when the mouse hovers over the label, using the following simple Java-based command:

methodsLabel.setCursor(java.awt.Cursor(java.awt.Cursor.HAND_CURSOR));

Special treatment is done to extract the class’s superclass, the interfaces that it implements and any possible class qualifiers (for example, “final”).

For those interested to dig within the code, all this is done in uiinspect‘s getMethodsPane function.

Next to the class label, a checkbox is presented (“Extra”). Clicking this checkbox displays additional meta-data information (qualifiers, interrupts and inheritance) in the methods pane beneath. Not all classes have all these extra meta-data fields – only the relevant extra meta-information fields are displayed. If there are is extra meta-data, then the checkbox is not displayed. This is done in the getMethodsTable function.

Methods or HG hierarchy panel

The utility’s main panel displays either a table of methods (functions) for a class object, or a tree hierarchy for an HG handle.

Class object methods

The methods table takes the information from the getObjMethods function, which is an adaptation of Matlab’s built-in methodsview function. Part of the adaptation is to hyperlink all class references (used in the methods’ inputs, outputs and meta-data), such that clicking them will open new corresponding uiinspect windows.

Class object methods panel

The methods data is displayed within a non-editable Java table (in the getMethodsTable function) that auto-resizes the columns. The table columns are sortable, even sortable on multiple columns by CTRL-clicking (methodsview allows only simple sorting). This is done using JIDE’s TreeTable component. The table is placed within a scroll-pane having automatic scrollbars that only appear as needed.

The table’s MouseMovedCallback property is set to tbMouseMoved, which updates the mouse cursor (either regular or pointed finger) based on the current mouse position in the table (whether over a hyperlinked cell or not).

The table’s MousePressedCallback property is set to tbMousePressed, which opens new uiinspect figure windows for the hyperlinked classes (if any) in the clicked cell.

HG hierarchy tree

For HG handles, getHandleTree creates a Java tree that displays the hierarchy of HG children (as recursively reported by any HG handle’s Children property). For convenience, I have chosen to use the built-in component com.mathworks.hg.peer.UITreePeer that underlies the built-in uitree function. For performance reasons, the tree is not fully evaluated: the inspected handle’s Parent is set as the tree’s Root. The root node is expanded to get all the parent’s Children (i.e., the inspected handle’s siblings), and then the inspected handle’s tree node is again expanded to display its direct descendents.

Normal dynamic HG handle tooltip

A MouseMovedCallback is set on the tree to process mouse hover events in the treeMouseMovedCallback function. This function updates the tree tooltip dynamically, in the sense that it presents a different tooltip for different handles (tree nodes).

Invalid HG handles (this can happen if the HG handle was deleted since the time that uiinspect generated the tree) are displayed with a special warning message.

Invalid HG handle tooltip

This dynamic tree behavior is achieved by storing the relevant handle information in the UserData of the different tree nodes. Unfortunately, starting in R2012a, Matlab has made a change in the internal support of Java objects, and the UserData property is no longer available. Such a case is detected and the data is stored in the tree nodes’ ApplicationData property instead (using setappdata(node,’userdata’,…) ).

Dynamic context (right-click) menu

A MousePressedCallback is set on the tree to process context (right-click) events in the treeMousePressedCallback function. The context-menu is also dynamic, in the sense that it presents a different context menu for different handles (tree nodes), again based on their user-data.

Left-clicking a node is not processed by treeMousePressedCallback, but rather by the tree’s NodeSelectedCallback which is processed in nodeSelected, and by NodeExpandedCallback which is processed by nodeExpanded. nodeSelected reloads uiinspect for the selected handle; nodeExpanded merely displays the expanded handle’s children.

Since the ‘+’ sign (which triggers nodeExpanded) and the handle icon (which triggers nodeSelected) are so close, we should be careful to click the ‘+’, otherwise the entire uiinspect window will reload the tree based on the clicked node… If anyone has a good idea how to solve this dilemma, then I’m all ears.

Like the methods table, the tree is also placed in a dynamic scroll-pane that displays scrollbars only as needed.

Callbacks panel

The callbacks panel, computed in getCbsData is based on a reflection of the object’s data as reported by the undocumented built-in classhandle function. I aggregate all the object’s events, as well as all the object property names that end with ‘Fcn’ or ‘Callback’. This takes care (I hope) of all the different manners by which different kinds of objects raise events that are trappable in Matlab callbacks. Specifically, it takes care of Java/Matlab classes as well as HG handles and COM objects. If anyone thinks that I have forgotten something, please let me know.

The getCbsPane function then displays the callbacks data (callbacks’ property name and value) in a Java table (JIDE PropertyTable, JIDE TreeTable, or failing those a simple JTable).

Modifiable object callbacks

The callbacks are automatically grouped by name into logical groups (in getTreeData). For example, all callbacks whose names start with “Mouse*” are grouped in the “Mouse callbacks” group. The last group is always called “Other callbacks” and contains all callbacks for which a matching sibling callback has not been found. The groups are automatically collapsed by default; if only a single group is found then this group is automatically opened (for example, in the case of uiinspect(0) ).

The callbacks table’s toolbar enables displaying the callbacks by groups or sorted alphabetically. It also has “expand” and “collapse” icons that affect all the groups.

A checkbox next to the table’s toolbar enables hiding standard Java Swing callbacks. This is important when we inspect Java controls and only wish to see its unique callbacks. When using this checkbox, red Java exceptions are sometimes displayed in the Matlab console – these are harmless and you can safely ignore them (I hope to find a way to prevent them one day).

The table’s right column displays the callback properties values (if available). This column is editable and we can interactively modify any callback’s property. As shown, we can enter callback value in either of Matlab’s supported formats: string, function handle and (for non-COM objects only!) a cell array of function handle and additional data. An error message will be displayed if the callback value is invalid or cannot be set for some reason.

If it is determined that there are no callbacks, then the callbacks panel is automatically minimized, to enable maximum space for the methods panel above it.

Properties inspector panel

The properties inspection panel, prepared in getPropsPane, is actually composed of two separate panes: the top pane uses the built-in Matlab component com.mathworks.mlwidgets.inspector.PropertyView, which in turn uses JIDE’s PropertyTable. PropertyView is the same component used by Matlab’s standard inspect function (that’s how I came to know it, if anyone wonders).

uiinspect's property inspector table

The benefit of using Matlab’s PropertyView component rather than JIDE’s PropertyTable is that PropertyView has the very useful method setObject which I use to point the component at the inspected object, which automatically infers its non-hidden properties and updates the table, saving me a lot of work.

There are two drawbacks of using Matlab’s PropertyView:

• PropertyView only displays non-hidden properties. One day when I have time, I intent to add the hidden properties to the resulting JIDE PropertyTable. But for now it only shows non-hidden properties.
• PropertyView causes a Matlab crash on some Matlab releases, in case dbstop if error is active (this can be replicated using Matlab’s standard inspect). I therefore regrettably need to disable this specific dbstop.

I’ve been meaning to do these two fixes ever since I released uiinspect back in 2007, but for now that’s the way it is…

The properties data is retrieved via the getPropsData function. This function uses the built-in Matlab functions meta.class.fromName(className) and metaclass(classObject) to get the class handle of Matlab classes (in getMetaClass); similarly, loadClass loads the class definition for a Java class. I inspect these class handles for their contained properties. I then use the fieldnames function to add static class fields, which are not standard properties (for example, “RED” is a static field of the java.awt.Color class).

From the class handle, I retrieve the full definition of each property. This includes meta-data such as whether the property is regular or hidden (undocumented); settable or not; gettable or not; and any additional qualifiers (e.g., Sealed, Dependent, Constant, Abstract, Transient, Default (factory) value).

Object properties tooltip

We now have a list of all properties and static fields, and this is used to display the entire properties data in the properties panel’s title (“Inspectable object properties”) tooltip. This tooltip, created in updateObjTooltip and getPropsHtml, uses some fancy HTML formatting to display all the object’s properties and values, color- and font-style-coded to show which of the properties is read-only, hidden, undefined etc.

The 'Other properties' meta-data table

The entire information is also displayed in the properties meta-data pane (“Other properties”) beneath JIDE’s inspector pane. Here we use a simple Java table to display the information in color-coding (gray for read-only properties; blue for static fields; red for irretrievable properties).

Separate checkboxes enable displaying all properties (by default only the properties that are NOT displayed in JIDE’s inspector table are displayed); and whether or not to display the extra meta-data in the properties table (by default only the property name and current value are displayed).

In some cases (e.g., Dot-Net objects), Matlab’s inspector does not know how to extract the property-bean information and so the PropertyView inspector is not shown, only the “other properties” table.

Both JIDE’s inspector table and the “other properties” table enable the user to modify existing values. Note that in some cases Matlab prevents interactive update of some properties, and in some other cases I have seen Matlab hang when trying to update a few specific properties. But in most cases updating the value does work as expected.

The combination of the inspector table, the meta-data table and the tooltip, enable users to fully understand the accessible properties of the inspected object. Of course, it would have been much better to merge the JIDE inspector table with the hidden properties (=additional rows) and meta-data (=additional columns). But let’s leave something for the future, shall we?

Auto-update mechanism

uiinspect auto-update notice

uiinspect employs the same auto-update background mechanism used by findjobj – after presenting the GUI, the utility silently checks the File Exchange webpage to see whether any newer version of this utility has been uploaded. If so, then a popup notice is presented with the date and description of the latest version. The popup enables users to download the newer version into their Matlab path, or skip. There is also an option to skip the update and not to remind ever again.

I find this background auto-update mechanism quite useful and generic. In fact, I uploaded it as a separate File Exchange utility today, following Brett Shoelson’s suggestion last month. You can find the underlying code in the checkVersion function.

TODO

• cleanup internal functions, remove duplicates etc.
• link property objects to another uiinspect window for these objects
• display object children (& link to them) – COM/Java
• find a way to merge the other-properties table with the inspector table (hidden props + meta-data)
• find a way to use the inspector without disabling dbstop if error
• Fix: some fields generate a Java Exception from: com.mathworks.mlwidgets.inspector.PropertyRootNode$PropertyListener$1$1.run
• Fix: using the “Hide standard callbacks” checkbox sometimes issues Java Exceptions on the console
• Fix: In HG tree view, sometimes the currently-inspected handle is not automatically selected

I would be happy if anyone can help with any of these.

Conclusion

I believe that this has been my longest blog post ever; certainly the one that I have labored most over. This correlates well with the uiinspect utility, which has been one of my most complex tasks. I’m guessing I must have invested 100-200 man-hours developing and improving it over the years.

I hope you find uiinspect as useful and as fun as I do. I believe that its source-code is certainly worth reading if you are interested in any advanced Matlab GUI programming, showing how Java GUI components can be combined in Matlab. Go ahead and download uiinspect from the Matlab file Exchange.

1. Uitable sorting Matlab's uitables can be sortable using simple undocumented features...
2. Minimize/maximize figure window Matlab figure windows can easily be maximized, minimized and restored using a bit of undocumented magic powder...
3. Uitable customization report In last week’s report about uitable sorting, I offered a report that I have written which covers uitable customization in detail. Several people have asked for more details about the...
4. JIDE Property Grids The JIDE components pre-bundled in Matlab enable creating user-customized property grid tables...

Last week I presented a detailed explanation of the uiinspect utility, which displays a GUI listing all the properties, callbacks and internal methods of an inspected object. Something like Matlab’s inspect tool on steroids. I explained that uiinspect uses com.mathworks.mlwidgets.inspector.PropertyView, which in turn uses JIDE’s PropertyTable. PropertyView automatically extracts the inspected object’s properties and displays them using a corresponding renderer in the PropertyTable.

propertiesGUI demo

We often need to present a non-object Matlab construct. For example, it is very common to store an application’s configuration properties in a struct (using Matlab objects is often impractical and unwarranted) — it would be very useful to present this configuration dynamically, without having to programmatically scan all struct fields and build a dedicated GUI table. Unfortunately, PropertyView cannot automatically retrospect Matlab structs, which is the reason that the built-in inspect function fails for such objects. In the general case, we may wish to present a PropertyTable using row-specific renderers and editors, when we have such an un-inspectable object (struct, cell array etc.), or even for a dynamically-created table of properties that is not stored in any single object.

Another limitation of PropertyView is that it immediately assigns updated property values to the inspected object, without allowing any sanity checks to be conducted on the new values, and without enabling the user to cancel property changes.

Last but not least, PropertyView does not enable users to easily modify the way that certain properties are presented. Boolean properties (true/false) always use a checkbox cell-renderer/editor, while we may wish to display a string (on/off) or combo-box instead.

In short, PropertyView is great unless you want to customize stuff, or do something slightly out-of-the-ordinary.

Enter propertiesGUI, the subject of today’s article.

The propertiesGUI utility

propertiesGUI, which can be downloaded from the Matlab File Exchange, presents a generic solution to this problem. propertiesGUI is based on Levente Hunyadi’s articles here back in 2010, and on his “Property grid” utility on the File Exchange.

propertiesGUI expects the following syntax:

[hPropsPane, parameters] = propertiesGUI(hParent, parameters)

where:

• hParent (input) is an optional handle of a parent GUI container (figure/uipanel/uitab) in which the properties table will appear. If missing or empty or 0, the table will be shown in a new modal dialog window; otherwise it will be embedded in the parent container.
• parameters (input) is an optional struct or object with data fields. The fields are processed separately to determine their corresponding cell renderer and editor. If parameters is not specified, then the global test_data will be used. If test_data is also empty, then a demo of several different data types will be used.
• hPropsPane (output) is the handle of the generated properties panel widget, which can be customized to display field descriptions, toolbar, etc.
• parameters (output) is the resulting (possibly-updated) parameters struct. Naturally, this is only relevant in case of a modal dialog.

When passing the properties in an input parameters struct, propertiesGUI automatically inspects each struct field and assigns a corresponding cell-editor with no description and a field label that reflects the field name. The properties are automatically set as modifiable (editable) and assigned a default callback function (propUpdatedCallback sub-function).

The global test_data variable is updated internally when the <OK> button is clicked. It is meant to enable easy data passing between the properties GUI and other application component. Using global vars is generally discouraged as bad programming, but it simplifies GUI component interaction and has performance advantages. In this specific case I decided that the benefits of simplicity outweigh the pedagogical aspects. After all, propertiesGUI is meant to illustrate how to customize/use a properties pane, not to teach GUI programming best practices. In your real-world application you should consider using other information-sharing mechanisms (getappdata or guidata, for example).

Usage demo

For an illustration of propertiesGUI, simply run it without any parameters to display its demo (see screenshot above):

propertiesGUI;

This creates (in the demoParameters sub-function) the following demo parameters struct:

      floating_point_property: 3.14159265358979
      signed_integer_property: 12
    unsigned_integer_property: 12
                flag_property: 1
                file_property: ''
              folder_property: 'C:\Yair'
                text_property: 'Sample text'
        fixed_choice_property: {'Yes'  'No'  'Maybe'}
     editable_choice_property: {'Yes'  'No'  'Maybe'  ''}
                date_property: [1x1 java.util.Date]
        another_date_property: 734895.957829132
                time_property: '22:59:16'
            password_property: '*****'
          IP_address_property: '10.20.30.40'
                  my_category: [1x1 struct]
               color_property: [0.4 0.5 0.6]
       another_color_property: [1x1 java.awt.Color]

A template for customization

propertiesGUI is meant to be used either as stand-alone, or as a template for customization. For example, we can attach a unique description to each property that will be shown in an internal sub-panel: see the customizePropertyPane and preparePropsList sub-functions.

We can have specific control over each property’s description, label, editability, cell-editor and callback function. See the preparePropsList sub-functions for some examples. Additional cell-editors/renderers can be added in the newProperty sub-function.

Specific control over the acceptable property values can be achieved by entering custom code into the checkProp sub-function. For example, checkProp already contains skeleton code to highlight missing mandatory field values in yellow, and non-editable properties in gray (see highlighted lines; the necessary modifications to make it work should be self-evident):

function isOk = checkProp(prop)
  isOk = true;
  oldWarn = warning('off','MATLAB:hg:JavaSetHGProperty');
  warning off MATLAB:hg:PossibleDeprecatedJavaSetHGProperty
  propName = get(prop, 'UserData');
  renderer = com.jidesoft.grid.CellRendererManager.getRenderer(get(prop,'Type'), get(prop,'EditorContext'));
  warning(oldWarn);
 
  mandatoryFields = {};  % TODO - add the mandatory field-names here  if any(strcmpi(propName, mandatoryFields)) && isempty(get(prop,'Value'))      propColor = java.awt.Color.yellow;      isOk = false;
  elseif ~prop.isEditable      %propColor = java.awt.Color.gray;      propColor = renderer.getBackground();
  else
      propColor = java.awt.Color.white;
  end
  renderer.setBackground(propColor);
end  % checkProp

So if we wish to check the validity of the input or user updates, we simply need to enter the relevant checks into checkProp.

Additional control over the appearance and functionality can be achieved via the hPropsPane reference that is returned by propertiesGUI.

As an example for such customizations, here is a variant of the properties pane, embedded within a uitab panel. Note the unique description of the properties, and the absence of the OK/Cancel buttons:

Properties panel Integrated within a GUI

TODO

Even as a template for customization, propertiesGUI is still far from perfect. Some potential items on my TODO list for this utility include:

• Fix the renderer/editor for numeric and cell arrays
• Enable more control over appearance and functionality via input parameters (for example: display or hide the OK/Cancel buttons)
• Add additional built-in cell editors/renderers: time, slider, point, rectangle (=position), font, …

Do you find propertiesGUI useful in your work? If so then please share your experience in a comment below. Suggestions for improvement of the utility will also be gladly accepted.

1. Advanced JIDE Property Grids JIDE property grids can use complex cell renderer and editor components and can signal property change events asynchronously to Matlab callbacks...
2. JIDE Property Grids The JIDE components pre-bundled in Matlab enable creating user-customized property grid tables...
3. Uitable customization report In last week’s report about uitable sorting, I offered a report that I have written which covers uitable customization in detail. Several people have asked for more details about the...
4. Date selection components The JIDE package, pre-bundled in Matlab, contains several GUI controls for selecting dates - this article explains how they can be used...

Closing a modal dialog may hang Matlab

The last time this happened was only two days ago, and by pure coincidence a reader’s comment yesterday happened on the same page where I explained the problem and solution. So I am taking this opportunity to explain the problem and the workaround, before my senility takes over and I forget again…

The symptom – Matlab hangs

The symptom is easily detectable: you’re happily working on your favorite Matlab GUI application, when suddenly some modal dialog window appears – perhaps a msgbox, a questdlg or an inputdlg. After clicking the <OK> button, Matlab seems to freeze, and any attempt to close the window or to stop the GUI fails. Forget about Ctrl-C – it simply hangs.

The only workaround is to open the operating system’s task manager and kill the Matlab process.

There are several odd things here: Firstly, the task manager shows that the Matlab process is not consuming any CPU. Secondly, the hang does not appear consistently – sometimes it does, other times not, making debugging extremely difficult. Moreover, when we debug the code, stepping through the source-code lines one at a time, everything seems to behave perfectly normal; it is only when running the program in continuous mode that the hang appears.

The solution – adding drawnow

The underlying reason for the hang appears to be that when we close the modal dialog window, it is sometimes not yet fully destroyed when we proceed with our program. It is my belief that this creates race-conditions which in some cases cause a deadlock between Matlab’s Main Thread and Java’s Event Dispatch Thread (EDT). Matlab-EDT synchronization problems are often time-dependent, which explains why the hang sometimes occurs and sometimes not.

While debugging EDT synchronization problems is notoriously difficult, the solution is often extremely simple — Just add a combination of drawnow and a short pause:

uiwait(msgbox(...));
drawnow; pause(0.05);  % this innocent line prevents the Matlab hang 
answer = questdlg(...);
drawnow; pause(0.05);  % this innocent line prevents the Matlab hangswitch answer
   ...

Adding drawnow and pause forces Matlab to flush EDT’s event queue, thereby ensuring synchronization before proceeding with the Matlab code. This is actually quite important when integrating Java GUI components in Matlab figures (especially before R2008b’s javaObjectEDT became available).

In the particular case of the Matlab dialog hang, drawnow is probably sufficient and pause is unnecessary. As noted by Bill York (MathWorks’ dev team leader who is responsible for such stuff) here, drawnow is similar but not exactly the same as pause. I often use both of them, since I discovered via trial-and-error that in the general case of EDT-related timing issues, different situations respond better to either of them, and I never remember which one to use to ensure proper rendering. So the drawnow/pause line appears very often in my code, quite possibly even in places where they are not strictly needed.

The actual needed value of the pause may vary, depending on GUI complexity and CPU power/load, but 0.05 seems to be large enough for most cases, while being small enough to be virtually unnoticeable by the user. In rare cases I needed to go as high as 0.1, but then again in some cases it is not needed at all.

1. Solving a MATLAB bug by subclassing Matlab's Image Processing Toolbox's impoint function contains an annoying bug that can be fixed using some undocumented properties....
2. Fixing a Java focus problem Java components added to Matlab GUIs do not participate in the standard focus cycle - this article explains how to fix this problem....
3. Spicing up Matlab uicontrol tooltips Matlab uicontrol tooltips can be spiced-up using HTML and CSS, including fonts, colors, tables and images...
4. Matlab and the Event Dispatch Thread (EDT) The Java Swing Event Dispatch Thread (EDT) is very important for Matlab GUI timings. This article explains the potential pitfalls and their avoidance using undocumented Matlab functionality....

All Matlab’s Handle Graphics (HG) property have default values. These values are used unless we specifically override the property value. For example, in R2012a, Matlab figure handles have 62 documented and 28 undocumented properties, all of which have some default value. When we create a new Matlab figure, we typically override only a handful of these properties.

How Matlab determines an HG property value

For example, we might override the figure’s Name, NumberTitle, Position, Visible, Colormap and perhaps a few other properties. All the others are either read-only (i.e., un-settable), or left at their default values. This is also true for all HG objects: axes, images, plots, patches, uicontrols, annotations, Java control containers etc.

Matlab makes a distinction between factory and default values. Users can modify the default values, but not the factory values, which makes sense. In essence, user-specified property values override the default values, which in turn override the factory values. I find that this makes the process of using default values quite intuitive. I like this factory/default design.

Matlab has a dedicated doc page explaining how we can use, set and reset the default property values.

Unfortunately, MathWorks has not seen fit to explain how to get the full list of current default defaults, nor how to get the factory values. Today’s article provides the missing manual pages and completes the picture.

Accessing default and factory property values

To access any specific property’s default value, we need to compose a fictitious property name from the string ‘Default’ or ‘Factory’, followed by the object’s type (‘Figure’, ‘Axes’, ‘Line’ etc., as reported by the object’s Type property), and lastly the property name. For example: DefaultFigureColor or FactoryAxesUnits. As with all HG properties, these names are case insensitive. These fictitious properties all belong to Matlab’s root (0) handle.

We can now get and set the values of any of these fictitious properties (naturally, factory properties cannot be set):

>> get(0,'FactoryFigureUnits')
ans =
pixels
 
>> get(0,'FactoryFigureColor')
ans =
     0     0     0
 
>> get(0,'DefaultFigureColor')
ans =
                       0.8                       0.8                       0.8
 
>> set(0,'DefaultFigureColor','y');  % new figures will now have a yellow background color...
 
>> set(0,'DefaultFigureColor','factory');  % resets the default value to the factory value [0,0,0]=black

Note that since the default and factory property names are fictitious (i.e., dynamic properties that are parsed on-the-fly), they do not appear when you get(0), getundoc(0) or even uiinspect(0).

My uiinspect utility reports the factory values in its property-details panel, along with additional meta-data such as whether the properties are settable, readable etc.

uiinspect's 'Other properties' meta-data table

Getting the full list of factory/default values

To get the long list of factory values, simply get a partial fictitious property name:

>> get(0,'factory')
ans = 
                         factoryFigureAlphamap: [1x64 double]
                       factoryFigureBusyAction: 'queue'
                    factoryFigureButtonDownFcn: ''
                         factoryFigureClipping: 'on'
                  factoryFigureCloseRequestFcn: 'closereq'
                            factoryFigureColor: [0 0 0]
                        (... 655 additional properties ...)
 
>> get(0,'default')
ans = 
               defaultTextColor: [0 0 0]
              defaultAxesXColor: [0 0 0]
              defaultAxesYColor: [0 0 0]
              defaultAxesZColor: [0 0 0]
          defaultPatchFaceColor: [0 0 0]
          defaultPatchEdgeColor: [0 0 0]
               defaultLineColor: [0 0 0]
    defaultFigureInvertHardcopy: 'on'
             defaultFigureColor: [0.8 0.8 0.8]
               defaultAxesColor: [1 1 1]
          defaultAxesColorOrder: [7x3 double]
          defaultFigureColormap: [64x3 double]
        defaultSurfaceEdgeColor: [0 0 0]
         defaultFigurePaperType: 'A4'
        defaultFigurePaperUnits: 'centimeters'
         defaultFigurePaperSize: [20.98404194812 29.67743169791]
          defaultFigurePosition: [200 200 560 420]
 
>> get(0,'DefaultAxes')
ans = 
        defaultAxesXColor: [0 0 0]
        defaultAxesYColor: [0 0 0]
        defaultAxesZColor: [0 0 0]
         defaultAxesColor: [1 1 1]
    defaultAxesColorOrder: [7x3 double]

We can see that the defaults list is much shorter than the factory list. There are very few actual default overrides of the factory values. In fact, if we try to get the default value of a property that was not overridden (e.g., DefaultFigureNumberTitle), Matlab is smart enough to return the factory value (in this case, FactoryFigureNumberTitle=’on’).

Hidden properties

Hidden (undocumented) properties are not shown by default, but you can always access them directly, and their list can also be seen if we set the root handle’s HideUndocumented property:

>> get(0,'FactoryAxesLooseInset')
ans =
         0.13         0.11         0.095         0.075
 
>> get(0,'DefaultPatchLineSmoothing')
ans =
off
 
>> set(0,'DefaultLineLineSmoothing','on');  % default appears to be 'off' for Windows, 'on' for Macs
 
>> set(0,'HideUndocumented','off')
>> allPropDefaults = get(0,'factory');
>> length(fieldnames(allPropDefaults))   % 661 documented + 277 undocumented properties
ans =
   938

Factory values internals

For those interested in some internals, the factory values are stored (and can be accessed) via the object’s UDD reference, or rather the schema.prop reference of the properties (additional information on UDD properties can be found here). For example:

>> get(0,'FactoryFigurePosition')
ans =
   100   100   660   520
 
>> hProp = findprop(handle(gcf),'pos')
hProp =
	schema.prop
 
>> get(hProp)
            Name: 'Position'
     Description: ''
        DataType: 'figurePositionType'
    FactoryValue: [100 100 660 520]
     AccessFlags: [1x1 struct]
         Visible: 'on'
     GetFunction: []
     SetFunction: []
 
>> hProp.FactoryValue
ans =
   100   100   660   520

Note that in this example, the FactoryFigurePosition value ([100 100 660 520]) is different than the DefaultFigurePosition value ([200 200 560 420]), which overrides it.

Conclusion

Setting default values enables easy setup of property values for all instances of an HG property in a Matlab session. It could be very tempting to add such setup to the startup.m file, so that such customizations automatically occur for all Matlab sessions. However, I strongly suggest against this: the moment you will try to run your application on any other computer or Matlab installation, you may find that your GUI/graphics look entirely different.

A much safer approach is to understand how these default values affect your application and then specifically set the desired property values in your m-code. This way, whatever the installation’s default values are, your application will always retain a consistent look-and-feel.

Have you found or ever used any interesting default or factory property value? If so, please share your experience in a comment.

1. Modifying default toolbar/menubar actions The default Matlab figure toolbar and menu actions can easily be modified using simple pure-Matlab code. This article explains how....
2. Plot LineSmoothing property LineSmoothing is a hidden and undocumented plot line property that creates anti-aliased (smooth unpixelized) lines in Matlab plots...
3. Axes LooseInset property Matlab plot axes have an undocumented LooseInset property that sets empty margins around the axes, and can be set to provide a tighter fit of the axes to their surroundings....
4. Setting class property types Matlab's class properties have a simple and effective mechanism for setting their type....

In one of my projects, I needed to present a radar (polar) plot. Such plots are usually drawn on a black background and I wanted all the plot controls to blend into this background.

Matlab figure having black toolbar background

For the plot itself I used a variation of Matlab’s buggy polar function, which I modified to enable proper dynamic resize / zoom / pan, bypass figure-renderer issues with patches and data-cursors, and other similar annoyances. Pretty standard stuff.

For the slider I’ve used a javax.swing.JSlider having a continuous-movement callback. Again, for readers of this blog this is nothing special:

[jSlider,hSlider] = javacomponent('javax.swing.JSlider',[0,0,.01,0.1],hFig);
set(hSlider, 'Units','norm','pos',[.15,0,.7,.05]);
set(jSlider, 'Background',java.awt.Color.black, ...
             'Value',0, 'Maximum',duration, ...
             'StateChangedCallback',{@cbSlider,hFig,axPlayback});

Setting the background color for all the GUI components to black was easy. But setting the toolbar’s background to black turned out to be a bit more interesting, and is the topic of this week’s article.

Standard Matlab figure toolbar - yuck!

The first step, naturally, is to get the toolbar’s handle:

hToolbar = findall(hFig,'tag','FigureToolBar');

In my case, I programmatically create the figure and use the default figure toolbar, whose tag value is always ‘FigureToolBar’. If I had used a custom toolbar, I would naturally use the corresponding tag (for example, if you create a custom toolbar using GUIDE, then the tag name will probably be ‘toolbar1′ or something similar).

Since I’m setting the figure programmatically, I need to manually remove several unuseful toolbar controls. I do this by directly accessing the toolbar control handles:

delete(findall(hToolbar,'tag','Plottools.PlottoolsOn'))
delete(findall(hToolbar,'tag','Plottools.PlottoolsOff'))
delete(findall(hToolbar,'tag','Annotation.InsertColorbar'))
delete(findall(hToolbar,'tag','DataManager.Linking'))
delete(findall(hToolbar,'tag','Standard.EditPlot'))

For setting the bgcolor, we get the toolbar’s underlying Java component, then sprinkle some Java magic power:

% ensure the toolbar is visible onscreen
drawnow;
 
% Get the underlying JToolBar component
jToolbar = get(get(hToolbar,'JavaContainer'),'ComponentPeer');
 
% Set the bgcolor to black
color = java.awt.Color.black;
jToolbar.setBackground(color);
jToolbar.getParent.getParent.setBackground(color);
 
% Remove the toolbar border, to blend into figure contents
jToolbar.setBorderPainted(false);
 
% Remove the separator line between toolbar and contents
jFrame = get(handle(hFig),'JavaFrame');
jFrame.showTopSeparator(false);

Unfortunately, this is not enough. The reason is that some of Matlab’s standard toolbar icons use non-opaque Java button controls (thereby showing the new black bgcolor), whereas other icons use opaque buttons, with a hard-coded gray background (I feel like spanking someone…). I’ve already touched upon this issue briefly a few years ago.

Matlab figure toolbar with black background, some opaque buttons

Luckily, all is not lost: we simply need to loop over all the JToolBar’s components and force them to be non-opaque with a black bgcolor. In cases where the component is compound (e.g., the Brush Data uisplittool), we need to set the bgcolor for all the sub-components:

jtbc = jToolbar.getComponents;
for idx=1:length(jtbc)
    jtbc(idx).setOpaque(false);
    jtbc(idx).setBackground(color);
    for childIdx = 1 : length(jtbc(idx).getComponents)
        jtbc(idx).getComponent(childIdx-1).setBackground(color);
    end
end

…finally ending up with the blended appearance that appears at the top of this article.

1. Customizing the standard figure toolbar, menubar The standard figure toolbar and menubar can easily be modified to include a list of recently-used files....
2. Figure toolbar components Matlab's toolbars can be customized using a combination of undocumented Matlab and Java hacks. This article describes how to access existing toolbar icons and how to add non-button toolbar components....
3. Figure toolbar customizations Matlab's toolbars can be customized using a combination of undocumented Matlab and Java hacks. This article describes how to customize the Matlab figure toolbar....
4. Javacomponent background color This article explains how to align Java component background color with a Matlab color....

I am extremely pleased to host guest blogger Christopher Albert. Chris and his colleagues created and maintain the enormously-interesting (IMHO) and complex Matclipse project, which connects Java’s popular Eclipse IDE with Matlab.

The open-source matclipse project is an interface between the Java integrated development environment (IDE) Eclipse and Matlab. It was developed at the Institute for Theoretical and Computational Physics (ITP) at Graz University of Technology (TU Graz) by a team that consisted of David Camhy, Winfried Kernbichler, Georg Huhs and Christopher Albert. Matclipse is part of a bigger endeavor, namely, the development of the teaching and learning environment MatlabTutor. It is released under the Eclipse Public License (full text).

The aim of the project is to facilitate the usage of Matlab directly from Eclipse under various operating systems (Linux, Windows, OS X). At the moment, it consists of a workbench, an Matlab editor, a Matlab console including a command window and a viewer for results, a Matlab command history and a Matlab workspace viewer.

The matclipse project is hosted on Google Code. Unfortunately, it does not contain a lot of external documentation, except a somewhat-outdated wiki, with installation instructions. A list of open issues is maintained here, where you can also report new issues. There is also a newsgroup forum/mailing-list with some activity.

Matlab Workbench

The Matlab workbench uses a tree structure for workspace. One can create Matlab projects there and, of course, all basic features from Eclipse are supported there. Projects can have a Matlab nature and files can be executed directly from the workbench without opening them in the editor.

The matclipse workbench within Eclipse (click for details)

Communication with Matlab

At the moment, the communication between Java and Matlab is based on the usage of pipes under Linux and OS X and on the usage of DCOM under Windows. In both cases it is necessary to have a local installation of Matlab. An extension to enable communication with a remote Matlab server is planned for the future but various issues (see below) have to resolved for a successful implementation of this feature.

In Linux as well as OS X there is no documented way of access to Matlab, to its memory and to its variables. For that reason, the most reliable way of communication was the usage of pipes. Since in this case, memory in Matlab and memory in Eclipse is strictly separated, any transfer of variables from Matlab to Eclipse needs time and doubles the amount of necessary memory. Therefore, this is not the best of all ways but without the help of MathWorks no other possibility is feasible. Under the operating system Windows, MathWorks provides DCOM as a possibility to control Matlab.

With pipes a remote access to Matlab on a remote Linux server is possible because pipes can be established through a ssh-tunnel. In addition, a remote file system from this server has to be mounted in Eclipse. A good candidate for this seems to be a ssh-based file system. This would enable the storage of programs directly from the editor on the server where Matlab could execute them.

Additional problems arises if graphical output is produced in Matlab or if Matlab tools like the help browser or the profile viewer has to be used. One way to handle this problem is the installation of an X11 server on the client which could display all additional windows from the server on the client through the X11 protocol in case of a correct setting of the display variable. Beside issues of security and speed, technically everything could run as on a local machine but the requirement is a reliable X11server used by Eclipse on the client side.

Another possibility to use a remote server is the conversion of Matlab figures to pictures (e.g. PNG) and to transfer those pictures through the pipe back to the client. The task of eclipse then would be to provide a viewer for this pictures. Of course, any interaction with the figures in a Matlab kind of way (zooming in and out, rotation, etc.) would not be possible. It would also require some overhead to detect what figures are new or have been modified to keep the time for transfer as low as possible. Also bookkeeping for figures has to be established to be able to remove deleted figures from the picture viewer. With the restrictions mentioned above this is possible. A usage of the help browser and profile viewer would not be possible in this case. Ideally, the user could have a choice between both methods.

Matlab Editor

Matclipse's Matlab editor

The current features are:

• Smart indentation and parenthesis matching.
• Detection of variables, operators, strings, commands and key words.
• Integration of the Matlab program mlint to mark errors and warnings directly in the editor.

Possible improvements for the future could be:

• Syntax highlighting with detection of all Matlab commands also from available toolboxes. The list of these commands has to be extracted from the Matlab help directory for a given version of Matlab. Commands from Matlab directly and from toolboxes can be shown in different colors.
• Direct link to Matlab help pages locally or situated at MathWorks.
• Evaluation of the whole file or of a selection of lines directly in Matlab.
• Export of a documented file in XML, Latex, PDF or HTML format. With syntax highlighting, indentation and links to Matlab help.
• Usage of a simple wiki-like markup language to display mathematical formulas, lists

Matlab Console

Matclipse's Matlab console

The current features of the command line are the following:

• Syntax highlighting is provided by the editor.
• Up- and down-scrolling in the command history.
• The Matlab command input had to be overwritten with a version which opens an input dialog box (inputdlg) in a modal Matlab window. There was no other way to enable the user to provide the input to Matlab. This is a clear disadvantage when using a remote server because this solution for sure requires a X11 server to display the modal window.
• The Matlab command pause without an additional time specified had to be overwritten. It is terminated now automatically after 2 seconds because there was no way to terminate the pause from the command line. In future one would have to think of an icon (or a shortcut) which would terminate the pause in similar way as CTRL-C is handled to stop execution of a command.
• CTRL-C does not work in the command line. An icon was provided in the console to stop execution of Matlab.
• Icons are provided to start and stop Matlab.
• Additional icons are available to bring figures in the foreground or to delete figures.
• There is a possibility to change the Matlab directory without a Matlab command.
• One can start the Matlab help browser from an icon.
• One can clear the results view from an icon (clc does not work).
• One can toggle Matlab console debug from an icon, to debug the XML communication between matclipse and Matlab. Note that this debug mode does not enable debugging the Matlab code itself – perhaps this will be added in the future.

Additional disadvantages and plans for the future are summarized here.

• When scrolling up or down the lines with %- should not be shown in the command line.
• When scrolling up or down one should be able to write some letters before starting to scroll. In this case one should get only a limited choice. E.g., if one writes plo one should only get command lines from history which start with these letters, like plot(x,y).
• This feature could be extended to allow for wildcards. E.g., if one writes *sin one should get only lines which contain sin at some position.
• Copy and Paste should be enabled with shortcuts everywhere. At the moment one needs the context menu for this.
• One should cycle round when reaching the end or the beginning of the command history.
• The Matlab command clc should be parsed and it should not be send to Matlab. Instead, the results view should be cleared.
• The Matlab command edit or edit(filename) should be parsed and it should not be send to Matlab. Instead, one should be placed in the correct editor window. If the filename does not exist, it should be created. If the Matlab extension *.m does not exist it should be added automatically.
• In addition to the icon for the Matlab help browser there should be an icon for the Matlab profile viewer.
• A debug mode should be enabled. This would require an integration of the editor into this task.
• At the moment, the workspace viewer is updated after each execution of the command line. This results in a time delay which gets bigger when there is a larger Matlab workspace. This update should only happen when the user switches to the tab with the workspace view. Otherwise, if the workspace view is not used, unnecessary time is wasted with the update.
• Sometimes there is the problem that Matlab is busy when the user tries to execute a script. This might be in connection with the use of mlint directly from the editor. The context menu Save and Run forces saving of the file which triggers mlint and then tries to execute the script. Maybe one has to wait until mlint is finished or it should be clearly shown to the user that the script could not be executed. Now this is sometimes confusing.

Matlab Command History

Matclipse's Matlab command history

In case a couple of lines are executed from the editor with execute selection, those lines a stored in one entry in the command history and can be recalled at once.

Additional features and fixes are recorded here:

• A selection of more than one line should be possible for execution, deleting and copying.
• Add delete old to the context menu to erase all entries from previous sessions.
• When there are more commands in one line, one should not show the \n, one should rather use a space instead for the display and keep it for execution. This is only cosmetics because it just looks ugly.
• Drag and drop to the command window is not really necessary because there exist faster ways with scrolling and the return key.
• The user should be able to collapse selections or everything belonging to some old session. Those things should be stored but not used when scrolling up or down in the command line.

Matlab Workspace Viewer

Matclipse's Matlab workspace browser

To avoid unnecessary transfer of data only a limited amount of data from a big array is requested from Matlab. The selection shows relevant information in a compressed form but never shows 100 x 100 array fully. Typically it shows a first block of data from the array accompanied by the last row and the last line to give a good and compact view on the data.

The exchange of information between Matlab and Eclipse is based on a XML description of the workspace generated within Matlab.

Additional features and fixes are recorded here:

• In addition to class, name and size one could show for numerical arrays the minimum and the maximum as well as the mean value to have a better overview. The appropriate Matlab routines are already modified to provide those information in the XML representation.
• To be able to view the whole content of a numeric data type one could add a context menu which then would transfer all data for this array for display in a style sheet like viewer.
• In the context menu (and with the delete button) one should be able to delete variables. This should be possible only on the first level. So specific content of cells and structures can not be deleted in such a way.
• When selecting one or more variables, it should be possible to edit a plot command which should then be send to Matlab. E.g., if you select the variables x and y then the context menu should open a small editor window with pre-filled commands like figure; plot(x,y). If the user does not like the choice then he can edit this field and finally the command is send to Matlab. There is no need to transfer the data to Eclipse.
• One could imagine a similar feature when selecting two numerical arrays for comparison. For this a Matlab routine could be provided to display the differences and similarities.

Multivariate analysis with Eigenvector

(the following is once again in my (Yair’s) own voice):

I would like to direct readers attention to Eigenvector Research (featured as a sidebar ad on this page). Eigenvector provides Matlab-based solutions for multivariate analysis (MVA) with a specific emphasis on chemical applications, i.e. chemometrics. Their products include the PLS_Toolbox for general MVA/chemometrics, and MIA_Toolbox for Multivariate Image Analysis. They also provide training and consulting services.

If you work with applications that produce large amounts of multivariate data, you may well find their products and services useful.

1. JMI – Java-to-Matlab Interface JMI enables calling Matlab functions from within Java. This article explains JMI's core functionality....
2. Matlab-Java interface using a static control The switchyard function design pattern can be very useful when setting Matlab callbacks to Java GUI controls. This article explains why and how....
3. Using Groovy in Matlab Groovy code can seamlessly be run from within Matlab. ...
4. Matlab and the Event Dispatch Thread (EDT) The Java Swing Event Dispatch Thread (EDT) is very important for Matlab GUI timings. This article explains the potential pitfalls and their avoidance using undocumented Matlab functionality....

Matlab’s Handle Graphics (HG) have been around for ages. Still, to this day it contains many hidden gems. Today I discuss HG’s Behavior property, which is a standard undocumented hidden property of all HG objects.

Behavior is not normally updated directly (although there is absolutely nothing to prevent this), but rather via the semi-documented built-in accessor functions hggetbehavior and hgaddbehavior. This manner of accessing Behavior is similar to its undocumented sibling property ApplicationData, which is accessed by the corresponding getappdata and setappdata functions.

Using HB behaviors

hggetbehavior with no input args displays a list of all pre-installed HG behaviors:

>> hggetbehavior
 
   Behavior Object Name         Target Handle
   --------------------------------------------
   'Plotedit'...................Any Graphics Object   
   'Print'......................Any Graphics Object   
   'Zoom'.......................Axes                  
   'Pan'........................Axes                  
   'Rotate3d'...................Axes                  
   'DataCursor'.................Axes and Axes Children
   'MCodeGeneration'............Axes and Axes Children
   'DataDescriptor'.............Axes and Axes Children
   'PlotTools'..................Any graphics object   
   'Linked'.....................Any graphics object   
   'Brush'......................Any graphics object

hggetbehavior can be passed a specific behavior name (or cell array of names), in which case it returns the relevant behavior object handle(s):

>> hBehavior = hggetbehavior(gca, 'Zoom')
hBehavior =
	graphics.zoombehavior
 
>> hBehavior = hggetbehavior(gca, {'Zoom', 'Pan'})
hBehavior =
	handle: 1-by-2

As the name indicates, the behavior object handle controls the behavior of the relevant action. For example, the behavior object for Zoom contains the following properties:

>> hBehavior = hggetbehavior(gca, 'Zoom');
>> get(hBehavior)
       Enable: 1        % settable: true/false
    Serialize: 1        % settable: true/false
         Name: 'Zoom'   % read-only
        Style: 'both'   % settable: 'horizontal', 'vertical' or 'both'

By setting the behavior’s properties, we can control whether the axes will have horizontal, vertical, 2D or no zooming enabled, regardless of whether or not the toolbar/menu-bar zoom item is selected:

hBehavior.Enable = false;         % or: set(hBehavior,'Enable',false)
hBehavior.Style  = 'horizontal';  % or: set(hBehavior,'Style','horizontal')

This mechanism is used internally by Matlab to disable zoom/pan/rotate3d (see %matlabroot%/toolbox/matlab/graphics/@graphics/@zoom/setAllowAxesZoom.m and similarly setAllowAxesPan, setAllowAxesRotate).

At this point, some readers may jump saying that we can already do this via the zoom object handle that is returned by the zoom function (where the Style property was renamed Motion, but never mind). However, I am just trying to show the general usage. Not all behaviors have similar documented customizable mechanisms. In fact, using behaviors we can control specific behaviors for separate HG handles in the same figure/axes.

For example, we can set a different callback function to different HG handles for displaying a plot data-tip (a.k.a. data cursor). I have explained in the past how to programmatically control data-tips, but doing so relies on the figure datacursormode, which is figure-wide. If we want to display different data-tips for different plot handles, we would need to add logic into our custom update function that would change the returned string based on the clicked handle. Using HG behavior we can achieve the same goal much easier:

% Use dataCursorLineFcn() for the line data-tip
bh = hggetbehavior(hLine,'DataCursor');
set(bh,'UpdateFcn',@dataCursorLineFcn);
 
% Use dataCursorAnnotationFcn() for the annotation data-tip
bh = hggetbehavior(hAnnotation,'DataCursor');
set(bh,'UpdateFcn',{@dataCursorAnnotationFcn,extraData});

Note: there is also the related semi-documented function hgbehaviorfactory, which is used internally by hggetbehavior and hgaddbehavior. I do not see any need for using hgbehaviorfactory directly, only hggetbehavior and hgaddbehavior.

Custom behaviors

The standard behavior objects are UDD schema objects (i.e., the old object-oriented mechanism in MATLAB). They are generally located in a separate folder beneath %matlabroot%/toolbox/matlab/graphics/@graphics/. For example, the Zoom behavior object is located in %matlabroot%/toolbox/matlab/graphics/@graphics/@zoombehavior/. These behavior object folders generally contain a schema.m file (that defines the behavior object class and its properties), and a dosupport.m function that returns a logical flag indicating whether or not the behavior is supported for the specified handle. These are pretty standard functions, here is an example:

% Zoom behavior's schema.m:
function schema
% Copyright 2003-2006 The MathWorks, Inc.
 
pk = findpackage('graphics');
cls = schema.class(pk,'zoombehavior');
 
p = schema.prop(cls,'Enable','bool');
p.FactoryValue = true;
 
p = schema.prop(cls,'Serialize','MATLAB array');
p.FactoryValue = true;
p.AccessFlags.Serialize = 'off';
 
p = schema.prop(cls,'Name','string');
p.AccessFlags.PublicSet = 'off';
p.AccessFlags.PublicGet = 'on';
p.FactoryValue = 'Zoom';
p.AccessFlags.Serialize = 'off';
 
% Enumeration Style Type
if (isempty(findtype('StyleChoice')))
    schema.EnumType('StyleChoice',{'horizontal','vertical','both'});
end
p = schema.prop(cls,'Style','StyleChoice');
p.FactoryValue = 'both';

% Zoom behavior's dosupport.m:
function [ret] = dosupport(~,hTarget)
% Copyright 2003-2009 The MathWorks, Inc.
 
% axes 
ret = ishghandle(hTarget,'axes');

All behaviors must define the Name property, and most behaviors also define the Serialize and Enable properties. In addition, different behaviors define other properties. For example, the DataCursor behavior defines the CreateNewDatatip flag and no less than 7 callbacks:

function schema
% Copyright 2003-2008 The MathWorks, Inc.
 
pk = findpackage('graphics');
cls = schema.class(pk,'datacursorbehavior');
 
p = schema.prop(cls,'Name','string');
p.AccessFlags.PublicSet = 'off';
p.AccessFlags.PublicGet = 'on';
p.FactoryValue = 'DataCursor';
 
schema.prop(cls,'StartDragFcn','MATLAB callback');
schema.prop(cls,'EndDragFcn','MATLAB callback');
schema.prop(cls,'UpdateFcn','MATLAB callback');
schema.prop(cls,'CreateFcn','MATLAB callback');
schema.prop(cls,'StartCreateFcn','MATLAB callback');
schema.prop(cls,'UpdateDataCursorFcn','MATLAB callback');
schema.prop(cls,'MoveDataCursorFcn','MATLAB callback');
 
p = schema.prop(cls,'CreateNewDatatip','bool');
p.FactoryValue = false;
p.Description = 'True will create a new datatip for every mouse click';
p = schema.prop(cls,'Enable','bool');
p.FactoryValue = true;
 
p = schema.prop(cls,'Serialize','MATLAB array');
p.FactoryValue = true;
p.AccessFlags.Serialize = 'off';

Why am I telling you all this? Because in addition to the standard behavior objects we can also specify custom behaviors to HG handles. All we need to do is mimic one of the standard behavior object classes in a user-defined class, and then use hgaddbehavior to add the behavior to an HG handle. Behaviors are differentiated by their Name property, so we can either use a new name for the new behavior, or override a standard behavior by reusing its name.

hgaddbehavior(hLine,myNewBehaviorObject)

If you wish the behavior to be serialized (saved) to disk when saving the figure, you should add the Serialize property to the class and set it to true, then use hgaddbehavior to add the behavior to the relevant HG handle. The Serialize property is searched-for by the hgsaveStructDbl function when saving figures (I described hgsaveStructDbl here). All the standard behaviors except DataDescriptor have the Serialize property (I don’t know why DataDescriptor doesn’t).

Just for the record, you can also use MCOS (not just UDD) class objects for the custom behavior, as mentioned by the internal comment within the hgbehaviorfactory function. Most standard behaviors use UDD schema classes; an example of an MCOS behavior is PlotEdit that is found at %matlabroot%/toolbox/matlab/graphics/+graphics/+internal/@PlotEditBehavor/PlotEditBehavor.m.

ishghandle‘s undocumented type input arg

Note that the Zoom behavior’s dosupport function uses an undocumented format of the built-in ishghandle function, namely accepting a second parameter that specifies a specific handle type, which presumably needs to correspond to the handle’s Type property:

ret = ishghandle(hTarget,'axes');

The hasbehavior function

Another semi-documented built-in function called hasbehavior is located right next to hggetbehavior and hgaddbehavior in the %matlabroot%/toolbox/matlab/graphics/ folder.

Despite its name, and the internal comments that specifically mention HG behaviors, this function is entirely independent of the HG behavior mechanism described above, and in fact makes use of the ApplicationData property rather than Behavior. I have no idea why this is so. It may be a design oversight or some half-baked attempt by a Mathworker apprentice to emulate the behavior mechanism. Even the function name is misleading: in fact, hasbehavior not only checks whether a handle has some “behavior” (in the 2-input args format) but also sets this flag (in the 3-input args format).

hasbehavior is used internally by the legend mechanism, to determine whether or not an HG object (line, scatter group, patch, annotation etc.) should be added to the legend. This can be very important for plot performance, since the legend would not need to be updated whenever these objects are modified in some manner:

hLines = plot(rand(3,3));
hasbehavior(hLines(1), 'legend', false);   % line will not be in legend
hasbehavior(hLines(2), 'legend', true);    % line will be in legend

(for anyone interested, the relevant code that checks this flag is located in %matlabroot%/toolbox/matlab/scribe/private/islegendable.m)

hasbehavior works by using a dedicated field in the handle’s ApplicationData struct with a logical flag value (true/false). The relevant field is called [name,'_hgbehavior'], where name is the name of the so-called “behavior”. In the example above, it creates a field called “legend_hgbehavior”.

Do you know of any neat uses for HG behaviors? If so, please post a comment below.

1. Undocumented scatter plot behavior The scatter plot function has an undocumented behavior when plotting more than 100 points: it returns a single unified patch object handle, rather than a patch handle for each specific...
2. Displaying hidden handle properties I present two ways of checking undocumented hidden properties in Matlab Handle Graphics (HG) handles...
3. Waterloo graphics Waterloo is an open-source library that can significantly improve Matlab GUI. ...
4. Waterloo graphics examples Some Matlab usage examples for the open-source Waterloo graphics package. ...

Continuing last week’s article on Matlab Handle Graphics’ (HG) undocumented Behavior functionality, today I describe another undocumented aspect of HG. Some of the low-level HG functions, such as line and path receive their input parameters in one of two formats:

HG line created using either of the input formats

• The regular fully-documented P-V pairs format:

  line('Marker','*', 'MarkerSize',8, 'MarkerEdgeColor','b', ...
       'Color','r', 'XData',1:5, 'YData',3:7)

• An undocumented struct-fields format:

  lineprops.Marker = '*';
  lineprops.MarkerSize = 15;
  lineprops.MarkerEdgeColor = 'b';
  lineprops.color = 'r';
  lineprops.xdata = 1:5;
  lineprops.ydata = 3:7;
   
  line(lineprops)

A sample usage of using the input struct technique can be seen (for example) in %matlabroot%/toolbox/matlab/graphics/@graphics/@datatip/datatip.m. I have also used it in my IB-Marlab application, which connects Matlab to Interactive Brokers. The idea is to make it easy for users to use whichever input format they feel more comfortable with: both P-V pairs or struct fields are supported.

Both input formats can be interchanged and mixed. We should just note that the latest (right-most) property is updated last and overrides any previous value of the same property. Here are some examples — can you guess the results of each of them?

line(lineprops, 'MarkerEdgeColor','g', 'MarkerSize',7)
line('MarkerEdgeColor','g', lineprops, 'MarkerSize',7)
line('MarkerEdgeColor','g', 'MarkerSize',7, lineprops)
line(1:4, 2:5, 'MarkerEdgeColor','g', 'MarkerSize',7, lineprops)
line(1:4, 2:5, 'MarkerEdgeColor','g', lineprops, 'MarkerSize',7)
line(1:4, 2:5, lineprops, 'MarkerEdgeColor','g', 'MarkerSize',7)

While the struct format is documented (although not widely-used) in the set function, it is not documented for line, path and other functions. Nor is it documented that we can mix the struct format with the P-V pairs format.

Note that not all HG functions support the struct input format. For example, line does, but plot does not. I do not understand the reason for this. It would seem reasonable for MathWorks to include this useful struct format for all their HG functions, and since set already supports it the extra effort would have been trivial. Maybe this will still happen in some future Matlab release…

1. ishghandle’s undocumented input parameter The built-in function ishghandle accepts a second input argument with the expected handle type....
2. getundoc – get undocumented object properties getundoc is a very simple utility that displays the hidden (undocumented) properties of a specified handle object....
3. Undocumented scatter plot behavior The scatter plot function has an undocumented behavior when plotting more than 100 points: it returns a single unified patch object handle, rather than a patch handle for each specific...
4. Undocumented scatter plot jitter Matlab's scatter plot can automatically jitter data to enable better visualization of distribution density. ...

Three years ago, I wrote an article about modifying Matlab’s look-and-feel (L&F, or LnF), using Java’s built-in support for replaceable LnFs. To date, that article has 27 comments by 13 different commenters (plus 20 responses by me), making it the second most active article on this website. I decided to follow up on that article with a demo that shows the effects that different L&Fs have on GUI controls, and a demonstration of the JTattoo library of professional L&Fs.

Today’s article and the demo are based on original work by Karthik Ponudurai, who has written a guest article here about an interesting technique to interface a Java GUI to a Matlab application, a couple of years ago.

Demo of using different Look-and-Feels in Matlab GUI

JTattoo is a 3rd-party open-source library. The purpose of including it in the demo, in addition to its natural use as a professional set of L&Fs, is to demonstrate how easy it is to integrate 3rd-party L&Fs in Matlab. In the demo I use the current latest available JTattoo library (1.6.7), but you can always download the latest version and replace the JTattoo.jar file. JTattoo contains a large set of different L&Fs that can be used independently (screenshots). The nice thing about L&Fs is that since all Matlab GUI is based on Java Swing, the new L&Fs automatically affect Matlab controls just like native Java ones.

The demo can be downloaded from the Matlab File Exchange. After downloading, unzip it into any folder on your Matlab path and run JTattooDemo.m.

A Matlab figure is displayed with two panels, one containing Matlab uicontrols (within a simple uiflowcontainer) and the other containing Java components.

Two main menus are available: System enables selecting the standard Swing L&Fs that are installed on your system (this varies a bit between platforms and Matlab releases); JTattoo enables selecting one of the JTattoo L&Fs. Once the user selects any of the L&F menu items, the entire figure is updated. This is done by calling javax.swing.SwingUtilities.updateComponentTreeUI() on the figure’s Java Frame‘s content pane. Both the Matlab and the Java controls within the figure are automatically updated by this Swing function to reflect the newly-selected L&F. Care is taken to update the L&F on the EDT, to prevent racing-condition issues.

It should be noted that the demo resets the L&F after updating the figure, otherwise any new figure or window would open using the newly-selected L&F. This is done in the updateInterface function as follows:

function updateInterface( lookandfeel )
 
    % Preserve the original L&F, before updating
    originalLnF = javax.swing.UIManager.getLookAndFeel;
 
    % Update the L&F in the demo figure as requested
    ... (all the existing code within the function)
 
    % Restore the original L&F for any new figure/window
    javax.swing.UIManager.setLookAndFeel(originalLnF);
 
end  % updateInterface

Note that after changing the L&Fs several times, some L&F properties night get “mixed-up” causing odd-looking L&Fs. The simplest solution in this case is to restart Matlab…

Matlab open training day (Israel) - click for details

Readers in Israel are invited to attend a free training seminar that I will present on advanced Matlab topics in Herzliya, on Thursday April 4, 2013. The seminar is free, but requires registration. Additional details here. I will speak in Hebrew, but the presentation will be in English and I will be happy to answer questions in English.

1. Modifying Matlab’s Look-and-Feel Matlab's entire Look-and-Feel (PLAF, or L&F) can be modified at the control or application level - this article shows how...
2. Real-time trading system demo A real-time Matlab-based end-to-end trading system demo is presented ...
3. Additional uicontrol tooltip hacks Matlab's uicontrol tooltips have several limitations that can be overcome using the control's underlying Java object....
4. Inactive Control Tooltips & Event Chaining Inactive Matlab uicontrols cannot normally display their tooltips. This article shows how to do this with a combination of undocumented Matlab and Java hacks....

Two weeks ago I wrote about Matlab Handle Graphics (HG) undocumented acceptance of structs as input parameter to some plotting functions. Continuing in a related matter, today I expose an undocumented input parameter for HG’s ishghandle function.

ishghandle is a fully-documented built-in Matlab function that returns a logical flag indicating whether the specified input is a valid HG handle or not. The input could be an invalid handle for several reasons: Perhaps it is not an HG handle in the first place, or perhaps the corresponding HG object has since been deleted (for example, its parent figure window was closed). The usage is very simple:

if ishghandle(myHandle)
    set(myHandle, ...);
end

But what if myHandle could be any of several possible HG types (e.g., plot line, patch or 3D surface) and our logic depends on the type? Of course, we could always add an extra condition to the logic:

if ishghandle(myHandle) && strcmp(get(h,'type'),'surface')
%or: if ishghandle(myHandle) && isa(handle(h),'surface')
    set(myHandle, ...);
end

There’s an undocumented alternative: we can simply specify the expected type as a second input argument to ishghandle. This is in fact the simplest and fastest alternative:

if ishghandle(myHandle,'surface')
    set(myHandle, ...);
end

If the second input argument is specified, then ishghandle will only return true if both (1) the input handle is a valid HG handle and (2) if its type is the specified second input argument. ishghandle‘s second input arg is case in-sensitive, so we can enter either ‘surface’, ‘Surface’ or ‘SURFACE’ (unlike the isa function, which is case-sensitive for some unknown reason).

I’ve first came across this ishghandle feature in a post by Sam Roberts. While the second-arg syntax is not very common in the Matlab code corpus, it does appear in several dozen places, for example:

• %matlabroot%/toolbox/matlab/graph2d/plotedit.m: if any(ishghandle(varargin{1}, 'figure'))
• %matlabroot%/toolbox/matlab/graph2d/subplot.m: if ~ishghandle(h, 'axes')
• %matlabroot%/toolbox/matlab/graph3d/camlight.m: ...any(ishghandle(args{1},'light'))
• %matlabroot%/toolbox/matlab/graph3d/colordef.m: ...~ishghandle(arg1, 'figure') && ~ishghandle(arg1, 'root')
• %matlabroot%/toolbox/matlab/graph3d/objbounds.m: if (ishghandle(h(i),'surface') || ishghandle(h(i),'line') || ishghandle(h(i),'image'))
• %matlabroot%/toolbox/matlab/graph3d/objbounds.m: elseif ishghandle(h(i),'patch')
• %matlabroot%/toolbox/matlab/graphics/@graphics/@pan/schema.m: ... ~ishghandle(valueProposed,'uicontextmenu')
• %matlabroot%/toolbox/matlab/scribe/selectobject.m: if ~ishghandle(obj,'axes') && ~ishghandle(obj,'figure') && ~ishghandle(obj,'uipanel')
• %matlabroot%/toolbox/matlab/uitools/@uitools/@uimode/createuimode.m: ... ishghandle(figureState.LastObject,'uicontrol') || ishghandle(figureState.LastObject,'uitable')
• %matlabroot%/toolbox/matlab/uitools/private/javacomponentundoc_helper.m: ishghandle(hParent, 'uicontainer') || ishghandle(hParent, 'uiflowcontainer') || ishghandle(hParent, 'uigridcontainer') ... ishghandle(hParent, 'uitoolbar') ... ishghandle(hParent, 'uisplittool') || ishghandle(hParent, 'uitogglesplittool') ... ishghandle(hParent, 'hgjavacomponent')
• %matlabroot%/toolbox/matlab/uitools/winmenu.m: ... ~ishghandle(h,'uimenu')
• %matlabroot%/toolbox/matlab/guide/guidemfile.m: mychildren(ishghandle(mychildren,'hggroup')) = [];

Did you happen to notice any other stock Matlab function with undocumented input arguments? If so, please write a short comment about it below.

1. HG’s undocumented parameters interface Some HG functions also accept inputs parameters in a struct fields rather than the normal P-V pairs format. ...
2. Undocumented scatter plot jitter Matlab's scatter plot can automatically jitter data to enable better visualization of distribution density. ...
3. Undocumented scatter plot behavior The scatter plot function has an undocumented behavior when plotting more than 100 points: it returns a single unified patch object handle, rather than a patch handle for each specific...
4. getundoc – get undocumented object properties getundoc is a very simple utility that displays the hidden (undocumented) properties of a specified handle object....

Mlint, Matlab’s static code-analysis parser, was written by Stephen Johnson (the original developer of the enormously successful lint parser for C/C++ back in 1977), when he was lured by MathWorks in 2002 to develop a similar tool for Matlab. Since its development (in R14 I believe), and especially since its incorporation in Matlab’s Editor in R2006a (Matlab 7.2), mlint has become a very important tool for reporting potential problems in m-files.

Unfortunately, to this day (R2013a), there is no documented manner of programmatically separating mlint warnings and errors, nor for accessing any of the multitude of features that are readily available in mlint. Naturally, there is (and has always been) an undocumented back door.

From its earliest beginnings, mlint has relied on C code (presumably modeled after lint). For many years mlint relied on a mex file (%matlabroot%/toolbox/matlab/codetools/mlintmex.mex*), which is basically just a wrapper for mlint.dll where the core algorithm resides. In recent releases, mlintmex, just like many other core mex files, was ported into a core Matlab library (libmwbuiltins.dll on Windows). However, the name and interface of the mlintmex function have remained unchanged over the years. Wrapping the core mlintmex function is the mlint m-function (%matlabroot%/toolbox/matlab/codetools/mlint.m) that calls mlintmex internally. In R2011b (Matlab 7.13) its official function name has changed to checkcode, although this was never documented in the release notes for some reason. However, using mlint still works even today. Wrapping all that is the mlintrpt function, which calls mlint/checkcode internally.

The core function mlintmex returns a long string with embedded newlines to separate the messages. For example:

>> str = mlintmex('perfTest.m')
str = 
L 3 (C 1): The value assigned to variable 'A' might be unused.
L 4 (C 1): The value assigned to variable 'B' might be unused.
L 5 (C 1-3): Variable 'ops', apparently a structure, is changed but the value seems to be unused.
L 12 (C 9): This statement (and possibly following ones) cannot be reached.
L 53 (C 19-25): The function 'subFunc' might be unused.
L 53 (C 27-35): Input argument 'iteration' might be unused. If this is OK, consider replacing it by ~.

We can parse this long string ourselves, but there is no need since mlint/checkcode do this for us, returning a struct array:

>> results = mlint('perfTest.m')
results = 
6x1 struct array with fields:
    message
    line
    column
    fix
>> results(5)
ans = 
    message: 'The function 'subFunc' might be unused.'
       line: 53
     column: [19 25]
        fix: 0

As can be seen, the message severity (warning/error) does not appear. This severity is obviously available since it is integrated in the Editor and the Code Analyzer report – orange for warnings, red for errors.

In one of my projects I needed to enable the user to dynamically create executable Matlab code that would then be run interactively. This enabled users to create dynamic data analyses functions without actually needing to know Matlab or to code all the nuts-and-bolts of a regular Matlab function. For this I needed to display warnings and errors-on-the-fly (the dynamic cell tooltips used a custom table cell-renderer). Here’s the end-result:

Analysis definition panel

Dynamic analysis alert tooltips

My solution was to use mlintmex, as follows:

% Get the relevant message strings
errMsgs = mlintmex('-m2', srcFileName);
allMsgs = mlintmex('-m0', srcFileName);
 
% Parse the strings to find newline characters
numErrors = length(strfind(regexprep(errMsgs,'\*\*\*.*',''),char(10)));
numAllMsg = length(strfind(regexprep(allMsgs,'\*\*\*.*',''),char(10)));
numWarns = numAllMsg - numErrors;

(and from the messages themselves [errMsgs,allMsgs] I extracted the actual error/warning location)

Alternatively, I could have used mlint directly, as I have recently explained:

% Note that mlint returns struct arrays, so the following are all structs, not strings
errMsgs = mlint('-m2',srcFileNames); % m2 = errors only
m1Msgs  = mlint('-m1',srcFileNames); % m1 = errors and severe warnings only
allMsgs = mlint('-m0',srcFileNames); % m0 = all errors and warnings

The original information about mlintmex and the undocumented -m0/m1/m2 options came from Urs (us) Schwartz, whose contributions are an endless source of such gems. Urs also provided a list of other undocumented mlint options:

'-all'
'-allmsg'
'-amb'
'-body'
'-callops'
'-calls'
'-com'
'-cyc'
% '-db'       % == -set + -ud + -tab
'-dty'
'-edit'
'-en'         % messages in English
'-id'
'-ja'         % messages in Japanese
'-lex'
'-m0'         % + other opt
'-m1'         % + other opt
'-m2'         % + other opt
'-m3'         % + other opt
'-mess'
'-msg'
'-notok'
'-pf'
'-set'
'-spmd'
'-stmt'
'-tab'
'-tmtree'
'-tmw'        % not valid anymore
'-toks'
'-tree'
'-ty'
'-ud'
'-yacc'       % ONLY: !mlint FILE -yacc -...

to which were added in recent years ‘-eml’, ‘-codegen’ etc. – see the checkcode doc page. Also note that not all Matlab releases support all options. For example, ‘-tmw’ is ignored in R2013a, returning the same data as ‘-all’ plus a warning about the ignored option.

Urs prepared a short utility called doli that accepts an m-file name and returns a struct whose fields are the respective outputs of mlint for each of the corresponding options:

>> results = doli('perfTest.m')
MLINT >   C:\Yair\Books\MATLAB Performance Tuning\Code\perfTest.m
OPTION>   -all       6
OPTION>   -allmsg    501
OPTION>   -amb       17
OPTION>   -body      6
OPTION>   -callops   15
OPTION>   -calls     15
OPTION>   -com       6
OPTION>   -cyc       8
OPTION>   -dty       162
OPTION>   -edit      92
OPTION>   -en        7
...

Some of these options are used by Urs’ farg and fdep utilities. Their usage of mlint rather than direct m-code parsing, is part of the reason that these functions are so lightningly fast.

For example, we can use the ‘-calls’ options to parse an m-file and get the names, type, and code location of its contained functions (explanation):

>> mlint('-calls','perfTest.m')
M0 1 10 perfTest
E0 51 3 perfTest
U1 3 5 randi
U1 4 5 num2cell
U1 4 14 randn
U1 6 1 whos
U1 7 1 tic
U1 7 6 save
U1 7 45 toc
U1 9 6 savefast
S0 53 19 subFunc
E0 60 3 subFunc
U1 55 8 isempty
U1 56 20 load
U1 57 29 sin

With so many useful features, I really cannot understand why they were never exposed to the public in a documented manner. After all, they have remained pretty-much unchanged for many years and can provide enormous benefits for developers of unit-tests and interactive analysis frameworks (as I have shown above).

As a side-note, in R2010a (Matlab 7.10), mlint was renamed “Code Analyzer”, but this was really just a name change – its core functionality has changed little in the past decade. Some might argue that new checks were added and the Editor interface has improved by allowing auto-fixes and message suppression. But for a tool that is over a decade old (much more, if you count lint’s development), I contend that these are not much. Don’t get me wrong – I have the utmost respect for Steve. Serious unix C/C++ development relies on his lint and yacc tools on a regular basis. I think they show astonishing ingenuity and intelligence. It’s just that I had expected more after a decade of mlint development (I bet it’s not due to Steve suddenly losing the touch).

Addendum: A little birdie tells me that Steve left MathWorks a few years ago, which does explain things… I apologize to Steve for any misguided snide on my part. As I said above, I have nothing but the utmost respect for his work. The question of why MathWorks left his mlint work hanging without serious continuation remains open.

1. Running VB code in Matlab Matlab does not natively enable running VB code, but a nice trick enables us to do just that...
2. Undocumented scatter plot behavior The scatter plot function has an undocumented behavior when plotting more than 100 points: it returns a single unified patch object handle, rather than a patch handle for each specific...
3. More undocumented timing features There are several undocumented ways in Matlab to get CPU and clock data...
4. tic / toc – undocumented option Matlab's built-in tic/toc functions have an undocumented option enabling multiple nested clockings...

When I wrote about the undocumented aspects of classdef properties half a year ago, I did not expect to learn of another major undocumented aspect in Matlab’s class-object system. last month I discovered the additional undocumented classdef Description and DetailedDescription attributes, and updated that article accordingly. But a few days ago I was pleasantly surprised to read Amro’s comment about an entirely new and undocumented aspect of Matlab MCOS classes.

Amro is a top contributor on StackOverflow, where he frequently answers questions before I even get any subscription notification about them… His answers are generally characterized by a deep technical understanding of Matlab, and I’ve learned quite a lot from him in the past few years. This time was no different.

In a nutshell, Amro found an undocumented way to specify a class object property’s type, in such a way that would prevent accidental setting to an incompatible value. For example, if we have a class with properties Width and Height, we probably want to restrict their possible values to numbers, to prevent setting a string or struct value.

In UDD classes, we can do this easily by setting the property’s DataType meta-property. An easy way to do this is by setting the second argument of the schema.prop function. A detailed explanation was provided here.

We can still do this today, since UDD classes are still supported, side-by-side with the newer MCOS classes. Unfortunately, MCOS does not provide a documented way of specifying the property type as in UDD.

One simple way to prevent unintentional MCOS property updates is to override the property’s set method. In fact, this was the solution of Jonas, another StackOverflow heavyweight:

classdef myClass
   properties
      myProperty = uint16(23); %# specify default value using correct type
   end
   methods
      function obj = set.myProperty(obj,val)
         if ~isa(val,'uint16')
            error('only uint16 values allowed')
         end
         %# assign the value
         obj.myProperty = val;
      end
   end
end

But it turns out that there’s a much cleaner and simpler solution, provided by Amro:

classdef Packet
    properties
        HeaderLength@uint16
        PayloadLength@uint16 = uint16(0);
        PacketType@char
    end
end

As Amro notes, if you try to set a property with the wrong type, you get an error:

>> p = Packet;
>> p.PacketType = 'tcp';  % ok
>> p.HeaderLength = 100;  % not ok - should be a uint16
While setting the 'HeaderLength' property of Packet:
Value must be 'uint16'.

This syntax apparently supports all primitive types (char, int32, double, struct, cell etc.), in addition to any user-defined ones (just use any class name).

Note that setting the type as above seems to override any set method that may have been specified for the property.

Amro came across this syntax in an internal class in R2013a (toolboxdir(‘matlab’)/graphics/+graphics/+internal/+figfile/@FigFile/FigFile.m), but it also worked in R2012a, and probably older versions as well…

I admit there’s not much original work here by me – it’s basically all by Amro (and Donn Shull for the UDD part). But I thought it’s important enough to bring to the attention of the community.

I love to discover such undocumented treasures by digging in Matlab’s function. If you ever discover other such buried treasures, please do let me know by email or a comment.

1. Creating a simple UDD class This article explains how to create and test custom UDD packages, classes and objects...
2. Types of undocumented Matlab aspects This article lists the different types of undocumented/unsupported/hidden aspects in Matlab...
3. Extending a Java class with UDD Java classes can easily be extended in Matlab, using pure Matlab code. ...
4. Setting desktop tab completions The Matlab desktop's Command-Window tab-completion can be customized for user-defined functions...

Readers of this blog are probably aware by now that I am currently writing my second book, MATLAB Performance Tuning (expected publication date: early 2014, CRC Press). During my work on this book, I encounter many surprising aspects of Matlab performance. In many cases these aspects are not un-documented per-se, but are certainly not well known in the Matlab community. So taking some artistic liberty coupled with some influence over this blog’s owner, I’ll mention some of these interesting discoveries here, even if they are not strictly-speaking undocumented.

Today’s post is about the well-known fwrite function, which is used to write binary data to file. In many cases, using fwrite provides the fastest alternative to saving data files (save(…,’-v6′) coming a close second). This function is in fact so low-level, and is used so often, that some readers may be surprised that its default speed can be improved. Today’s article applies equally to the fprintf function, which is used to save data in text format.

Apparently, there are things to be learned even with such standard low-level functions; there’s a deep moral here I guess.

Flushing and buffering

Unlike C/C++’s implementation, Matlab’s fprintf and fwrite automatically flush the output buffer whenever they are called, even when '\n' is not present in the output stream. This is not mentioned outright in the main documentation, but is stated loud and clear in the official technical support solution 1-PV371.

The only exception to this rule is when the file was fopen‘ed with the 'W' or 'A' specifiers (which for some inexplicable reason is NOT mentioned in the technical solution!), or when outputting to the MATLAB’s Command Window (more precisely, to STDOUT (fid=1) and STDERR (fid=2)). Writing data without buffering in this manner severely degrades I/O performance:

data = randi(250,1e6,1);  % 1M integer values between 1-250
 
% Standard unbuffered writing - slow
fid = fopen('test.dat', 'wb');
tic, for idx = 1:length(data), fwrite(fid,data(idx)); end, toc
fclose(fid);
  => Elapsed time is 14.006194 seconds.
 
% Buffered writing – x4 faster
fid = fopen('test.dat', 'Wb');
tic, for idx = 1:length(data), fwrite(fid,data(idx)); end, toc
fclose(fid);
  => Elapsed time is 3.471557 seconds.

If I were in a generous mood, I could say that we could infer this information from fopen‘s doc page, where it mentions using the 'W' and 'A' permission specifiers to prevent automatic flushing, although it qualifies this with the very misleading statement that these specifiers are “Used with tape drives“. So first of all, who ever uses tape drives with Matlab nowadays?! Secondly, these specifiers are very useful for regular buffered I/O on standard disks and other I/O interfaces. I really think this was a poor choice of words. At the very least some extra clarification about these specifiers could be added.

It was also (IMHO) a poor design choice by MathWorks in the first place to break consistency with the C/C++ implementation for 'w' and associate the functionality to 'W' (and similarly, 'a' vs. 'A'). C’s fopen was in widespread usage for a decade before Matlab was invented, so there is really no excuse, certainly when Matlab’s fopen was so clearly modeled after the C implementation. It would have been more reasonable (again – IMHO) to preserve consistency of 'w' and 'a' for a default of buffered I/O (which is faster!), while providing the non-buffered functionality in 'W' and 'A'.

The vast majority of Matlab users fopen their files using 'w' and not 'W'. Even Matlab’s own documentation always uses 'w' and not 'W'. So coupled with the poorly-worded qualification about the tape drives, and the unintuitive inconsistency with C’s implementation, Matlab users could well be excused for not taking advantage of this feature.

Chunking I/O

The idea of buffering, and the reason behind the speedup above, is that I/O is faster when writing full pages (typically 4KB, but this changes on different platforms) and when bunched together to remove the disk access time between adjacent writes. This idea can be extended by preparing the entire file data in memory, and then using a single fwrite to write everything at once:

fid = fopen('test.dat', 'wb');
tic, fwrite(fid,data); toc
fclose(fid);
  => Elapsed time is 0.014025 seconds.

In fact, assembling the entire data in memory, within a long numeric or char array, and then using a single fwrite to save this array to file, is almost as fast as we can expect to get (example). Further improvement lies in optimizing the array assembly (which is CPU and memory-intensive) rather than the I/O itself.

In this example, the I/O was so fast (14mS) that it makes sense to write everything at once. But for enormously large data files and slower disks (I use a local SSD; network hard disks are way slower), writing the entire file’s data in this manner might take long minutes. In such cases, it is advisable to deliberately break up the data into smaller chunks, and fwrite them separately in a loop, all the time providing feedback to the user about the I/O’s progress. This could help improve the operation’s perceived performance. Here’s a bare-bones example:

h = waitbar(0, 'Saving data...', 'Name','Saving data...');
cN = 100;  % number of steps/chunks
 
% Divide the data into chunks (last chunk is smaller than the rest)
dN = length(data);
dataIdx = [1 : round(dN/cN) : dN, dN+1];  % cN+1 chunk location indexes
 
% Save the data
fid = fopen('test.dat', 'Wb');
for chunkIdx = 0 : cN-1
   % Update the progress bar
   fraction = chunkIdx/cN;
   msg = sprintf('Saving data... (%d%% done)', round(100*fraction));
   waitbar(fraction, h, msg);
 
   % Save the next data chunk
   chunkData = data(dataIdx(chunkIdx+1) : dataIdx(chunkIdx+2)-1);
   fwrite(fid,chunkData);
end
 
fclose(fid);
close(h);

Of course, rather than using a plain-ol’ waitbar window, we could integrate a progress bar directly into our GUI. Using my statusbar utility is one way to do it, but there are of course many other possible ways to dynamically present progress:

Dynamically updating progress using the statusbar utility

Note: the entire article above applies equally well to fprintf in addition to fwrite. Storing and loading data in binary format (using fwrite/fread) is often faster than text format (using fprintf/fscanf/textscan), so we should generally use text format only if the file needs to be human-readable for any reason.

Do you know of any other trick to store data efficiently? If so, please share it in a comment.

Next week: some surprising performance aspects of Matlab’s save function.

1. Improving save performance There are many different ways of improving Matlab's standard save function performance. ...
2. Plot performance Undocumented inner plot mechanisms can be used to significantly improved plotting performance...
3. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...
4. Preallocation performance Preallocation is a standard Matlab speedup technique. Still, it has several undocumented aspects. ...

I’d like to break the regular flow of weekly articles, to announce that I will visit New York City on May 21-24, to speak at the Matlab Computational Finance Conference (May 23).

I will be very happy to meet you and discuss how I could bring value to your needs, either financial-oriented or not. We can meet at the conference, or elsewhere in NYC on other days: Tuesday (May 21), Wednesday (May 22) or Friday (May 24). If you would like to schedule a meeting, please email me.

Matlab Computational Finance Conference

The Matlab Computational Finance conference is an event that takes place more-or-less annually, in different locations. In 2012 it was held in London; in 2011 it was virtual (online). This year it will be held in New York City on May 23, at the Marriott Marquis hotel, located in midtown (1535 Broadway, next to Times Square).

The free, full-day conference showcases real-world examples of how financial organizations use Matlab to develop risk, trading, investment management, and insurance applications. Highlights include:

• Customer presentations from leading industry practitioners in trading, risk, valuation, portfolio analysis, insurance, and regulatory compliance
• Master class tutorials delivered by senior MathWorks engineers to enhance your understanding of Matlab features and functionality
• Panel discussion with senior Matlab developers
• An opportunity to network, exchange ideas, and discuss challenges and successes with industry peers and Matlab experts
• And naturally, my presentation, slated for 16:50 (I suggest rechecking the agenda close to the conference, to ensure that the time-slot hasn’t changed)

Here is the abstract of my presentation:

MATLAB has traditionally been used for analyzing data off-line, presenting analytic recommendations that were then acted-upon manually. However, MATLAB supports direct interface with data feeds and online brokers, as well as the ability to present sophisticated graphics and user interfaces – all in real time.

This presentation will demonstrate an end-to-end demo trading system in MATLAB, highlighting MATLAB’s potential as a platform of choice. Interactive Brokers will be used to demonstrate live market data feed and account/portfolio input, as well as for sending trading orders to the market. The system’s user interface (GUI) showcases MATLAB’s hidden visualization and interactivity potential, for tracking order executions and charting financial time-series in real-time. Some best-practices for improving real-time performance shall also be discussed.

If you are interested in any aspect of computational finance, you will surely find some interesting presentations at this conference. So if you’re in town, I urge you to attend (hey – it’s free and I hear there’s drinks…). You can register here. If you can’t attend in person, you will be able to see recordings of the proceedings on the conference website some time later.

1. 2012 perspective & plans for 2013 2012 has seen a continued steady growth in website content and readership. This post takes an overview of past achievements and future plans. ...
2. Consulting If you have reached this page, you are probably aware by now of my reputation in the Matlab community. After over 20 years of software development in a variety of...
3. IB-Matlab IB-Matlab is a commercial application that connects Matlab with Interactive-Brokers (IB). IB-Matlab provides a Matlab wrapper interface to IB’s API Java connector. This enables quantitative traders and algo traders to...
4. 2010 perspective & plans for 2011 2009 and 2010 have seen steady growth in readership of this website. This post takes an overview of past achievements and future plans....

Two weeks ago I posted an article about improving fwrite‘s performance. fwrite is normally used to store binary data in some custom pre-defined format. But we often don’t need or want to use such low-level functions. Matlab’s built-in save function is an easy and very convenient way to store data in both binary and text formats. This data can later be loaded back into Matlab using the load function. Today’s article will show little-known tricks of improving save‘s performance.

MAT is Matlab’s default data format for the save function. This format is publicly available and adaptors are available for other programming languages (C, C#, Java). Matlab 6 and earlier did not employ automatic data compression; Matlab versions 7.0 (R14) through 7.2 (R2006a) use GZIP compression; Matlab 7.3 (R2006b) and newer can use an HDF5-variant format, which apparently also uses GZIP (level-3) compression, although MathWorks might have done better to pay the license cost of employing SZIP (thanks to Malcolm Lidierth for the clarification). Note that Matlab’s 7.3 format is not a pure HDF5 file, but rather a HDF5 variant that uses an undocumented internal format.

The following table summarizes the available options for saving data using the save function:

HDF5 uses a generic format to store data of any conceivable type, and has a non-significant storage overhead in order to describe the file’s contents. Moreover, Matlab’s HDF5 implementation does not by default compress non-numeric data (struct and cell arrays). For this reason, HDF5 files are typically larger and slower than non-HDF5 MAT files, especially if the data contains cell arrays or structs. This holds true for both pure-HDF files (saved via the hdf and hdf5 set of functions, for HDF4 and HDF5 formats respectively), and v7.3-format MAT files.

Perhaps for this reason the default preference is for save to use –v7, even on new releases that support –v7.3. This preference can be changed in Matlab’s Preferences/General window (or we could always specify the –v7/-v7.3 switch directly when using save):

Matlab's preferences for saving binary data

Over the years, MathWorks has fixed several inefficiencies when reading HDF5 files (ref1, ref2). Some of these fixes include patches for older releases, and readers are advised to download and install the appropriate patches if you do not use the latest Matlab release (currently R2013a). There are still a couple of open bugs regarding HDF5 performance and compression that may affect save.

One might think that due to the generic descriptive file header and the increased I/O, as well as the open bug, the -v7.3 (HDF5) format would always be slower than –v7 (MAT) format in save and load. This is indeed often the case, but not always:

A = randi(20,1000,1200,40,'int32');  % 48M int32s => 184 MB
B = randn(500,1000,20);              % 80M doubles => 78 MB
ops.algo = 'test';                   % non-numeric
 
tic, save('test1.mat','-v7','ops','A','B'); toc
% => Elapsed time is 11.940455 seconds.   % file size: 114 MB
 
tic, save('test2.mat','-v7.3','ops','A','B'); toc
% => Elapsed time is 6.963135 seconds.    % file size: 116 MB

In this case, the HDF5 format was much faster than MAT, offsetting the benefits of the MAT’s reduced I/O. This example shows that we need to check our specific application’s data files on a case-by-case basis. For some files –v7 may be better; for others –v7.3 is best. The widely-accepted conventional wisdom of only using the new –v7.3 format for enormous (>2GB) files is inappropriate. In general, if the data contains many non-numeric elements, the resulting –v7.3 HDF5 file is much larger and slower than the –v7 MAT file, while if the data is mostly numeric, then –v7.3 would be faster and comparable in size.

Surprisingly, we can often sacrifice compression to (paradoxically) achieve better performance, for both save and load, at the expense of much larger file size. This is done by saving the numeric data in uncompressed HDF5 format, using the savefast utility on the Matlab File Exchange, using the same syntax as save:

tic, savefast('test3.mat','ops','A','B'); toc
% => Elapsed time is 3.164903 seconds.   % file size: 259 MB

Even better performance, and similar or somewhat lower file size, can be achieved by using save’s uncompressed format –v6. The –v6 format is consistently faster than both –v7 and –v7.3, at the expense of a larger file size. However, save –v6 cannot save Unicode and class objects, and is limited to <2GB file sizes.

Another lesson here is that depending on the relative size of the numeric and non-numeric data being saved, different data format may be advisable. As the application evolves and the saved data’s size and mixture change, we might need to revisit the format decision. Here is a summary on one specific computer, using the numeric variable A (184 MB) above, together with a cell-array of varying size:

B = num2cell(randn(1,dataSize));  % dataSize = 1e3, 1e4, 1e5, 1e6

As noted, and as can be clearly seen in the table, compression is not enabled for non-numeric data in the save –v7.3 (HDF5) option and savefast. However, we can implement our own save variant that does compress, by using low-level HDF5 primitives in m-code or mex c-code.

A general conclusion that can be drawn from all this is that in the specific case of save, the additional time for compression is often NOT offset by the reduced I/O. So the general rule is to use –v6 whenever possible.

Although save –v6 does not compress its data, it does store data in a more compact manner than Matlab memory. So, while our test set’s cell array held 114 MB of Matlab memory, on disk it was only stored within 55 MB (= 244 MB – 189 MB).

The performance of saving non-numeric data can be dramatically improved (and the file size reduced correspondingly) by manually serializing the data into a series of uint8 bytes that can easily be saved very compactly. When loading the files, we would simply deserialize the loaded data. I recommend using Christian Kothe’s excellent Fast serialize/deserialize utility on Matlab’s File Exchange. The huge gain in performance and file size when using serialized data is absolutely amazing (esp. for data types that -v6 cannot save), for all the save alternatives:

B = num2cell(randn(1,1e6));  % 1M cell array, 114 MB in Matlab memory
B_ser = hlp_serialize(B);

Serializing data in this manner enables save –v6 to be used even for Unicode and class objects (that would otherwise require –v7), as well as huge data (that would otherwise require >2GB files, and usage of –v7.3). One user has reported that the run-time for saving a 2.5GB cell-array of structs was reduced from hours to a single minute using serialization (despite the fact that he was using a non-optimized serialization, not Christian’s faster utility). In addition to the performance benefits, saving class objects in this manner avoids a memory leak bug that occurs when saving objects to MAT files on Matlab releases R2011b-R2012b (7.13-8.0).

When the data is purely numeric, we could use hdf5write or h5create + h5write, in addition to save and savefast (see related). Note that hdf5write will be phased out in a future Matlab release; MathWorks advises to use h5create + h5write instead. Here are the corresponding results for 184 MB of numeric data on a standard 5400 RPM hard disk and an SSD:

As noted, Matlab’s HDF5 implementation is generally suboptimal. Better performance can be achieved by using the low-level HDF5 functions, rather than the high-level hdfwrite, hdf5read functions.

In addition to HDF5, Matlab also supports the HDF4 standard, using a separate set of built-in hdf functions. Despite their common name and origin, HDF4 and HDF5 are incompatible; use different data formats; and employ different designs, APIs and Matlab access functions. HDF4 is generally much slower than HDF5.

While save’s –v7.3 format is significantly slower than the alternatives for storing entire data elements, one specific case in which –v7.3 should indeed be considered is when we need to update or load just a small part of the data, on R2011b or newer. This could potentially save a lot of I/O, especially for large MAT files where only a small part is updated or loaded.

1. Improving fwrite performance Standard file writing performance can be improved in Matlab in surprising ways. ...
2. MLintFailureFiles or: Why can’t I save my m-file?! Sometimes Matlab gets into a state where it cannot use a valid m-file. This article explains what can be done. ...
3. datestr performance Caching is a simple and very effective means to improve code performance, as demonstrated for the datestr function....
4. Plot performance Undocumented inner plot mechanisms can be used to significantly improved plotting performance...

Exactly three years ago, I posted information (here and here) about Matlab’s upcoming new graphics engine, so-called HG2 (Handle Graphics version 2). At the time, I was sure that HG2 was just around the corner. But three years and six releases have passed, Matlab 7 turned into Matlab 8, and HG1 is still in use. I decided that it was time to revisit the latest state of HG2, as reflected in the latest release, R2013a (Matlab 8.1).

In the past few years, development of HG2 has apparently progressed to a stage where most of the kinks were ironed out. The latest HG2 appears to be quite stable, and in my experience most GUI/graphics utilities run as-is, without any need for tweaking. This is good news, which leads me to think that HG2 will be released soon. It is possible that this could happen as early as the upcoming release (R2013b, 8.2) but I have a gut feeling that it will be in R2014a. I also have a gut feeling that MathWorks will name that release 9.0 rather than 8.3, in keeping with its arch-rival Mathematica.

HG2 has improved grid lines, plot anti-aliasing and customizable everything (more on this below). Here’s a simple plot line as it appears in both HG1 and HG2:

hFig = figure('pos',[100,100,300,250]);
x = -10:0.1:10;
y = 1e7*sin(x)./x; 
hLine = plot(x,y);
box off; grid on;
title('HG2 plot');

Same plot in HG1 and HG2

We can see that MathWorks has invested heavily in improving usability. The graphics are now much more visually appealing than before. A lot of thought has gone into small details such as the plot colors and the axes gray shades. The changes are small when taken separately, but the overall gestalt is striking. HG2 will definitely justify my license maintenance cost.

Highly customizable

Matlab in HG2 mode acts and behaves pretty much as you would expect. There are no visible changes to the Desktop or the graphics interface. The major difference is that all graphics handles, whether interactive controls (figure, uicontrols, uitables, etc.) or graph elements (axes, lines, patches, etc.) are instances of class objects (e.g., matlab.ui.Figure or matlab.graphics.chart.primitive.Line) rather than numeric values. This makes it easy to issue commands such as:

hFig.Color = 'w';
 
hAxes.Title.Color = 'm';  % default: [0,0,0] = black
hAxes.YRuler.SecondaryLabel.String = 'millions';  % original: 'x10^{6}'
hAxes.YRuler.SecondaryLabel.FontAngle = 'italic';  % default: 'normal'
hAxes.YRuler.Axle.LineStyle = 'dotted';  % default: 'solid'
hAxes.YRuler.Axle.ColorData = uint8([0,100,0,255])';  %=dark green; default: [51 51 51 255], corresponding to [0.2 0.2 0.2 1]
hAxes.YBaseline.Color = 'b';  % default: [0.2 0.2 0.2]
hAxes.YBaseline.Visible = 'on';  % default: 'off'
hAxes.XRuler.Axle.Visible = 'off';  % default: 'on'
 
hLine.Color = [1,0,0];  %='red'

rather than using the corresponding set(…) or get(…) functions, which are still supported for backward compatibility.

Customized HG2 plot

Notice how much more customizable HG2 is compared to HG1. I am pretty excited from the huge number of additional possible customizations in HG2 compared to HG1. It is real a pity that many of these customizations rely on hidden/undocumented properties (see below). Hopefully this will change when HG2 is officially released.

Some observations

Here are a few observations that I collected on the latest HG2, as reflected in R2013a:

1. Java is still supported (hurray!). The warnings about the figure’s JavaFrame property becoming deprecated have fortunately not been fulfilled (hopefully never). All the Java-based GUI tricks shown on this blog and in my book still work, excluding some minor things here and there which are due to inter-release changes rather than to the new HG2 engine.
2. In order to access the top-level Java Frame of a figure window, we now need to use javaFrame.fHG2Client rather than javaFrame.fHG1Client. The relevant code should now look something like this, in order to be fully-compatible with older Matlab releases:

   jFrame = get(handle(hFig), 'JavaFrame');
   try
       % This works up to R2011a
       jFrame.fFigureClient.setClientDockable(true);
   catch
       try
           % This works from R2008b and up, up to HG2
           jFrame.fHG1Client.setClientDockable(true);
       catch
           % This works in HG2
           jFrame.fHG2Client.setClientDockable(true);
       end
   end

3. Anti-aliasing of plot elements (a.k.a. line -smoothing) is now ‘on’ by default (double hurray!). Apparently, MathWorks solved the problems with the existing undocumented LineSmoothing property. Still, for some unknown reason, LineSmoothing remains a hidden/undocumented property. Note that for some objects the property name is different. For example, the axes title (which is a text object of class matlab.graphics.primitive.Text) has a new property called Smoothing that controls anti-aliasing (unlike LineSmoothing, Smoothing appears to be an un-hidden fully-documented property).
4. Many new properties have been added to graphic objects, that enable customization of different aspects. For example, we can customize the axes grid-lines, containing box and exponent labels in ways that were impossible in HG1 (triple hurray!). Note that many of these new properties are hidden/undocumented (why the hell for???), so we need a utility such as my uiinspect or getundoc to detect them. Some of the useful new axes properties include *Ruler, *Baseline, *GridHandle, BoxFrame and BackDrop (I showed an example usage of *Ruler and *Baseline above). I have absolutely no idea why these so-useful properties are kept hidden, it simply makes no sense.
5. Some existing HG1 properties are missing. For example, the UserData property is no longer available for some Java objects (this is a real pity — I depended on it for many functionalities, such as storing node-specific data in uitree/JTree nodes). Similarly, axes no longer have *LimInclude properties (this actually makes sense – these properties are still available in plot lines, where they actually have a use).
6. Some existing HG1 properties now issue a warning, although they still work. For example:

   >> hAxes.DrawMode = 'fast';
   Warning: The DrawMode property will be removed in a future release.
   (Type "warning off MATLAB:hg:willberemoved" to suppress this warning.) 
    
   >> hLine.LineSmoothing
   Warning: The LineSmoothing property will be removed in a future release.
   (Type "warning off MATLAB:hg:willberemoved" to suppress this warning.) 
   ans =
   on

7. There is an open bug on R2012b and R2013a whereby the clf function does not delete javacomponent objects. This bug does not affect HG2, where clf works properly.
8. Some GUI components are being placed a pixel or two sideways in HG2 compared to HG1. This has no visual importance except in very rare cases, but it does affect my findjobj utility, which relies on the component’s position to find its underlying Java object.
9. The default axes and labels color has changed from black to gray ([0.2 0.2 0.2]). Grid lines now use an even lighter gray shade. Visually I think that this is a great change, since it focuses the viewer’s focus on the plot contents rather than the external decorations.
10. The default axes plot color order has changed. The standard plot color is no longer blue (as it was for ages in Matlab), but a bluish tint; the second color is no longer red but light green; the third color is reddish rather than dark green, etc.:

    % HG1
    >> get(0,'defaultAxesColorOrder')
    ans =
                0            0            1
                0          0.5            0
                1            0            0
                0         0.75         0.75
             0.75            0         0.75
             0.75         0.75            0
             0.25         0.25         0.25
     
    %HG2
    >> get(0,'defaultAxesColorOrder')
    ans =
         0.070588      0.40784      0.70196
          0.92941      0.14118      0.14902
          0.60784       0.7451      0.23922
          0.48235      0.17647       0.4549
                1      0.78039            0
          0.30196       0.7451      0.93333
          0.82353       0.4549            0

11. HG2 axes no longer forget the previous plot color (unless we used hold all) — in HG2 color cycling is on by default. Note that this causes some visual discrepancies between HG1 and HG2 in plots that use hold on and have multiple plot lines: In HG1 they were all blue; in HG2 the first is bluish, the second is greenish, then reddish etc.
12. GUIDE is still the same-ol’ GUIDE (sigh!). The figure toolbar and menubar have likewise not been upgraded, as far as I could tell.
13. HG2 performance appears to be generally slower than HG1. Hopefully this will improve by the time HG2 is released, since performance has been one of HG1′s drawbacks all along. In my tests, most GUI/graphic aspects ran only slightly slower in HG2, except for 2D plots that were significantly slower. This is corroborated by running bench: on my computer, HG1 yields 0.4 for 2D and 0.22 for 3D; in HG2 the performance is 2.1 for 2D and 0.18 for 3D. Looks like the 2D performance still needs some work…

Testing HG2

As noted in my original article, we can start Matlab in HG2 mode by simply adding the startup (command-line) switch -hgVersion 2 to the Matlab command (note the space between the -hgVersion and the “2″). For example, in Windows, all you need to do is to copy your Matlab shortcut sideways (so that you will always have the standard HG1 version available), then right-click the shortcut, select “Properties”, then add -hgVersion 2 to the Target field. You will probably want to also add the “HG2″ descriptor to the shortcut name, in the “General” tab:

Matlab startup switch for HG2

If you have any Matlab application that relies on GUI or graphics, I urge you to test it on the new HG2 system. It’s trivially simple and your application should work exactly the same, or better. If you do detect some incompatibility, please post a comment or shoot me an email. In due course I expect that MathWorks will open an official channel for this, but in the meantime I’ll be sure to pass the information to the relevant person.

Do take a moment for testing HG2 – we can all contribute to ensure that when HG2 does come out it will be perfect. It’s in our interest.

NYC visit

If you happen to be in New York next week, I urge you to attend the MATLAB Computational Conference on Thursday May 23 (free registration; my presentation is scheduled for 4:50pm). I would be happy to meet you to discuss how I could bring value to your needs, either financial-oriented or not. We could meet at the conference, or anywhere in New York on Wednesday May 22 or Friday May 24.

1. Plot LimInclude properties The plot objects' XLimInclude, YLimInclude, ZLimInclude, ALimInclude and CLimInclude properties are an important feature, that has both functional and performance implications....
2. FIG files format FIG files are actually MAT files in disguise. This article explains how this can be useful in Matlab applications....
3. Panel-level uicontrols Matlab's uipanel contains a hidden handle to the title label, which can be modified into a checkbox or radio-button control...
4. Multi-column (grid) legend This article explains how to use undocumented axes listeners for implementing multi-column plot legends...

Displaying images in Matlab figures has always been sort-of awkward. We need to read the image file into a Matlab matrix (possibly with accompanying colormap), then create an invisible axis in our figure window, and finally display the image data within this axis, typically with the imagesc function. In practice, this is not as difficult as it may seem, and can be done in 3 lines of code (imread(); axes(); imagesc();). To display images in uicontrols, we need to set their CData property with the image data. Unfortunately, this is only supported for static (non-animated) images.

But what if we want to display an animated image? There are several image formats that enable animation, GIF being the most well-known. Animated GIFs are used ubiquitously, for anything from logos to simple visualizations. I often use them myself in my utilities and this blog, to illustrate functionality or usage. Creating animated GIFs is so easy — I often use Picasion, but there are numerous alternatives.

So let’s assume that I have the animated GIF featured here:

animated GIF

This example serves as the logo in a demo live trading application that I’ll present tomorrow at the MATLAB Computational Finance Conference.

To display this image in a Matlab GUI, we could use a mini-browser control that we can easily integrate within the figure. But an even simpler and less resource-intensive solution is to integrate an HTML panel:

hFig = figure('Color','w'); 
je = javax.swing.JEditorPane('text/html', '<html><img src="file:/C:\Yair\animated.gif"/></html>');
[hj, hc] =  javacomponent(je,[],hFig);
set(hc, 'pos', [10,10,70,70])

Animated GIF image integrated in Matlab figure

Notes:

1. Although the GIF image is 64×64 pixels, we set the containing JEditorPane‘s size to 70×70, to account for the border and a small margin. You can obviously play with the size.
2. The image source was specified using the file:/ protocol, as I explained here.

If our animated GIF uses transparency, we should set the container’s background to the same color as the figure, as I explained here:

bgcolor = num2cell(get(hFig,'Color'));
hj.setBackground(java.awt.Color(bgcolor{:}));

Animated transparent GIF image integrated in Matlab figure

For anyone wondering, we could just as easily have used a simple JLabel rather than a JEditorPane (see here). The only difference is that we would need to also take care of the background:

hFig = figure('Color','w');
je = javax.swing.JLabel('<html><img src="file:/C:\Yair\animated.gif"/></html>');
je.setBackground(java.awt.Color(1,1,1));  %=white
[hj, hc] =  javacomponent(je,[],hFig);
set(hc, 'pos', [10,10,70,70])

We can similarly display animated GIFs on any uicontrol, due to the inherent nature of uicontrols that they accept HTML contents. So, we can place our animated GIF in buttons, drop-downs, listboxes etc. It’s not as straight-forward in drop-downs/listboxes as it is in buttons, but then again not many people need animated images in drop-downs or listboxes as they do in buttons. For push/toggle buttons, checkboxes and radio-buttons, displaying animated GIFs is trivially simple:

h1 = uicontrol('style','push',  'pos', [10,10,70,70], 'String','<html><img src="file:/C:\Yair\animated.gif"/></html>');
h2 = uicontrol('style','check', 'pos', [90,10,70,70], 'String','<html><img src="file:/C:\Yair\animated.gif"/></html>');
h3 = uicontrol('style','radio', 'pos',[170,10,70,70], 'String','<html><img src="file:/C:\Yair\animated.gif"/></html>');

Animated transparent GIF image integrated in Matlab uicontrols

The standard documented manner of displaying animated GIFs in Matlab is to imread all image frames from the GIF file, then start a timer that will periodically replace the axes image (or uicontrol CData property) with the next image frame in an endless loop. This is relatively tedious to set up, results in noticeable flicker and is relatively CPU intensive. I believe that the undocumented mechanism I’ve explained above is simpler to code and lighter on graphic resources.

Today’s article was a tribute to two old friends, whom I never met in person so far: a client who has recently asked me about this issue, and a member of a Graphics team who said he liked my related posts. I hope to meet both of them at the conference tomorrow. If you are interested in any aspect of computational finance, you will surely find some interesting presentations at this conference. So if you’re in town, I urge you to attend (free registration; my presentation is scheduled for 4:50pm). Otherwise, we could meet anywhere in New York tomorrow (Friday May 24), to discuss how I could bring value to your needs, either financial-oriented or not.

1. Displaying hidden handle properties I present two ways of checking undocumented hidden properties in Matlab Handle Graphics (HG) handles...
2. Spicing up Matlab uicontrol tooltips Matlab uicontrol tooltips can be spiced-up using HTML and CSS, including fonts, colors, tables and images...
3. Images in Matlab uicontrols & labels Images can be added to Matlab controls and labels in a variety of manners, documented and undocumented. ...
4. Multi-line uitable column headers Matlab uitables can present long column headers in multiple lines, for improved readability. ...

Last week I gave a presentation at the MATLAB Computational Finance Conference in New York. The room was packed-full with close to 200 professionals in the finance industry. The energy and feedback were tremendous, it was a great experience. The presentation (PDF format) is provided here.

I presented a demo application that showed how Matlab can be used to create a full end-to-end trading system, highlighting Matlab’s potential as a platform of choice. I used Interactive Brokers to demonstrate live market data feed and account/portfolio input, as well as for sending trading orders to the market, via the IB-Matlab connector:

Real-time trading system demo in Matlab (click for details)

The demo system’s user interface showcased the hidden visualization and interactivity potential of MATLAB for tracking order executions and charting financial time-series in real time. The undocumented features used in the demo include:

• Animated GIF logo image (rotating Dollar sign)
• Customized data table
• Log box with rich HTML contents: icons and color-coded messages
• Draggable (resizable) panel borders (note that this feature does not work in the new HG2 – I hope it will be fixed by the time that HG2 is released)
• Maximized figure window

The demo source code is provided here (tradingDemo.m and supporting files). Note that this is provided as-is, free of charge but without any warranty or support. You would naturally need IB-Matlab and an Interactive Brokers account to run it.

If you came to the conference, I want to thank you for being an excellent audience. I hope we have a chance to work together on your projects. Shoot me an email if you’d like my help in any consulting, training or development work.

1. JTattoo look-and-feel demo A demo GUI that shows the effects of using different look-and-feels, including the JTatoo library, is presented. ...
2. Setting system tray icons System-tray icons can be programmatically set and controlled from within Matlab, using new functionality available since R2007b....
3. Setting system tray popup messages System-tray icons and messages can be programmatically set and controlled from within Matlab, using new functionality available since R2007b....
4. Changing system preferences programmatically Matlab user/system preferences can be changed programmatically, from within your Matlab application or from the Matlab desktop command prompt. This post details how....