Gaffer Tape and Matlab... Its all you need

Non-blocking Output Socket

2010-06-19T21:12:00.009+08:00

My MATLAB TCP/IP example has been quite popular, and I myself use it at least once a week on various applications. However, one of the main limitations when compared to our C++ style output sockets was the limitation of only 1 client per socket.

What was worse, is that I knew it COULD be done. I had just not found the time to write it! Cue a spare Saturday!

Non-blocking Output Socket @ MATLAB Central

The Problem

As per my previous server.m examples, you begin by constructing a Java ServerSocket.

>> server_socket = ServerSocket(output_port); 
>> server_socket.setSoTimeout(1000);

This attempts to bind the ServerSocket to the specified port. HOWEVER, it is not listening for incoming connections! To do this, you must tell the ServerSocket to accept().

>> output_socket = server_socket.accept();

This is a BLOCKING call. That is, the code will wait for either the timeout (set above) to occur, or a client to connect.

For a lot of what I do, the output socket exists mainly as a debugging / monitoring interface. So I don't even care if noone is listening. So what we need is something that manages the connection of clients and the distribution of any written data to each of them.

Enter OutputSocket and ListeningThread.

OutputSocket

OutputSocket provides a very simple interface:

//--------------------------------------------------------------------- 
// Description : Create an OutputSocket on the specified port 
//--------------------------------------------------------------------- 
public OutputSocket(int port) throws IOException

//--------------------------------------------------------------------- 
// Description : Write the supplied byte array to all connected clients 
//--------------------------------------------------------------------- 
public void write(byte[] data)  

//--------------------------------------------------------------------- 
// Description : Close the OutputSocket. This closes all client 
//               connections and stops listening for new connections. 
//--------------------------------------------------------------------- 
public void close()

ListeningThread

The OutputSocket utilises ListeningThread to manage the accepting of new connections. ListeningThread implements the Runnable interface so can be started in its own thread (vital!).

//-------------------------------------------------------------------------
// Description : Helper class that listens for new TCP/IP client
//               connections on a ServerSocket
//
// Parameters  : server_socket          - ServerSocket to listen on
//               connection_list        - List of connected client
//                                        Sockets to update
//               connection_stream_list - List of connected client
//                                        DataOutputStreams to update
//---------------------------------------------------------------------
public ListeningThread(ServerSocket server_socket,
                   List         connection_list,                    
                   List         connection_stream_list)

//--------------------------------------------------------------------- 
// Description : Called on Thread.start()
 //--------------------------------------------------------------------- 
public void run()

//---------------------------------------------------------------------
// Description : Stop the thread
//--------------------------------------------------------------------- 
public void stop()

Interaction

The OutputSocket constructor creates the ServerSocket as before and stores it as a member variable. It also creates 2 empty lists, one for the Socket client connections and one for the associated DataOutputStreams used to write to them.

It supplies a reference to these to the ListeningThread on construction and then starts it running in its own Thread. This is the important part. OutputSocket and ListeningThread both share a reference to the DataOutputStream list, so as ListeningThread creates connections and output streams, these are pushed onto the client list.

When OutputSocket::write() is called, it writes the data across all output streams (be there 0 or 100!)

When you have finished with the OutputSocket, a call to close() will stop the ListeningThread from running and close all client connections.

Usage from within MATLAB

Place the .class files in a suitable location (I've stuck them in my current working directory).

>> % Add the directory containing the .class files to your Java path
>> javaaddpath(pwd)
>> % Create the OutputSocket
>> output_socket = OutputSocket(1234)

output_socket =

OutputSocket@1c68b20

>> % Write some data to the output socket - text will do for now
>> for i = 1:100
output_socket.write(int8(sprintf('This is line %03d\r\n', i)));
pause(1);
end
>> % cleanup the socket - close connections
>> output_socket.close()
>> clear output_socket % no longer needed / useful

As you can see, multiple connections are supported. They can connect/disconnect at any time. Even if there are no connections, a call to OutputSocket::write() will be successful.

**Update**

I've included in the revised submission to MATLABCentral a wrapper MATLAB class that helps manage the lifetime of the Java object.

This is VERY useful because if you happen to clear the Java object (ie clear output_socket) before calling .close() the Thread and ServerSocket etc remain in memory (and stay connected!) until MATLAB is closed!

Suggested usage of MatlabOutputSocket below.

>> javaaddpath(pwd); % get the Java class
>> output_socket = MatlabOutputSocket(1234);
for i = 1:100
output_socket.write(int8(sprintf('Line %03d\r\n', i)));
pause(1);
end
>> output_socket.close()
>> clear output_socket

I've included the Java source file in the MATLABCentral submission, so feel free to have a look at the source code, have a fiddle, recompile.

To do this you will need to install the Java Development Kit (JDK).

To compile the classes, navigate to the directory containing the .java file and type:

>> javac OutputSocket.java

NOTE: You will have to repeat the call to javaaddpath() for the new files to be seen by MATLAB.

TCP/IP Socket Communications in MATLAB - Part 2

2009-09-07T14:00:00.005+08:00

So my little blog post about TCP/IP Socket Communications in MATLAB was rather popular despite being a very simple example.

One of the main limitations that people found when trying to utilise the server/client scripts for their own applications, was that it was incredibly inefficient at shifting large volumes of data around. This is thanks to the following lines:

message = zeros(1, bytes_available, 'uint8');
for i = 1:bytes_available
    message(i) = d_input_stream.readByte;
end

For each individual byte within the message, there was the overhead of a function call.

I mentioned in comments on MATLAB Central that I made a Java class that bypassed this overhead and allowed for more efficient transfer. Being in a caring sharing mood (and getting sick of emailing it to people all the time!) I thought I would upload it also.

The is available on the Mathworks File Exchange: TCP/IP Socket Comms Example using Java Class

Compiled Java?

The client/server scripts are essentially identical to their previous iterations.

The server still uses a ServerSocket and the resulting DataOutputStream to which we write data.

The client side uses a Socket to connect to the specified host and port which provides us an InputStream which we wrap in a DataInputStream to read data from. However, instead of using the interface of DataInputStream directly, we hand off to another function (the new DataReader class) to perform the read.

The code for the example client is outlined below. I have bolded the changes. (The server is unchanged).

client.m

% CLIENT connect to a server and read a message
%
% Usage - message = client(host, port, number_of_retries)
function message = client(host, port, number_of_retries)

    import java.net.Socket
    import java.io.*

    if (nargin < number_of_retries =" 20;" style="color: rgb(0, 153, 0);">% set to -1 for infinite
    end
    
    retry        = 0;
    input_socket = [];
    message      = [];

    while true

        retry = retry + 1;
        if ((number_of_retries > 0) && (retry > number_of_retries))
            fprintf(1, 'Too many retries\n');
            break;
        end
        
        try
            fprintf(1, 'Retry %d connecting to %s:%d\n', ...
                    retry, host, port);

            % throws if unable to connect
            input_socket = Socket(host, port);

            % get a buffered data input stream from the socket
            input_stream   = input_socket.getInputStream;
            d_input_stream = DataInputStream(input_stream);

            fprintf(1, 'Connected to server\n');

            % read data from the socket - wait a short time first
            pause(0.5);
            bytes_available = input_stream.available;
            fprintf(1, 'Reading %d bytes\n', bytes_available);
            
            data_reader = DataReader(d_input_stream);
        message     = data_reader.readBuffer(bytes_available);
            
            message = char(message'); % Data comes out as a column vector
            
            % cleanup
            input_socket.close;
            break;
            
        catch
            if ~isempty(input_socket)
                input_socket.close;
            end

            % pause before retrying
            pause(1);
   end
    end
end

Instead of looping for each byte, we now ask the DataReader object to read the specified number of bytes and return them to us. So a single function call per read. This could be used to read buffers of expected data sizes.

DataReader

Below is the Java source for the DataReader class (available in the MATLAB File Exchange also).

import java.io.*;

class DataReader
{
public DataReader(DataInput data_input)
{
   m_data_input = data_input;
}

public byte[] readBuffer(int length)
{
   byte[] buffer = new byte[length];

   try
   {
       m_data_input.readFully(buffer, 0, length);
   }

   catch (StreamCorruptedException e)
   {
       System.out.println("Stream Corrupted Exception Occured");
       buffer = new byte[0];
   }
   catch (EOFException e)
   {
       System.out.println("EOF Reached");
       buffer = new byte[0];
   }
   catch (IOException e)
   {
       System.out.println("IO Exception Occured");
       buffer = new byte[0];
   }

   return buffer;
}

private DataInput m_data_input;
}

It is very simple class that is given a DataInput to read from, and has a single public method readBuffer(int length) that returns a byte array.

If you have the Java Software Development Kit installed, you can compile the class with:

C:\matlab\matlab_socket> javac data_reader.java

Running the Client/Server example with DataReader

This is the same as the previous example, with one exception. You must tell MATLAB where to find the compiled DataReader class (In my case it is located in the C:\matlab\matlab_socket directory):

>> javaaddpath('C:\matlab\matlab_socket');

Opening up two instances of Matlab:

% Instance 1
>> message = char(mod(1:1000, 255)+1);
>> server(message, 3000, 10)
Try 1 waiting for client to connect to this host on port : 3000
Try 2 waiting for client to connect to this host on port : 3000
Try 3 waiting for client to connect to this host on port : 3000
Try 4 waiting for client to connect to this host on port : 3000
Client connected
Writing 1000 bytes

% Instance 2 (simultaneously)
% NOTE: If the 'server' was runnning on a non local machine, substitute its IP address
% or host name here:
%   data = client('10.61.1.200', 2666); % To connect to server at IP 10.61.1.200:2666
>> javaaddpath('C:\matlab\matlab_socket');
>> data = client('localhost', 3000)
Retry 1 connecting to localhost:3000
Connected to server
Reading 1000 bytes

data =



 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~                                                                                                     

 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~                                                                                                     

 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~                                                                                                     

 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~

Why not call readFully() from within client.m?

The more astute readers may be asking the above question. Why not do something like this in client.m:

message = zeros(1, bytes_available, 'uint8');
d_input_stream.readFully(message, 0 bytes_available);

At first glance that seems great. It runs without error, but hey, you don't seem to be getting any output?¿?

The problem is that DataInputStream.readFully() takes a reference to a byte array to populate.

When you pass things around in MATLAB that are to be modified, MATLAB supplies a copy. Thus the Java most likely sees a byte array reference, and populates it, but it is only the copy that is populated, not the original message array.

Until there exists some way in MATLAB to pass references to Java methods, I am stuck using helper Java classes (If anyone else has ideas on how this could be done better, feel free to comment!).

Mathworks File Exchange Profile Information - Part 2

2008-09-18T15:40:00.005+08:00

Ok, A quick little update here before I head off on holidays!

Continuing on from my previous post I realised that I wanted to watch how my submissions were performing over time (my memory is not quite what it used to be).

I had a crazy idea, could I create a Timer that would scrape my user statistics from the Mathworks File Exchange site and save them in a format such that I could plot them easily???

Introducing Monitor File Exchange Statistics.

Plotting my stats for the past week:
I'm not a big fan of the look of the submitted files plot as it can get a bit busy. I'm envisaging a lightweight GUI with a drop-down selector for each submitted file might be nicer. Particularly for those people with a large number of submitted files. I'll update the submission after I get back to do something like this.

To set the automatic scraping of your File Exchange statistics you need to do the following:

Download the required MATLAB files from here. Need Monitor File Exchange Stats and Retrieve File Exchange Profile Information.
Place the files somewhere in your MATLAB path. Note, if you already have a startup.m, just add the InitialiseFileExchangeInfoTimer and UpdateFileExchangeInfoTimerFcn to your existing startup.m and call InitialiseFileExchangeInfoTimer from within.
Modify FEX_AUTHOR_ID and UPDATE_PERIOD with your File Exchange Author ID and your desired scrape frequency (seconds). The cache file defaults to being stored in your MATLAB root directory, modify FEX_SAVE_FILE if you wish to customise this location.
Save any changes and restart MATLAB.
Each time you restart Matlab you will be presented with a plot of your current and historical stats (for as long as you have been scraping).
To manually plot your current stats, call plotLocalFileExchangeInfo(cache_file)

In terms of file size, I've had the script running every 10 minutes for nearly a week and my cache file is only 45 kB.

Mathworks File Exchange Profile Information - Part 1

2008-09-04T12:53:00.002+08:00

I have been uploading files to the Mathworks File Exchange and have found that I get a sick pleasure looking at my profile page each morning and seeing what people are downloading.

So I thought I would write some MATLAB that would streamline my morning operation as well as gather some additional stats such as downloads/rank over time.

Part 1 - Retrieving File Exchange Profile Information

I've developed a simple function that will, given an Author ID download the Author's profile page from the Mathworks File Exchange and parse it for the Name, Total Downloads, Rank and the list of submitted files.

The function is available on the Mathworks File Exchange: Retrieve File Exchange Profile Information

function file_exchange_info = getFileExchangeProfileInfo(author_id)
%GETFILEEXCHANGEPROFILEINFO Retrieve File Exchange information
%   file_exchange_info = GETFILEEXCHANGEPROFILEINFO(author_id) retrieves an
%   Author's information from the Mathworks File Exchange
%   (http://www.mathworks.com/matlabcentral/fileexchange/)
%
%   file_exchange_info is a structure with the following fields:
%
%   .author_id       - As supplied to the function
%   .author_name     - Author name
%   .rank            - Author rank
%   .num_downloads   - Total number of downloads
%   .submitted_files - An array of structures with the following fields:
%       .id             - File Exchange submission ID
%       .num_downloads  - Number of downloads for this submission
%       .name           - Number of reviews for this file
%
%   If the File Exchange page could not be loaded, file_exchange_info will be
%   returned empty.
%
% Examples:
% file_exchange_info = getFileExchangeProfileInfo(1109866) % My profile
%
% See also urlread

To use the script simply call it supplying the author if interest's Author ID. For example lets have a look at Stuart McGarrity's profile page with Author ID 126174 (currently the number 1 ranked author):

>> info = getFileExchangeProfileInfo(126174)
info =
   author_id: 126174
 author_name: 'Stuart McGarrity'
        rank: 1
num_downloads: 163181
submitted_files: [1x23 struct]

Looking at each submitted file in particular:

>> for i = 1:length(info.submitted_files)
info.submitted_files(i)
end


ans =
          id: 2262
        name: '802.11b PHY Simulink Model'
num_downloads: 21054
ans =
          id: 722
        name: 'Bluetooth modulation and frequency hopping'
num_downloads: 19999
ans =
          id: 724
        name: 'DTMF generator and receiver'
num_downloads: 15117
ans =
          id: 907
        name: 'Bluetooth voice transmission'
num_downloads: 12225
ans =
          id: 3213
        name: '802.11b PHY MATLAB Code'
num_downloads: 11803
ans =
          id: 2283
        name: 'Bluetooth Full Duplex Voice  and Data Transmission'
num_downloads: 10830
ans =
          id: 746
        name: 'IS-95A CDMA Power Control'
num_downloads: 10406
ans =
          id: 787
        name: 'IS-95A Mobile Phone Call Processing'
num_downloads: 9214
ans =
          id: 4380
        name: 'MATLAB for C/C++ Programmers'
num_downloads: 9174
ans =
          id: 9060
        name: 'Handling Large Data Sets Efficiently in MATLAB'
num_downloads: 8329
ans =
          id: 7595
        name: [1x59 char]
num_downloads: 8165
ans =
          id: 2596
        name: '10Base-T Ethernet'
num_downloads: 5366
ans =
          id: 1550
        name: 'Packet Switch'
num_downloads: 4564
ans =
          id: 9622
        name: 'MDF Import Tool and Function'
num_downloads: 3491
ans =
          id: 3939
        name: 'Import Fig File to Axes'
num_downloads: 2386
ans =
          id: 6528
        name: 'Introduction to MATLAB 7 Webinar Demonstrations'
num_downloads: 2168
ans =
          id: 13548
        name: 'chkmem'
num_downloads: 1941
ans =
          id: 18972
        name: [1x78 char]
num_downloads: 1941
ans =
          id: 16075
        name: 'Textscantool'
num_downloads: 1511
ans =
          id: 18971
        name: [1x70 char]
num_downloads: 1410
ans =
          id: 14438
        name: [1x69 char]
num_downloads: 924
ans =
          id: 9298
        name: 'Data and M-Files for Demonstrations on MATLAB Demo Page'
num_downloads: 663
ans =
          id: 19540
        name: [1x77 char]
num_downloads: 614

Part 2 of this post will detail how I will be using the information returned by this function over time.

Now all I need to do is write it!

TCP/IP Socket Communications in MATLAB

2008-08-19T22:03:00.002+08:00

I often see people asking about network communications on the MATLAB Newsgroup. Often this is for the communication between instances of MATLAB.

Using the ability to call Java directly from within MATLAB, I'm going to provide a short example of a client/server written solely in MATLAB and usable from Release 14 onwards (possibly even earlier).

The example is available on the Mathworks File Exchange: Simple TCP/IP Socket Comms Example

I'm working on a little TCP/IP comms library at the moment using these techniques. It will provide a nice layer of abstraction and allow you to use Sockets as you would in other programming languages (as well as one can in a single thread). Keep an eye out for it on the File Exchange.

Interpreted Java?

Amazingly we can execute Java code, even from within the Command Window without the need to compile. For example, the traditional example:

>> import java.lang.*
>> System.out.println('Hello World')
Hello World

To perform socket communications, we utilise the Java Socket and Input/OutputStream classes to pass data around via TCP/IP sockets.

On the server side we use (unsurprisingly) a ServerSocket, which once a client has been accepted, provides a Socket around which we wrap a DataOutputStream to which we can write data.

On the client side we use a Socket to connect to the specified host and port which provides us an InputStream which we wrap in a DataInputStream to read data from.

The code for the example server and client is outlined below.

client.m

% CLIENT connect to a server and read a message
%
% Usage - message = client(host, port, number_of_retries)
function message = client(host, port, number_of_retries)

import java.net.Socket
    import java.io.*

    if (nargin <>
        number_of_retries = 20; % set to -1 for infinite
    end
    
    retry        = 0;
    input_socket = [];
    message      = [];

    while true

        retry = retry + 1;
        if ((number_of_retries > 0) && (retry > number_of_retries))
            fprintf(1, 'Too many retries\n');
            break;
        end
        
        try
            fprintf(1, 'Retry %d connecting to %s:%d\n', ...
                    retry, host, port);

            % throws if unable to connect
            input_socket = Socket(host, port);

            % get a buffered data input stream from the socket
            input_stream   = input_socket.getInputStream;
            d_input_stream = DataInputStream(input_stream);

            fprintf(1, 'Connected to server\n');

            % read data from the socket - wait a short time first
            pause(0.5);
            bytes_available = input_stream.available;
            fprintf(1, 'Reading %d bytes\n', bytes_available);
            
            message = zeros(1, bytes_available, 'uint8');
            for i = 1:bytes_available
                message(i) = d_input_stream.readByte;
            end
            
            message = char(message);
            
            % cleanup
            input_socket.close;
            break;
            
        catch
            if ~isempty(input_socket)
                input_socket.close;
            end

            % pause before retrying
            pause(1);
   end
    end
end

server.m

% SERVER Write a message over the specified port
% 
% Usage - server(message, output_port, number_of_retries)
function server(message, output_port, number_of_retries)

    import java.net.ServerSocket
    import java.io.*

    if (nargin <>
        number_of_retries = 20; % set to -1 for infinite
    end
    retry             = 0;

    server_socket  = [];
    output_socket  = [];

    while true

        retry = retry + 1;

        try
            if ((number_of_retries > 0) && (retry > number_of_retries))
                fprintf(1, 'Too many retries\n');
                break;
            end

            fprintf(1, ['Try %d waiting for client to connect to this ' ...
                        'host on port : %d\n'], retry, output_port);

            % wait for 1 second for client to connect server socket
            server_socket = ServerSocket(output_port);
            server_socket.setSoTimeout(1000);

            output_socket = server_socket.accept;

            fprintf(1, 'Client connected\n');

            output_stream   = output_socket.getOutputStream;
            d_output_stream = DataOutputStream(output_stream);

            % output the data over the DataOutputStream
            % Convert to stream of bytes
            fprintf(1, 'Writing %d bytes\n', length(message))
            d_output_stream.writeBytes(char(message));
            d_output_stream.flush;
            
            % clean up
            server_socket.close;
            output_socket.close;
            break;
            
        catch
            if ~isempty(server_socket)
                server_socket.close
            end

            if ~isempty(output_socket)
                output_socket.close
            end

            % pause before retrying
            pause(1);
   end
end
end

Opening up two instances of Matlab:

% Instance 1
>> message = char(mod(1:1000, 255)+1);
>> server(message, 3000, 10)
Try 1 waiting for client to connect to this host on port : 3000
Try 2 waiting for client to connect to this host on port : 3000
Try 3 waiting for client to connect to this host on port : 3000
Try 4 waiting for client to connect to this host on port : 3000
Client connected
Writing 1000 bytes

% Instance 2 (simultaneously)
% NOTE: If the 'server' was runnning on a non local machine, substitute its IP address
% or host name here:
%   data = client('10.61.1.200', 2666); % To connect to server at IP 10.61.1.200:2666
>> data = client('localhost', 3000)
Retry 1 connecting to localhost:3000
Retry 2 connecting to localhost:3000
Connected to server
Reading 1000 bytes

data =



 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~                                                                                                     

 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~                                                                                                     

 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~                                                                                                     

 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~

This code can be expanded to read/write arbitrary data types, and SHOULD be expanded to properly deal with errors (ie not getting all of the buffer on receive end), but it serves as a simple example of how to get communication between MATLAB and other applications / instances of MATLAB.

Optimisation through MEX files

2008-08-12T22:53:00.002+08:00

Whilst MATLAB is an excellent expressive tool, it can occasionally run a little bit slow for our liking. However, the folks at Mathworks have provided an interface that can be used to speed up code execution in particular circumstances.

MEX Files

MATLAB allows for compilation of C or Fortran sub-routines into a DLL (or equivalent) such that it can be called from within MATLAB as per any other function.

I'll be using a simple example I came across a while ago when attempting to read in large GPS logs containing on the order of a million GPS position records. As expected, the process of parsing these files took some time. What was unexpected was where the code was using up CPU time.

A quick run of the MATLAB Profiler revealed that approximately 50% of my processing time was spent in the calculation of the NMEA checksum (defined here). The MATLAB calculateChecksum function used is outlined below.

%===============================================================================
% Description : Calculate the NMEA Checksum for the supplied string. Calculated
%               as the successive bitwise exclusive OR of all characters
%===============================================================================
function checksum = calculateChecksum(sentence)
    
% Initialise checksum 
checksum = uint8(0);      

for i_char = 1:length(sentence) 
checksum = bitxor(checksum, uint8(sentence(i_char))); 
end      

checksum = dec2hex(checksum, 2);         

end

To demonstrate the CPU usage of the above code snippet, a short test function was created:

function test_Checksum

nmea_sentence = 'GPGGA,195237,4308.639,S,07744.402,E,1,03,3.2,365.3,M,-34.5,M,1001,';
cs = '';

tic
  for i = 1:50000
cs = calculateChecksum(nmea_sentence);
  end
toc

  % Verify Checksum
  if (~strcmp(cs, '7F'))
error('Incorrect Checksum calculated');
  end    
end

The result of this function when executed several times:

Elapsed time is 4.990806 seconds.
Elapsed time is 4.978824 seconds.
Elapsed time is 5.029520 seconds.

Looking at the profiler output (run independently):

The majority of the time is spent performing the iterative XOR and the conversion from decimal to hexadecimal.

Using this example from Mathworks as a guide I created a simple MEX compatible C function that would calculate the 2 character hexadecimal checksum from a supplied string.

#include "mex.h"
#include <stdio.h>


void calculateChecksumFunction(const char* in_string, char *out_string)
{
int checksum_as_int = 0;
int i, str_length = strlen(in_string);

for (i = 0; i < style="color: rgb(51, 102, 255);">int)*(in_string++);
}

checksum_as_int &= 0xFF;
sprintf(out_string, "%02X", checksum_as_int);
}

//****************************************************************
void mexFunction(int nlhs, mxArray *plhs[],
       int nrhs, const mxArray *prhs[])
{
char *input_buf, *output_buf;
int   buflen, status;

 /* Check for proper number of arguments. */
if (nrhs != 1)
mexErrMsgTxt("One input required.");
else if (nlhs > 1)
mexErrMsgTxt("Too many output arguments.");

/* Input must be a string. */
if (mxIsChar(prhs[0]) != 1)
mexErrMsgTxt("Input must be a string.");

/* Input must be a row vector. */
if (mxGetM(prhs[0]) != 1)
mexErrMsgTxt("Input must be a row vector.");

/* Get the length of the input string. */
buflen = (mxGetM(prhs[0]) * mxGetN(prhs[0])) + 1;

/* Allocate memory for input and output strings.
   * output string should be 2 ASCII characters (plus terminator) */
input_buf = mxCalloc(buflen, sizeof(char));
output_buf = mxCalloc(3, sizeof(char));

/* Copy the string data from prhs[0] into a C string 
   * input_buf. */
status = mxGetString(prhs[0], input_buf, buflen);
if (status != 0)
mexWarnMsgTxt("Not enough space. String is truncated.");

/* Calculate checksum and store result in output_buf */
calculateChecksumFunction(input_buf, output_buf);

/* Format return as a mex-string */
plhs[0] = mxCreateString(output_buf);

return;
}

This MEX compatible C file was then compiled using the 'mex' command from the MATLAB command window:

mex calculateChecksumMEX.c

This created a DLL in the same directory named calculateChecksumMEX.dll.

Substituting a call to calculateChecksumMEX in the test function redirects the processing to the created DLL.

The speed improvement is immediately noticeable:

Elapsed time is 0.423503 seconds.
Elapsed time is 0.425224 seconds.
Elapsed time is 0.430266 seconds.

An order of magnitude speed improvement was gained through the simple technique of identifying and isolating portions of code which were using the most CPU time and performing these operations in an an efficient C sub-routine.

Now MEX is not the silver bullet for every slow performing MATLAB function, but can prove to be useful. I would always recommend running the MATLAB Profiler over your code at least once to identify regions of poor performance. Poorly written MATLAB can run orders of magnitude slower than well written MATLAB.

Display Cursor Coordinates

2008-07-29T21:27:00.005+08:00

A simple callback function that prints the current cursor location in plot coordinates into the plot window in a user specified location/format/colour.

DisplayCursorLocation

% cursorLocation - WindowButtonMotionFcn displaying cursor location in plot
%===============================================================================
% Description : Display the current cursor location within the bounds of a
%               figure window. Assigned as a WindowButtonMotionFcn callback
%               function. Only updates when mouse is moved over plot contents.
%
% Parameters  : obj         - Figure originating the callback
%               event       - not used (but required)
%               location    - Location within plot window for text. Can be
%                             'BottomLeft', 'BottomRight', 'TopRight', 'TopLeft'
%                             or a [1x2] array of XY location
%               format_str  - A sprintf format string that will accept 2 float
%                             parameters. ie 'X: %.3f, Y: %.3f'
%               text_color  - either a color character (ie 'r') or a RGB
%                             triplet (ie [1.0 1.0 0.5])
%
% Return      : None
%
% Usage       : Assign to a Figure WindowButtonMotionFcn callback:
%                 set(fig_handle, 'WindowButtonMotionFcn',
%                     @(obj, event)cursorLocation(obj, event, 'BottomLeft',
%                     'X: %.3f, Y: %.3f', 'r')
%
% Author      : Rodney Thomson
%               http://iheartmatlab.blogspot.com
%===============================================================================
function cursorLocation(obj, event, location, format_str, text_color)

The cursorLocation function is assigned as the WindowButtonMotionFcn for a figure. Any time the mouse is moved over the specified figure, the callback function will be executed.

The callback function retrieves the cursor location in plot axes coordinates and uses the supplied sprintf format to produce a text label which is printed in a specific location in the plot. This location can be a preset value or an arbitrary [X,Y] coordinate.

Example usage:

t = linspace(-5,5);
y = sinc(t); f = figure; plot(t, y, 'r');
set(f, 'WindowButtonMotionFcn',                           ...
@(obj, event)cursorLocation(obj, event, 'BottomLeft', ...
                          ' X: %.3f\n Y: %.3f', 'r')

If you wanted to avoid setting the WindowButtonMotionFcn callback yourself, you could use the following wrapper function:

function displayCursorLocation(figure_handle, location, format_string, text_color)

  set(figure_handle, 'WindowButtonMotionFcn', ...
      @(obj, event)cursorLocation(obj, event, location, format_string, text_color));
end

Recursive Directory Function Execution

2008-07-22T20:35:00.003+08:00

A simple function that provides a large amount of flexibility in its use.

DirectoryRecurse

% directoryRecurse - Recurse through sub directories executing function pointer
%===============================================================================
% Description   : Recurses through each directory, passing the full directory
%                 path and any extraneous arguments (varargin) to the specified 
%                 function pointer
%
% Parameters    : directory        - Top level directory begin recursion from
%                 function_pointer - function to execute with each directory as
%                                    its first argument
%                 varargin         - Any extra arguments that should be passed 
%                                    to the function pointer.
%
% Call Sequence : directoryRecurse(directory, function_pointer, varargin)
%
%                 IE: To execute the 'rmdir' command with the 's' parameter over 
%                     'c:\tmp' and all subdirectories
%
%                     directoryRecurse('c:\tmp', @rmdir, 's')
%
% Author        : Rodney Thomson
%                 http://iheartmatlab.blogspot.com
%===============================================================================
function directoryRecurse(directory, function_pointer, varargin)

    contents    = dir(directory);
    directories = find([contents.isdir]);
    
    % For loop will be skipped when directory contains no sub-directories
    for i_dir = directories
        
        sub_directory  = contents(i_dir).name;
        full_directory = fullfile(directory, sub_directory);
        
        % ignore '.' and '..'
        if (strcmp(sub_directory, '.') || strcmp(sub_directory, '..'))
            continue;
        end 
        
        % Recurse down
        directoryRecurse(full_directory, function_pointer, varargin{:});
    end
    
    % execute the callback with any supplied parameters.
    % Due to recursion will execute in a bottom up manner
    function_pointer(directory, varargin{:});
end

The directoryRecurse function finds all directories below the supplied directory and executes the supplied function with the full directory path as the first argument. Any extra arguments supplied to directoryRecurse are passed onto the function supplied in function_pointer.

Its a hard concept to explain with words, so here are a few useful examples:

% add current directory and any sub-directory to the Matlab search path
directoryRecurse(pwd, @addpath);

% delete all contents of 'c:\tmp', requires passing the 's' flag to the rmdir function
directoryRecurse('c:\tmp', @rmdir, 's')

Sound Card Spectral Analyser GUI

2008-07-15T22:55:00.004+08:00

Glancing at the title of this post you might be mistaken for thinking I have repeated my previous post! This is not the case.

SoundcardSpectralAnalyserGui

This GUI (developed using GUIDE) took the SoundcardSpectralAnalysis functionality and wrapped it in a Matlab class. An object of this class was then utilised by the GUI in order to display real time analysis of acoustic data sampled off the system sound card.

Object Oriented Matlab

A Matlab class named SoundcardSpectralAnalyser was developed. It has a constructor, and start(), stop() and set() methods.

Unfortunately, as I am not yet using Matlab 2008a, I do not have access to its wonderful new OO interface. I will not go into too much detail regarding the older OO mechanic because eventually it will be outdated... And it is nothing to be celebrated really. (Hmmm maybe i should register ihatematlab.blogspot.com)

Constructor
The responsibility of the constructor is to initialise any member variables and define the returned structure as a Matlab class of type 'SoundcardSpectralAnalyser'.

Note that the constructor calls out to the set() method to initialise parameters from the variable length argument list. This has to be called AFTER the class type declaration to ensure the correct set method is used.

function this = SoundcardSpectralAnalyser(time_plot, freq_plot, varargin)

  % Initialise default parameters if not supplied
  this.Fs          = 44000;
  this.n_bits      = 16;
  this.n_channels  = 2;
  this.update_rate = 5;

  this.time_plot = time_plot;
  this.freq_plot = freq_plot;

  this.audio_recorder = [];

  this = class(this, 'SoundcardSpectralAnalyser');

  % Set parameters as supplied
  this = set(this, varargin{:});

end

Start()
The start method extracted the initialisation and audiorecorder starting functionality from the SoundcardSpectrumAnalysis function. Separating this ensures that an object of SoundcardSpectrumAnalyser can be constructed at one time but not started until a later point in time. Also, subsequent audiorecorder objects will be using the current analysis parameters (sample rate, sample size, number of channels and update rate).

The start() method also contains the TimerFcn callback responsible for updating the supplied plots. This is functionally unchanged from the script version.

function this = start(this)

  % Setup the audiorecorder which will acquire data off default soundcard
  this.audio_recorder = audiorecorder(this.Fs, this.n_bits, this.n_channels);

  set(this.audio_recorder, 'TimerFcn', {@audioRecorderTimerCallback, ...
                                        this.time_plot, this.freq_plot});
  set(this.audio_recorder, 'TimerPeriod', 1/this.update_rate);
  set(this.audio_recorder, 'BufferLength', 1/this.update_rate);

  record(this.audio_recorder);  
end

Stop()
The stop method is responsible for stopping the audiorecorder object if it had been created.

function this = stop(this)  

  if (~isempty(this.audio_recorder))
      stop(this.audio_recorder);
  end    
end

Set()
The set method takes in a variable length argument list. This list comprises of Value/Key parameter pairs for setting the sample rate, sample size, number of channels and update rate. All pairs can be set simultaneously or 1 at a time.

I was considering using the inputparser class to do this behaviour, but unfortunately I could not figure out how I could set only 1 of the parameters externally without the remaining parameters in the struct being returned as the specified defaults. Also, it was not available in my oldest version of Matlab (R14 SP2).

function this = set (this, varargin)

  if (mod(length(varargin), 2) ~= 0)
      warning('Parameters must be supplied in Key/Value pairs.');
      return;
  end

  for i_param = 1:2:(length(varargin) - 1)
    
      switch varargin{i_param}
          case 'Fs'
              this.Fs = varargin{i_param+1};
          case 'SampleSize'
              this.n_bits = varargin{i_param+1};
          case 'Channels'
              this.n_channels = varargin{i_param+1};
          case 'UpdateRate'
              this.update_rate = varargin{i_param+1};
          otherwise
              warning('Unknown parameter : %s\n', varargin{i_param});
      end
  end
end

GUI Development in GUIDE

I developed the GUI using Matlab's inbuilt GUI editor known as GUIDE (Graphical User Interface Development Environment). This provides a 'designer' like layout tool which will create the required .fig and .m files for your GUI. It also automatically creates hooks for object callbacks in the .m file for your convenience.

I will not go into too much detail regarding the GUIDE tool. I'd personally recommend firing up the editor (type 'guide' at the Matlab Command Window) and just throw some things on there, put some fprintf() statements in the generated callbacks and go click crazy. I will go into a couple of things that annoy about GUI development in Matlab in particular.

Disclaimer: This GUI was developed using R14 SP2... Guide may have been improved in later releases. I'm yet to investigate and the issues I have found with it may have been fixed!

Figure Resizing
Ok, the default behaviour of a Matlab GUI is to disallow resizing of the main figure window. This is convenient for a Matlab GUI developer... not so much for a Matlab GUI user. Particularly if there a plots involved as users with high resolution screens may wish to take advantage of their expensive toys.

So.. you bring up the Property Inspector for the main figure and set the Resize property to 'on'. Run the GUI and *yay* you can resize the window. But all the contents stay wedged in the bottom left. Hmmm thats no better.

Now, go into the Tools->GUI Options... menu and change the resize behaviour to 'Proportional'. Run the GUI and observe the behaviour.

Hey, thats a little bit better. Any Axes you have in the GUI are being proportionally resized and overall, things look good. However, the proportional resize affects ALL objects - pushbuttons, panels. Things can start to look a bit odd and often careful laying out of objects turns into a complete mess.

So what is the alternative?

Define a ResizeFcn on the main figure window. This is then called when the user resizes the GUI window. From within this callback you can query the current figure position and size and shuffle around your objects manually.

This process is quite tedious but unfortunately required for full control of resize behaviour (There appears to be no 'anchoring' of panels/objects).

Check out SoundcardSpectralAnalyser_GUI_ResizeCallback in the main GUI .m file for an example of the code required to achieve this control.

The desired behaviour of my Resize function is to keep the Parameters panel at the bottom of the figure, keeping its height constant but adjusting the width to match the figure. The remaining figure space is divided equally for the time and frequency domain plots (ensuring there was enough room around the axes for labels).

I did not put any limitations in the Resize function to enforce a minimum figure size as at 30 lines it was getting long enough.

Figure Axes
It was annoying having to remember to pad the size of the Axes to ensure sufficient size for the tick labels/text labels. Ideally the size specified by the Axes would be the maximum bounds containing titles/xlabel/ylabel/colorbars etc. Matlab should be smart enough to dynamically resize the drawn chart area internally to cope with this (as it does with a figure window currently.. actually that gives me an idea.. embedding a figure into a GUI for this very reason. Stay tuned).

Orphaned Objects
Something I noticed whilst developing the GUI was that I was often getting orphaned instances of my SoundcardSpectralAnalyser object (and subsequently the audiorecorder object) when there was an error and the GUI did not close properly.

When I tried to run 'clear classes' I received a warning that X instances of SoundcardSpectralAnalyser exist and classes could not be cleared. Doing a findall(0, 'Type', 'SoundcardSpectralAnalyser') yielded no results, hence there is no way of clearing these.

This may be a bug that has been fixed in more recent versions.

Overall, I feel that Matlab GUIs have a long way to go from that implemented in R14 SP2. I will have a look at 2007b (most recent installed version that I use) and see if things have improved. With care, reasonable GUIs can be developed in Matlab but most of the effort will go into tasks that should really be much simpler.

Sound Card Spectral Analysis

2008-07-01T23:44:00.017+08:00

As my first post on this blog I thought I would introduce a very simple function which exploits some of Matlab's high level data acquisition and plotting abilities.

Although this blog's title is 'iheartmatlab' I will also explore areas about Matlab that I don't like so much.

Ok, onto the first code example:

SoundcardSpectralAnalysis

%===============================================================================
% Description : Acquire acoustic data from default system soundcard and plot
% in both time and frequency domain.
%
% Parameters : Fs          - Acquisition sample frequency   [44000] Hz
%              n_bits      - Sample size                    [16] bits
%              n_channels  - Number of channels to acquire
%                            from sound card                [2]
%              update_rate - Polls sound card for data this
%                            many times per second          [5] Hz
%===============================================================================
function soundcardSpectralAnalysis(Fs, n_bits, n_channels, update_rate)

As per the description this function will continuously acquire data from the soundcard at the specified sample frequency, sample size and update rate for however many channels your soundcad supports.

Default Parameters

One thing I don't like about Matlab is the lack of an efficient manner to define the value of default parameters. I have had to resort to the following to initialise my parameters:

% Initialise default parameters if not supplied
if (~exist('Fs', 'var'))
  Fs = 44000;
end
if (~exist('n_bits', 'var'))
  n_bits = 16;
end
if (~exist('n_channels', 'var'))
  n_channels = 2;
end
if (~exist('update_rate', 'var'))
  update_rate = 5;
end

The code checks whether a variable exists within the workspace; and if it does not, it creates it with the default value. There are other possible methods that could have been utilised such as:

if (nargin < 4)
   update_rate = 5;
    if (nargin < 3)
       n_channels = 2;
        if (nargin < 2)
           n_bits = 16;
            if (nargin < 1)
               sample_frequency = 44000;
            end
        end
    end
end

Although this method uses slightly less lines of code, the level of nesting makes it a bit hard to understand on first glance the purpose of the code. And the approach implemented is at least insensitive to changes in the order of parameters (which is unlikely... but you never know).

My ideal dream solution for default parameters would be similar to C++ (the other language with which I have some experience):

function soundcardSpectralAnalysis(Fs = 44000, n_bits = 16, n_channels = 2, update_rate = 5)

If a parameter does NOT have an assignment against it, then its deemed to be a required parameter.

Initialising Plots

When you are producing plots in Matlab and are going to be continuously updating the contents of the plot, typically overdrawing or updating the previous result (ie updating an line spectrum or vessel track) then I would recommend initializing a plot with your desired visual properties and then updating only the raw data values contained by the plot.

Too often i've seen:

tic
figure
for i = 1:100
   cla
   plot(i, i, '.')
   drawnow
end
toc
Elapsed time is 4.899789 seconds.

A much more efficient alternative:

tic
figure;
point_plot = plot(nan, nan, '.');
for i = 1:100
   set(point_plot, 'XData', i);
   set(point_plot, 'YData', i);
   drawnow
end
toc
Elapsed time is 1.086440 seconds.

So the SoundcardSpectralAnalysis function initialises the time and frequency domain plots and sets up axis bounds / labels:

plot_colors = hsv(n_channels);

% Initialise plots, one above each other in a single figure window
figure;

% Time Domain plot
subplot(2,1,1)
hold on
for i_channel = 1:n_channels
   time_domain_plots(i_channel) = plot(nan, nan, ...
                                       'Color', plot_colors(i_channel, :));
end
xlabel('Sample')
ylabel('Counts')

y_max = 2^(n_bits-1);
ylim([-y_max y_max]);

% Frequency Domain plot
subplot(2,1,2)
hold on
for i_channel = 1:n_channels
   freq_domain_plots(i_channel) = plot(nan, nan, ...
                                       'Color', plot_colors(i_channel, :));
end
xlabel('Frequency (Hz)')
ylabel('dB re 1 count/sqrt(Hz)')
xlim([0 Fs/2])
ylim([0 70])

Audio Recorder

Now we come to the guts of the data acquisition, the Matlab inbuilt audiorecorder.

% Setup the audiorecorder which will acquire data off default soundcard
audio_recorder = audiorecorder(Fs, n_bits, n_channels);

set(audio_recorder, 'TimerFcn', {@audioRecorderTimerCallback, ...
                                audio_recorder,              ...
                                time_domain_plots,           ...
                                freq_domain_plots});
set(audio_recorder, 'TimerPeriod', 1/update_rate);
set(audio_recorder, 'BufferLength', 1/update_rate);

% Start the recorder
record(audio_recorder);

The audiorecorder is a nice simple high level abstraction allowing us to retrieve data from a soundcard. However there is limited control as to how often and how much data is retrieved from the sound card.

Ideally for this application we would determine the number of samples to be read from the sound card (sample_frequency/update_rate) and enter a loop reading this many samples each time. However audiorecorder does not contain this functionality. The only option is to specify a timer callback function and its timer period.

In theory this should work perfectly, however in the real world, timers don't get called EXACTLY every 0.2 seconds (or however long their interval). Depending on CPU load and interrupt timings it can vary.

The effect on audiorecoder is that often you will receive too much / too little data when you query it for the currently recorded data. Unfortunately i could not figure out a simple work around. So this function wil occasionally 'skip' and update and include that data into the next frame's analysis. Sorry folks!

Now.. what makes this code work? Simple its this:

set(audio_recorder, 'TimerFcn', {@audioRecorderTimerCallback, ...
                                audio_recorder,              ...
                                time_domain_plots,           ...
                                freq_domain_plots});

This says that for each time period as configured, execute the 'audioRecorderTimerCallback' function and supply to it these three parameters.

By then starting the recorder you are beginning recording and starting the timer.

Audio Recorder Callback

The callback function we assigned to the audiorecorder will get called approximately ever 1/update_rate seconds. I wont go into too much detail regarding the callback as this post is long enough as is.

The function initially queries the audiorecorder for some of its properties required for accurate frequency analysis.

It then stops the recorder, retrieves the recorded data before starting the recorder again.

% stop the recorder, grab the data, restart the recorder. May miss some data
stop(obj);
data = getaudiodata(obj, data_format);
record(obj);

It then calculates the power spectrum of the audio data and updates the XData and YData fields of the appropriate time / frequency domain plots before forcing a drawing update of the plots.

Error Handling

You might have noticed that the majority of the code in the Audio Recorder callback function is contained within a try block. This is for 2 reasons:

1) If a systematic error is occuring within the function, then it will continue to occur within the function. Each time that function is called. Which is every time the Audio Recorder timer function is triggered. And because at the command line you do not have access to the soundcardSpectralAnalysis workspace, you cannot stop the timer. Your only solution is to close Matlab.
2) You wish to stop the spectral analysis? Well, just closing the figure window will acheive this.

The corresponding catch block does the following:

catch exception
    % Stop the recorder and exit
   stop(obj)
   rethrow(exception)
end

Most important line is "stop(obj)". This stops the recorder and hence the timer function. The error is then rethrown to alert the user to the issue.

In the case of 1) above, you will be able to debug the code if you were extending its functionality withough having to restart Matlab often. In the case of 2), closing the figure window means that when it attempts to update the plot data fields, they do not exist, causing an error to be thrown. Hence the function can be stopped nicely.

Interesting Issue with Audio Recorder Callback Function

The more observant readers might say:
"Why supply "audio_recorder" to the audioRecorderTimerCallback. Particularly as the audio_recorder is the initiater of the callback available as the "obj" variable, and also because audio_recorder is not even used within the callback function whatsoever!"

And I would completely agree with you.

However. If you remove "audio_recorder" from the callback function parameter list, then the timer is never started and never executed:

set(audio_recorder, 'TimerFcn', {@audioRecorderTimerCallback, ...
                                time_domain_plots,           ...
                                freq_domain_plots});

function audioRecorderTimerCallback(obj, event, ...
                                   time_domain_plots, freq_domain_plots)

!!!!FAILS!!!!

It is particularly puzzling seeing as though audio_recorder is not used anywhere within the callback function. This can be illustrated by calling "clear('audio_recorder')" at the start of the function. It still functions as per expected.

It might be a bug, but possibly me misunderstanding the behaviour of the callback functions. If anyone has any suggestions, feel free to leave them in the comments below.

Anyhow, thanks for reading my long post about a very simple function. Hopefully its highlighted some interesting aspects of Matlab that you may not have been too familiar with.

I aim to provide a new sample each week along with a little explanation.