Synchronization of ALSA Sequencer

Takashi Iwai <tiwai@suse.de>

What's New

Introduction

This is the first "experimental" implementation of synchronization on the ALSA sequencer system.

ALSA sequencer was designed to capable of synchronization. Thie means two different ways on the concept of ALSA sequencer system.

  1. ALSA sequencer behaves as a sync master:
    The sequencer core sends sync events periodically to other devices, so that they can be synchronized with the played queue.
  2. ALSA sequencer behaves as a sync slave:
    The sequencer core receives sync events from other devices, and tries to synchronize the queue scheduling with them.
For example, if you have a MIDI device which outputs MIDI real-time clock, the ALSA sequencer can be synchronized with it. The tempo is modified according to the received period. On the other hand, ALSA sequencer can generate MIDI real-time clock to any other ports. If the tempo on the master queue is changed, the clock period varies in responce to it.

The sync capability is not bound only to MIDI ticks but includes also the "real-time". The word "real-time" is very vague. In this case, "real-time" stands for the wall-clock time, usually represented by hour, minute and second, which is contrary to "virtual-time" like MIDI clock ticks with varying time unit. Such an example is SMPTE synchronization. The ALSA sequencer receives SMPTE (or MTC, equivalently) signals from other device and corrects the playing speed in concert with the measured period between received signals. When the real-time sync is slowed down, the MIDI clock will be also slowed.

Implementation Details

For ease of explanation, I'd like to decompose the sync mechanism to the following three parts:

  1. communication between device and queue.
  2. generation of periodic sync messages as sync master.
  3. calculation of queue tempo as sync slave.

Device-Queue Commnucation

The commnucation of device and queue could be implemented as ordinary event routing/delivering. A sync signal could be regarded as a normal event with a certain type, and tranmitted from/to a certain sync port. The sync port works as signal converter, which exchanges an sync event to the preferred signal for an external device. For example, a MIDI port can send/receive MIDI clock message byte, or may compose sync events from MTC or full SMPTE SYSEX. Thus sequencer core itself doesn't have to know which format is used but only how long the expected period of tick or time is.

However, ALSA sequencer can deliver events only between "sequencer ports". Since a queue is of course not a port, the event cannot be directly dispatched to any queue. For solving this, I added the new feature: a queue creates a special port when it is created as sync-enabled. This port is assigned to system client (0) with the static port number queue# + 16, so that the queue and the port correspond uniquely. This port accepts both sync events and queue-control events, such as event start, stop and continue, although such control messages can be sent to timer port (0:1), too. This port works also as a sync generator. The sync signal from the corresponding queue is delivered from this port to destinations. The port is deleted automatically when the queue is deleted.

How to activate a queue as sync master or slave? This is established via subscription as usual port connections. When a queue port is connected as sender, the queue works as sync master and sends sync events to the destinations periodically. On the other hand, when a queue port is connected as receiver, the queue works as sync slave. The queue tempo is controlled through received sync events. The period of sync events to be sent or received (sync resolution) must be specified. Thus the resolution and flags are added in snd_seq_subscribe_info_t structure. (See the next section for more details.)

	  master queue #0               client (sync master)
	        +                               |
	master queue port (0:16)                V
	        |                          sync signal
		V                         [subsctiption]
           sync signal                          |
	  [subscription]                        V
	        |                      slave queue port (0:17) 
	        V                               +
	client(s) (sync slaves)           slave queue #1

Queue as Sync Master

The behavior of queue as sync master is simple. It generates sync events periodically as specified in subscription. The sync resolution can be either in ticks or nanoseconds according to the sync mode. The ALSA sequencer has had dual queue system, which consists of both tick and real-time queues. The former corresponds to MIDI tick schedule, while another corresponds to real-time (as the meaning opposite virtual-time) schedule. The sync mode chooses which queue (tick or real-time) is used. Theoretically a queue can generate sync signals in several different periods. This is not the problem, because subscription mechanism may handle anonymous broadcasting. However, multiple destination is not yet implemented so far.

Queue as Sync Slave

The behavior as sync slave is more complicated. A slave queue measures the time difference between the currently and the previously received sync events, and re-calculates the tempo of queue. Which tempo parameter is modified is dependent upon sync mode (tick or real-time). When a queue is running on tick slave mode, the MIDI tempo is modified so that the events scheduled only on the tick queue are affected. The pace of real-time queue doesn't vary. On the other hand, on real-tiem slave mode, the increment parameter of internal timer-interrupt is modified according to the sync period. That is, when sync period is slower than expected, the increment of real-time queue becomes smaller. This results in the slow down of pace of both real-time and tick queues, although MIDI tempo is kept constant, because the increment of tick is calculated from the increment of real-time queue.

What happens if a MIDI tempo event is explicitly sent to a queue running on tick slave mode? It overwrites the MIDI tempo on the queue in fact, but the tempo will be again changed with synchronization to the received signals. Thus, it's a good idea to send a MIDI tempo event with a proper initial value once before sending sync signals, because sync adjustment takes a little time due to PLL (see below). In the case of real-time queue, there is no way to reset the increment of timer interrupts explicitly.

The calculation of new tempo is based on software PLL. When the phase difference between internal and external clocks becomes too much, the queue runs as free-wheel mode. Various schemes would be possible for the adjustment of tempo. In the current version, a simple scheme like the following is used.

On tick sync mode,
T(i) = x * T(i-1) + (1-x) * DT(i) / R,
where T(i) is the tempo of tick queue in nanosecond per tick, and DT(i) is the time difference between two received sync signals, R is the sync resolution (= ticks per sync), and x is the factor between 0 and 1. The larger x is, more stable but less sensitive becomes the scheme.

In the case of real-time sync mode,
t(i) = t_0 * R / (x * DT(i-1) + (1-x) * DT(i)),
where t(i) is the increment of time in interrupt, t_0 is the base incerement (= period of interrupt), and R is the sync resolution in nanosec per sync.

In addition to these calculations, the tempo or the increment is adjusted for correction of difference of between internal and external phases. The accuracy or robustness of scheme should be more discussed. The parameters will be configurable in the future version, but currently fixed.

Note that a queue can work both as sync master and slave simultaneously. In such a case, the actual period of generated sync signals varies corresponding with the received sync signals.

Sync Event Structure

The sync event is defined as an event with SND_SEQ_EVENT_CLOCK type. It takes snd_seq_ev_queue_control_t as its data structure.

typedef struct {
	unsigned char queue;			/* affected queue */
	unsigned char sync_format;
	unsigned char sync_time_format;		/* opt: time format */
	unsigned char pad[1];			/* reserved */
	union {
		signed int value;		/* affected value (e.g. tempo) */
		snd_seq_timestamp_t time;
		unsigned int position;		/* sync position */
		unsigned int d32[2];
		unsigned char d8[8];
	} param;
} snd_seq_ev_queue_control_t;
The queue field should be set to the queue number, although it is ignored in the driver unlike the control to timer port (because we can know which queue is used uniquely from the port number).

The following three fields after queue are used for synchronization only. The sync_format contains the format of the sync signal. The available formats are as below:

/* mode */
#define SND_SEQ_SYNC_TICK		0x80
#define SND_SEQ_SYNC_TIME		0x40
#define SND_SEQ_SYNC_MODE		0xc0		/* mask */
/* private format */
#define SND_SEQ_SYNC_FMT_PRIVATE_CLOCK	(SND_SEQ_SYNC_TICK|0)
#define SND_SEQ_SYNC_FMT_PRIVATE_TIME	(SND_SEQ_SYNC_TIME|0)
/* pre-defined format */
#define SND_SEQ_SYNC_FMT_MIDI_CLOCK	(SND_SEQ_SYNC_TICK|1)
#define SND_SEQ_SYNC_FMT_MTC		(SND_SEQ_SYNC_TIME|1)
#define SND_SEQ_SYNC_FMT_DTL		(SND_SEQ_SYNC_TIME|2)
#define SND_SEQ_SYNC_FMT_SMPTE		(SND_SEQ_SYNC_TIME|3)
#define SND_SEQ_SYNC_FMT_MIDI_TICK	(SND_SEQ_SYNC_TIME|4)
SND_SEQ_SYNC_FMT_PRIVATE_XXXX formats are available for users freely to use with arbitrary type and time resolution. The other formats above are pre-defined. The format is classified to tick and real-time mode by the bit flag SND_SEQ_SYNC_TICK and SND_SEQ_SYNC_TIME.

The next sync_time_format specifies the SMPTE format on real-time mode. The available formats are

#define SND_SEQ_SYNC_FPS_24		0
#define SND_SEQ_SYNC_FPS_25		1
#define SND_SEQ_SYNC_FPS_30_DP		2
#define SND_SEQ_SYNC_FPS_30_NDP		3

The sync event takes param.position as the clock or time position.

Time Accuracy

Triggering of sync events from master queue is done in timer interrupts as well as normal dispatched events. Thus, the accuracy of sync signal period depends on the frequency of timer interrupt. Most of sync signals require relatively high frequency. For example, MTC with 30fps needs 120 sync signals per second, i.e. in 8.3 msec period. Thus for the accurate sync signals at least 1k Hz frequency is preferred.

Unfortunately, there is no handy interrupt source with such a high frequency on i386 architecture. Although there is RTC (real-time clock), the interrupt is not provided for kernel thread in the standard kernel driver. I hacked RTC driver so that ALSA timer can use high frequency RTC interrupts. The ALSA timer routine may use BH or tasklet in option, because RTC interrupts are likely lost with higher frequency than 2048 Hz.

I'm not 100% sure that RTC is enough reliable for the purpose like the above. If another reliable high-frequency timer is equipped on your soundcard, you'd better to use it instead.

API

The API was slightly modified but I've tried to keep the compatibility as much as possible. At least, both source and binary compatibility are maintained at this moment.

For creating a sync queue, the following function is added to alsa-lib.

int snd_seq_alloc_sync_queue(snd_seq_t *seq, char *name);
When a queue is allocated with this function, a queue port is created and assigned to the queue. This function calls SND_SEQ_IOCTL_CREATE_QUEUE ioctl with SND_SEQ_QUEUE_FLG_SYNC set to flag field.

The mapping of queue port is obtained via the following macro in asequencer.h:

#define snd_seq_queue_sync_port(q)	((q) + 16)
Users should use always this macro instead of calculating the port number directly, since the mapping might be changed in future.

For activating the queue as sync master or slave, use the normal subscription but with additional parameters set in snd_seq_port_subscrie_t. Two new fields, sync and opt.sync_info, are added in snd_seq_port_subscribe_t structure:

typedef struct {
	snd_seq_addr_t sender;		/* sender address */
	snd_seq_addr_t dest;		/* destination address */
	unsigned char queue;		/* input time-stamp queue (optional) */
	unsigned int exclusive: 1,	/* exclusive mode */
	    realtime: 1,		/* realtime timestamp */
	    convert_time: 1;		/* convert timestamp */
#ifdef SND_SEQ_SYNC_TEST
	unsigned int sync: 1;		/* sync */
#endif
	int midi_channels;		/* midi channels setup, zero = do not care */
	int midi_voices;		/* midi voices setup, zero = do not care */
	int synth_voices;		/* synth voices setup, zero = do not care */
#ifdef SND_SEQ_SYNC_TEST
	union {
		char reserved[32];
		snd_seq_queue_sync_t sync_info;
	} opt;
#endif
} snd_seq_port_subscribe_t;
For connecting to the queue port, sync field must be set to 1. The opt.sync_info stores the additional information for sync, and defined as follows:
typedef struct {
	unsigned char format;		/* sync format */
	unsigned char time_format;	/* SMPTE time format */
	unsigned char info[6];		/* format dependent info */
	union {
		snd_seq_queue_tick_sync_t tick;
		snd_seq_queue_time_sync_t time;
	} param;
} snd_seq_queue_sync_t;
The format and time_format specify the format (SND_SEQ_SYNC_FMT_XXX) and the time code (SND_SEQ_SYNC_FPS_XXX) of given synchronization. The info field contains the format-specific additional information. The param field takes different type according to its sync format. When the synchronization is on tick mode, the following data is used:
typedef struct {
	unsigned int ppq;		/* ticks per quarter-note */
	unsigned int ticks;		/* ticks per clock */
	/* slave stuffs */
	int max_tick_diff, max_tick_diff2;
	int x0, x1;
} snd_seq_queue_tick_sync_t;
The ppq and ticks specify the resolution of synchronization. For example, when ppq = 24 and ticks = 2 are given, 12 (= 24 / 2) sync signals per quarter note shall be generated. The ppq can take a special value 0, which means that ppq of sync signal is always equal with ppq of queue. The rest of data are for slave mode. They are not used, so far.

For real-time sync mode, the following data is used instead:

typedef struct {
	unsigned int resolution;	/* frame resolution in nsec */
	unsigned int subframes;		/* # of subframes */
	/* slave stuffs */
	int max_time_diff, phase_correct_time;
	int x0, x1;
} snd_seq_queue_time_sync_t;
The resolution specifies the expected resolution of sync signals (frame) in nanoseconds. For example, 25 fps signals should set resolution to 40,000,000 (=1e9/25). The subframes field specifies the division of frame. For example, MTC mode sets 4 in this field, so that the sync signal is generated at each quarter frame. The rest of data are for slave mode.

For obtaining sync signals from a queue, call a new ALSA lib function, snd_seq_add_sync_master. For example, to get a sync signal at each 4 ticks, you may set up like below:

snd_seq_queue_sync_t sync_info;
snd_seq_addr_t dest;

memset(&sync_info, 0, sizeof(sync_info));
sync_info.format = SND_SEQ_SYNC_PRIVATE_CLOCK;
sync_info.ppq = 0;
sync_info.ticks = 4;
dest.client = my_client;
dest.port = my_port;
snd_seq_add_sync_master(handle, queue, &dest, &sync_info);

If you use a pre-defined sync format, you may skip the initialization of sync_info data. For example, to get a MTC signal, just call:

snd_seq_add_sync_master_mtc(handle, queue, &dest, time_format);

For disabling the sync mastering, use the following function:

snd_seq_remove_sync_master(handle, queue, &dest);

The opposite direction, the queue port as receiver, is achieved by another function, snd_seq_set_sync_slave. Instead of specifying destination client and port, give the source destination and port as argument. The sync_info argument is identical with snd_seq_add_sync_master.

snd_seq_set_sync_slave(handle, queue, &source, &sync_info);
For disabling the sync slave, use the following function:
snd_seq_reset_sync_slave(handle, queue, &source);

Files and Patches

Installation

  1. Extract the RTC patch archive.
    	% tar xvfz rtc-patches.tgz
    	rtc-patches/
    	rtc-patches/rtc-2.2.16.dif
    	rtc-patches/rtc-suse-7.0-2.2.16.dif
    	rtc-patches/rtc-2.4.0-test7.dif
    
    There are three patches in the archive. rtc-2.2.16.dif is for the standard 2.2.16 kernel, rtc-suse-7.0-2.2.16.dif is for the patched 2.2.16 kernel on SuSE-7.0 distro, and rtc-2.4.0-test7.dif is for the 2.4.0-test7 kernel.
  2. Apply the appropriate RTC patch to your Linux kernel.
    	# cd /usr/src/linux
    	# patch -p1 < rtc-patches/rtc-2.2.16.dif
    
  3. Compile and reboot the kernel. You need to enable RTC in char device, of course. If you built RTC as module (only on 2.4.x kernel), you don't have to reboot but just reload the rtc.o module.
  4. Dwonload the latest ALSA driver from CVS server or grab 0.5.9c archive. Even if you're using 0.5.x, it'd be better to get the fresh source from CVS, because there might have been some changes since the last release. For getting the sources of 0.5.x version, run cvs with -rver-0-5-patches option like below:
    	% cvs co -rver-0-5-patches alsa-driver
    
  5. Apply the patch to ALSA driver. Compile the driver with cvscompile script. You'll find CONFIG_SND_RTC is enabled in include/config.h.
  6. Install the driver.
  7. Apply the patch to ALSA library, and compile, install it.
  8. Add the following lines in /etc/modules.conf:
    options snd-timer snd_timer_limit=2
    alias snd-timer-1 snd-rtctimer
    options snd-seq snd_seq_default_timer=1 snd_seq_default_timer_resolution=2000
    options snd-rtctimer rtctimer_freq=2048
    
    The rtctimer_freq option of snd-rtctimer module is the frequency of RTC interrupt in HZ. It must be power of two up to 8192. The second option of snd-seq specifies the preferred resolution of interrupt. In the case above, 2048 will be chosen actually.
  9. Load the driver.

Examples

The Jazz can generate MIDI clock and MTC. For enabling this feature, add ".alsa_sync_output 2" (for midi clock) or ".alsa_sync_output 3" (for MTC) to your jazz.cfg file. In the case of MTC, you'll need to set up the time format via ".alsa_sync_output_format" in jazz.cfg. It takes a value from 0 to 3, which corresponds to 24fps to 30 non-drop fps.

After setting this configuration, jazz will ask you the sync output device. Choose the preferred destination.

The Jazz works also on sync slave mode. For enabling this feature, add ".clocksource" option in jazz.cfg. The argument should be 2 (for midi clock) or 3 (for MTC). (The MTC-slave mode is not supported yet.) Then as well as sync mastering, Jazz will ask the clock source device.

On the slave mode, jazz doesn't start playing even when play button is pushed. It then waits for the start signal from source device, then synchronizes the tempo with it.

Know Problems and TODO

Additional Events

Security Issue

Acknoledgments

The implementation of sync mechanism was very inspired by Frank van de Pol's proposal, which had been posted sevral months ago on ALSA devel ML. I had contact with him and got the idea how could sync stuff be realized.


Takashi Iwai tiwai@suse.de; Go to ALSA Related Stuffs