Ideas for ALSA sequencer kernel
===============================

draft 0.01, 26 April 1998
Frank van de Pol (F.K.W.van.de.Pol@inter.nl.net)

This version is not yet well structured, it's basicly a brain-drain of
mine... All subjects are presented in random order :-)


Current Situation
-----------------

The currently available sequencer interfacer for Linux is the /dev/sequencer
and /dev/music from the OSS. Though these interfaces are sufficiently
usefull for most sequencer applications they have a few shortcomings:

- Only one application at a time can have access
- Because of non realtime character of a time-shared system like Linux the
  driver offers a queue in the kernel which is needed to prevent events to be 
  scheduled too late. This queue introduces big latency in event processing.
- It's one big monotlithic driver.

Especially the 2nd issue restricts building midi oriented applications that
can perform on-par with applications on Apple Macinctoshes and Atari ST's
regarding real-time response. Examples:

1) If one wants to have a sequencer perform a MIDI THRU function, it will
   suffer form big delays, and using the OUT_OFF_BAND ioctl() results in
   hanging notes because the driver has no clue what events are playing. 

2) Events have to be enqueue ahead. Parameter changes (eg. volume control, 
   muting tracks or intruments) will not be effective instantly.



New Sequencer
-------------

To overcome these disadvantages I'd like to propose a new architecture for
scheduling and dispatching MIDI and MIDI oriented events within the Linux
sound driver. Note that this is still 'Paper ware', and has yet to be
developed. 

Some of the idea's I'll present are inspired by the MidiShare "MIDI
operating system" (http://www.grame.fr/english/MidiShare.html), which exists
for Apple mac's.

This new sequencer is intended as a replacement for /dev/music. 

Hightlights:

* Supports multiple concurrent clients, both in userland and in kernel space
* kernel space clients can be loaded as modules
* sequencer takes care of dispatching of events, any client can send
  messages to any other client (or clients)
* superfast event routing in case of kernel module clients
* it's only a sequencer framework, all I/O (eg. MIDI, synth) has to be
  performed by clients. There's nothing but clients
* processing of high-level MIDI oriented messages: note on at timestamp xxx, 
  control change at timestamp xxx


Ascii graphic of architecture:


  =========================
  ||                     ||
  ||                    \||/                 \
  ||  +--------+      +--\/------------+      |
  ||  |        |      |                |      |
  ||  | Timing | -->- | Priority Queue |      |
  ||  |        |      |                |      |
  ||  +--------+      +--------||------+      |
  ||                           ||             \ Sequencer
  ||                          \||/            /   Core
  ||  +--------+  +------------\/--------+    |
  ||  | Client |  |                      |    |
  /\  | Manager|  |     Event Router     |    |
 /||\ |        |  |                      |    |
  ||  +--------+  +---||-----------||----+    |
  ||                 \||/         \||/        /
  ||               +--\/--+     +--\/--+
  ||               |Client|     |Client|   
  ||               |  1   | ... |  n   |
  ||               +--||--+     +--||--+
  ||                  ||           ||
  ||                 \||/         \||/
  ||                  \/           \/
  ===================================
                  

All events flow through the queue. Because a priority queue is used instead
of a simple FIFO, it's no problem to accept events from multiple clients.
(At long as the queue is not full anyway, a well behaved application should
send only as much events that are needed to achieve tight playback (eg. 1
second), and not to try keep the queue overflowing.) 

Clients can either be user-land applications, accessing /dev/seq or kernel
modules that can directly submit events, and are directly called when a
event it dispatched to them (even from interrupt mode).

To change internal parameters of the sequencer like eg. tempo & 
synchronisation a special client will (always) be present within the system.
If an application wants to have a tempo change it then simply can send a
TEMPO event into the system.


User-land clients
-----------------

The applications that are seen as user-land clients just open a /dev/seq
device (or something similar), and read()/write() data from/to it, just like
they used to do with the /dev/music device. For registering and
interrogation the sequencer an ioctl() interface is used.

I'm not sure yet how to provide interface to multiple clients; I'm thinking
of a device that can be opened r/w multiple times, or of that's not possible
or very difficult a range of devices (/dev/seq0 .../def/seqxxxx). The latter
is far more ugly than the first one.

The presense of multiple user-land applications allows for some nice
features:

- Bank managers, synth editors can be run parralel with sequencers
- recording source for a sequencer does not need to be a MIDI input port,
  but can also be the output of some other client, like for instance a bank
  manager. All sysex to setup a midi device can so be recorded in the
  sequencer.
- A single application does not need to provide all the funcionality one can
  think of. Why put a GM/XG mixer application within Cubase if one can use
  an externel application, and let these interact. If one needs a 'meter
  bridge' that shows the levels for all the MIDI channels in the system, but
  the sequencer doesn't provide such a thing, a second 'meter bridge only' 
  client can be used! Similar for mixers, bank managers (download samples to
  wavetable synth (GUS!) on reception of program changes).


Kernel mode clients
-------------------

The kernel mode clients reside in kernel modules. These can be stand-alone
loadable kernel modules, or modules that offer other functionality (eg. MIDI
driver, soundcard driver).

The module level interface exists of following:

- client can register itself, provide information about itself and it's
  capabilities to the system by calling a function exported by the
  sequencer.
- client registers a call-back function that will be called when a event is
  to be dispatched to the client. This can be called from an interrupt
  handler. All the call-back function addresses are stored in some sort of
  a jump table.
- the client informs sequencer what broadcasted messages it's interested in.
- the client can call an (exported) function within the sequencer to enqueue
  a message.
- unregistering also goes by calling unregister function.

Because the events can be dispatched to kernel mode client immediately, this
offers possibilities to provide good midi thru and filtering functions.

Example: 

Midi input driver receives midi bytes (interrupt driven), once a complete
midi message is received, it is send to the sequencer. This event (which has
current timestamp) goes directly to the clients that requested reception of
the note events. If there is a kernel mode midi thru module, it is directly
called, the midi event is perhaps transformed (swapped channel/port), and
send back to the sequencer, where it's directly dispatched to the midi (or
synth) client, which plays the event. Nice low-latency midi patch-bay!
Because midi runs at 31.250 kps, it will take 0.96 ms to receive a note on
message (without running state). Sending this event will also take 0.96 ms,
so the event arrives 1.92 ms later at the playback device, which is faster
than the typical delay within syntheseisers before a sound is produced.

Other applications (apart from drivers) could be support modules for
high-end sequencers, that need fine-graded real-time control.



Client communication
--------------------

The event passing mechanism is well suited for real-time controls, note
events etc. But to access very specific functions of a device (client) like
for instance downloading samples to a sample player, or changing the
microcode for a DSP or onboard processor a different interface will have to
be provided. Make this a special 'for the device or application', or use the
sequencer as a multiplexer to pass the data to the driver.


MIDI ports
----------

Note that there is no such thing as a MIDI port, synth device or mixer in
this picture. The idea behind this is that a driver for a midi port should
be implemented as a 'kernel module'. This 'MIDI Port' module can of course
be part of a low-level midi port, and register is from there. There is no
need to have both a MIDI lowlevel module and a MIDI sequencer interface
module.

To prevent the missery of stuck notes a low-level midi driver should keep an
image of which notes are active. In case a note_off message is missed
(which is an application error!) the driver can shut the notes when asked.

The precense of a priority queue also offers the opportunitiy to process
NOTE events what come with an length. On receiption of such an event (by eg.
a midi driver client), the note can be started and the corresponding note
off can be enqueued. Using this facility the chance of hanging notes because
of abrubt abortion of a midi player will be reduced to 0.


OSS Compatibility
-----------------

Because allmost every MIDI application that exists for unix makes use of the
OSS /dev/sequencer or /dev/music interface, it's a must to provide backwards
compatibility for these applications. Users then can use the new sequencer
engine, while keeping their old applications. The application writers then
can migrate to the new sequencer and make use of improved capabilities.

To achieve the compatibilty a wrapper for /dev/sequencer and /dev/music
devices has to be implemented (as a loadable module). This wrapper can
simply map the OSS events to the corresponding sequencer calls.


Using OSS as the workhorse
--------------------------

At the time of writing for this document, the only currently available midi
and syth driver is the OSS. If a client task is developed that presents
itself to the sequencer core as a bunch of input and output device, and
simply does read/writes to the OSS /dev/music interface (perhaps directly
call the exported functions); a sequencer can be developed and used while
using the OSS as workhorse / low-level driver.


Client Registration
-------------------

To get an application to know what other clients have registered, a few
functions have to be provieded to interrogate what clientss are present, and
what capabilities they have.

eg.:
	Client 1
		Name: System Timer
		Capabilities: Tempo, Sync input, Sync output

	Client 2
		Name: GUS MIDI Port
		Capabilities: MIDI input, MIDI output, Sync input, 
			Sync output

	Client 3
		Name: GUS GF1
		Capabilities: MIDI input

	Client 4
		Name: GUS MAX Codec
		Capabilites: Sync input, Sync input, Sync output

	Client 5
		Name: Timidity (Soft Synth)
		Capabilites: MIDI input

	Client 7
		Name: XG Editor
		Capabilities: MIDI output

	Client 8
		Name: My Sequencer
		Capabilites: MIDI intput, MIDI output 



Event Structure
---------------
          
All the events have (apart) from their specific content a few common fields
        - timestamp (in midi ticks), like oss /dev/music
	- message type/id (eg. NOTE_ON, CHANGE_TEMPO,...)
        - destination, to which client(s) is the message to be send. An
	  event can be send to either:
           a) a specific client, in this case the client number has to
              be given.
           b) all clients that have registered for this (class of) event
              this is basicly a broadcast.
	  This destination also can have (for eg. note events) a port and
          channel. 

Events a modeled after MIDI, but are not restricted to be MIDI events. Any
event that one can think of can be fed into the sequencer and dispatched at
the specified time to the specified device(s). 

Apart from this addressing scheme, a client can also request to get every
message, even one that are meant for other clients. This promiscuous-mode
allows a device to snoop all data.

 
Apart for the obious events like MIDI note on/off, control change etc. Some
other events can be thought of:


- Announcement that new client has registered
- Announcement that client as unregistered (is gone)
- Change of capabilities for a client messages (?)
- Other 'change of state' messages
- Change tempo, timer resolution
- trigger /dev/dsp devices for instantanious sample starting
- wake-up, for creating periodic tasks (when rescheduled) within the kernel 
  drivers.



Difference between /dev/sequencer and /dev/music
------------------------------------------------

For some (historical?) reason OSS provides two different sequencer
interfaces, /dev/sequencer (the old one), and /dev/sequencer2, also known
as /dev/music. Is there a good reason why a new sequencer core also should
provide 2 interfaces? What's exactly the difference between these
interfaces. For backwards compatiblity I understand why these two should be
implemented, but is there any reason why the functionalty cannot be provided
by one (good) sequencer?

If the only real issue is that the old interface gives lower-level access to
a synth device, and such can't be achieved by a simple interface wrapper
(eg. access to every single voice in the GUS for playing MOD files), it
could be an idea to provide such synths with a CAP_LOWLEVEL_SYNTH, or
CAP_LOWLEVEL_AWE32 capability flag. (And perhaps a message to switch from
one mode to another.

Idea: If one wants to have access to individual voices, these can also be
addressed as a buch of midi channels (eg. 0..31), with each channel
representing a singe voices.  



Synchronisation
---------------

One of the points the currently available sequencer solution lacks is
synchronisation. Synchronisation can be used in a few places:

- Normal master clock is the system timer (typically 10ms). 
- To get higher resolution, any timer within the system can be used. Most
  soundcards have a timer onboard that is capable of generating interrupts.
  This can be used as master clock for the sequencer.
- Audio playback can also act as a timesource for synchronisation. By using
  a counter of the number of samples played for syncing the master clock,
  it's a good starting point to get MIDI in sync with digital audio.
- MIDI clock can also be used as time source for the sequencer. Or one can
  decide to use MTC. These two protocols can be received and used to adjust 
  the internal clock, or even simpler can be transmitted.
- Some cards have a special synchronisation port (SMPTE code, FSK or
  something similar). This port can also play it's role in the
  synchronisation game.