Foundational code and system dependent functionality
- The following set of functionality is needed:
- Process and thread creation and control. Being
able to spawn new process and threads of execution within processes
allows multi-threaded programming utilizing possible hardware
- Synchronization primitives: counting semaphores,
recursive and non-recursive mutexes, and condition variables. These
allow controlled access to data, sequencing and synchronization of
operations between multiple threads of control.
- Interprocess message passing. Being able to send
and receive messages between multiple threads of execution enables
communicating arbitrary data in a flexible manner. For DTPF,
packet-based (datagram-based) message
queues which maintain ordering of messages and do not drop messages are
- Interprocess shared memory. Mapping memory
between processes can reduce copying large amounts data between
processes, reducing run time and dmands on message passing.
- Explicit dynamic linking. To support a
plug-in architecture, dynamic linking of executable code from library
files is needed.
- What software will be used to handle platform
interfaces: YARP, ACE, MUSCLE, SystemLayer? How are these interfaces
presented to the framework implementors and application programmers?
- YARP is not suitable for use in the core of DTPF as it does
support OSX at this time.
- ACE provides a wide set of functionality for developing
networked applications. ACE supports OSX and other platforms (Linux,
Windows) that may be targeted in the future. Further investigation into
using ACE has cast doubt on its utility. The core ACE library itself is
rather large, openly available documentation of the programming
interface is incomplete, and some of the functionality desired for DTPF
is platform dependent or requires conditionally included code using the
- MUSCLE provides a level of platform independence but does
as much functionality as ACE. As discussed in the
information, MUSCLE does provide a message class that is useful for
DTPF. This class is a container for name/value pairs and supports
operations to 'flatten' a message to an array of bytes.
- The SystemLayer developed to support our earlier software
projects (SoundStream, SenseStream) provides the needed
functionality and some degree of system independence, but has
only been developed on BeOS and Linux. The most current SystemLayer has
only been developed and tested under Linux. Porting the SystemLayer to
OS X appears feasible with a small amount of effort on the order of
- ACE will not be used.
- The SystemLayer will be ported to OS X and will be used to
provide system abstraction and independence.
- The muscle::Message class will be used as a container that
can be converted to an array of bytes for transmission across IPC
communication mechanisms (sockets) or storage to a file. Any components
of MUSCLE used should be included with DTPF. This is possible due to
licensing MUSCLE is distributed under and the relatively small
footprint of the muscle::Message class and its dependencies.
- Facade interfaces will be used
to hide the framework implementation and API from direct references to
components of MUSCLE. These facade interfaces will reduce the
complexity of the presented API and decrease coupling to particular
software packages. Reducing coupling in this manner helps reduce the
risk of depending on one package, and allows for future optimizations
for a specific platform.
Management of operating system resources.
- Using the SysV functionality on OS X appears to be the most
direct route to a port of the SystemLayer message queues and shared
memory. Managing the operating system level resources involved with
these services requires some thought to avoid 'leaking' these system
level resources. Since DTPF is intended to support development of
additional software which may be going through a development process or
otherwise have instabilities that result in crashing of a process, open
resources need to be reclaimed properly. Not needing to explicitly
close these resources on normal program shutdown it is also desireable.
- The SystemLayer will be extended to support resource
- On launch a process will initialize the SystemLayer which
will create and open a 'run-lock' file that will be unlink()'ed to be
removed automatically by the OS upon termination of the process through
normal or erroneous termination.
- A shared memory segment will be created to contain record
of the resources allocated by the process. This shared memory segment
will be configured to remain present unless explicitly removed by the
program on normal program shutdown.
- An identifier for the shared memory segment will be written
to a second file that is created so as to persist unless explicitly
removed from disk.
- How is synchronization of data
communication from the start of a 'pipeline' of connected nodes to the
- We plan to implement different processing modes, depending
requirements. This first phase of the framework will only directly
support an off-line mode. A real-time mode will be included in a future
stage of the framework.
- In an off-line processing mode, the rate of processing is
controlled by the throughput of the nodes involved, not by real-time
- The framework needs to support feedback connections. If A
is connected to B which is connected to C, it should be possible to
make a connection from C to A. This is necessary to realize some of the
sensory processing models envisioned.
- In the off-line mode, transfer of buffers will be "pull"
is, a downstream node sends a request for the next buffer to its
upstream connected node. The upstream node receives the request and
sends the next buffer, when that buffer is ready. The downstream node
will then receive that next buffer, and will be notified of receipt of
that method through some kind of "buffer received" event method.
- In the real-time situation, processing will be "push"
upstream node will send a buffer when the buffer is ready, to the
downstream node. The downstream node will receive that buffer in a
similar manner to the off-line case, except that the downstream node
does not have to ask for the buffer. (Our initial implementation of
real-time processing may be carried out in part by making the
downstream node's request for the next buffer do nothing).
- Modification rights to a buffer
will be supported with the framework send buffer and receive buffer
node methods. Nodes will gain access to a buffer through the framework.
Reference counting of buffers will be used to help enforce this. Where
is this handled? By nodes, buffer groups, the buffers themselves?
- How are 'broadcast' connections, in which one node sends
the same data to multiple consumer nodes, handled?
- Do plug-in nodes need to be
loaded into a separate process? Should plug-ins be allowed to be loaded
in the address space of the client application making the request to
- How exactly are specific formats
handled in a manner that makes the core framework, specifically the
roster, independent of specific format implementations. What are the
ramifications of the solution to this? How can two format descriptions
of the same type be considered to be compatible?
- The formats are subclasses of parameter groups which hide
the general hierarchy functionality of parameter groups.
- This would allow generic UI specification of format
between two specific format descriptions can be determined by
examining allowed attribute values.
- How are client listeners
handled? Does this impose requirements on the structure of client
applications (i.e., requiring an event handling loop)?
- How is node event receiving and
- Need to allow for concurrent execution but not use
too many threads.
- Nodes are controlled by events corresponding to the various
actions that can be performed on a node.
- A framework provided event looper class
handles running a thread which listens for event messages on a
- Each event looper has exactly one thread of execution. The
event looper is responsible for starting and stopping that thread.
- Events are filtered as appropriate (some events may be
targeted to the event looper) and dispatched by direct invocation of hook
methods on a node instance.
- A single event looper can dispatch events for multiple
nodes in the same process address space. Events will indicate the
target node to allow proper dispatching by an event looper handling
more than one node. Nodes implementations themselves can start any
number of internal worker threads as needed.
|Figure 2: Node and
- Nodes in the unregistered nodes state can be added to an
provided the node and event looper instances are in the same process
space. After addition the node will be ready to be registered if the
thread is running, otherwise the node will not be able to be registered.
- An unregistered or stopped node can be removed from an
looper. After removal a stopped node will be unregistered.
- An event looper for an unregistered node can be created by
framework on behalf of the node implementation or other program code.
- An event looper has synchronization operations that allow
code executing in other threads to safely use event looper and node