Target Communication Framework Specification

Copyright (c) 2007-2019 Wind River Systems, Inc. and others. Made available under the EPL 2.0
Agent portion made available under your choice of EPL 2.0 or EDL v1.0 dual-license.

Direct comments, questions to the tcf-dev@eclipse.org mailing list

Table of Contents

Version History

Version Date Change
0.1 2008-01-10 Initial contribution
1.0 2008-05-06 Approved
1.1 2009-03-04 Added N message

Overview

Today almost every device software development tool on the market has its own method of communication with target system. Communication methods often require individual setup, configuration and maintenance, and impose unnecessary limitations. The goal of the Target Communication Framework is to establish a common ground in the area of communication protocols between development tools and embedded devices.

The goal is a single protocol used to communicate between all tools and targets:

Goals

Definitions

Peer:
a communication endpoint. Both hosts and targets are called peers. A peer can act as a client or a server depending on the services it implements.
Service:
a group of related commands, events and their semantics define a service. A service can be discovered, added or removed as a group at a communication endpoint.
Message:
a packet of data, formatted according to the framework specification and transmitted over a communication channel.
Channel:
a communication link connecting two endpoints (peers). A single channel may be used to communicate with multiple services. Multiple channels may be used to connect the same peers, however no command or event ordering is guaranteed across channels.
Command:
a message sent to a remote peer in order to request some predefined action there.
Result:
a message sent as a response to a command.
Event:
a message sent to all interested parties in order to notify them about state changes.

Requirements

Framework Software Design Considerations

The framework will be packaged, distributed and installed on a host as separate product. It should be installed as system service and require no configuration for most common case – target connected over TCP or UDP on a local network. For more complicated setup, framework should have easily accessible and user friendly GUI with all relevant configuration options.

The framework should use a dynamic discovery protocol to locate targets and other hosts running instances of the framework when possible, and maintain a dynamic list of available communication endpoints, as well as lists of services available at each endpoint. Host discovery is needed to locate hosts able to proxy communications for targets, which are not accessible otherwise - for example, targets connected with RS232 or JTAG to a remote host. It should also be possible to add target configuration manually. Development tools will access this data through the Locator Service API and use it, for example, to present a user a list of available targets that have capabilities needed by a particular tool.

The framework should provide software libraries to be used by tools and target agents developers. The libraries should be available at least for ANSI C and Java. On the host side, at least Windows, Solaris and Linux must be supported. Libraries will provide APIs for low-level communication protocol, Locator Service, preferred marshaling and predefined common services.

The proposed target communication protocol is text-based. It allows extensions, which define messages with blocks of binary data, but it is not a recommended data formatting, and its usage is supposed to be limited. Text-based protocols have both advantages and disadvantages in compare with binary protocols.

Advantages:

Disadvantages:

A possible alternative to consider is binary, variable length encoding like BaseStream.

Concurrency

Concurrent asynchronous communication is much faster then synchronous, because it alleviates communication channel latency and allows better bandwidth utilization. But it also requires proper design of framework software. Concurrency, in general, implies multithreading. However, systems developed with global multithreading, are often unreliable and prone to different kinds of thread synchronization problems, which are often very difficult to locate and resolve. We therefore strongly recommend that the software is designed to follow the compartment threading model, which simplifies thread synchronization and promotes reliable software design. In this model each thread execution path is strictly contained in predefined subset of code (compartment), and no code, except for reentrant libraries, is executed by multiple threads. Each compartment has a message queue and other threads communicate with the compartment thread by posting messages to the queue.

The framework APIs are designed to be compatible with the compartment threading model. Hence the API functions do not contain any thread synchronization primitives to protect against multiple threads using the functions. All framework APIs belong to a single compartment and should be used by a single thread. The same thread is used to dispatch events and command results. Concurrency is achieved by declaring API functions to be asynchronous. Asynchronous functions do not have any return value, and returns immediately, most of the time before the intended job is done. They take additional arguments to specify a callback function and callback data. In object-oriented languages such as Java, this is typically done by a single callback object argument containing both the data and the function. The result listener is called asynchronously when the job is done. This approach is commonly known as asynchronous, event-driven or callback-based programming.

One important characteristic of an asynchronous code is that the methods defined by the user will often be called from within the framework itself, rather than from the user's application code. The framework often plays the role of the main program in coordinating and sequencing application activity. This phenomenon is called Inversion of Control (also known as the Hollywood Principle - “Don't call us, we'll call you”).

Reflection

Communication between development tools and embedded devices must allow a host to collect target side data and build a reflection of target state. Reflection is usually incomplete – a subset of all remote data. Reflection is always delayed – it represents a remote peer state in the past. Reflection can be updated by polling for data changes or by listening to events (event is communication message that is sent asynchronously by a peer to notify others about state change). Reflection is correct if it represents a state that actually did happen on remote peer.

Reflection is coherent if it is exactly equal to a subset of the peer state at a single moment of time and that moment of time is not too far in the past. Non-coherent reflection can have parts of data representing peer state at different moments of time. Coherent reflection is more valuable for a user, because non-coherent reflection can have logically conflicting data if that data was collected at different time.

Traditionally, debuggers would ensure coherence of state reflection by collecting data only while target is suspended, and flushing all (or most) reflection data (reducing observed subset to zero) when target is resumed. This approach does not work well for multithreaded, multicore or real time targets. Maintaining correctness and coherence of a non-empty reflection while target is running requires additional support from target agent, communication software and debugger itself.

Since remote peer state is changing over time, coherent reflection can be built only if:

Message ordering

The transmission order of commands, results and events is important, it conveys valuable information about target state transitions and it should be preserved when possible. Consider an example:

Client transmits:

    Command X=2

Then, as result of some activity of another client or the target itself, X is assigned value 3.

Target transmits:

    Event X=3
    Result X=2

Now the client has to show the value of X to a user. If the order of messages is preserved, the client will know that command was executed after X was assigned 3, the last message contains the last known value of X and 2 is the correct value to show. If the target is allowed to transmit events and results in arbitrary order, the client will have no clue what to show – 2 or 3. In fact, the client will have to make a tough decision about each message it receives: either trust the message data as the correct last known target state, or assume the message came in out-of-order and ignore it, or re-request the information from the target.

Note that re-requesting data from the target, in general, does not solve the problem of interpretation of messages when order is not preserved. For example, after sending a request to read value of X, X could change at about the same time, and client could receive:

    Event X=2
    Result X=3
    Event X=4

If order is not preserved, it is still impossible to tell which value of X is the last one. A client could assume value of X unknown every time it receives a notification of X change, and then re-request the data again. But this is expensive and, if events coming in frequently, client can end up in infinite loop re-requesting the data again and again, and it will never have trustworthy data about current target state.

Developers should be careful when using multithreading or multiple queues in software design – it can easily cause message reordering.

The framework itself is required to preserve message order. However, if for whatever reason a target agent cannot preserve message order, the result will be that clients of the service can receive messages in the wrong order. When this is the case it should be well documented, so tools developers are aware and can make the best of the situation. In most cases it will not cause any trouble, but there is no perfect way to restore the actual sequence of events and maintain data coherency after ordering was lost, and in some cases it can severely impact tool functionality and user experience.

Transport Layer

Tools are required to be transport protocol agnostic, so most of the layer functionality is used internally by framework and is not exposed to clients. This layer maintains a collection of transport protocol handlers. Each handler is designed to provide:

Existing service discovery protocols can be used together with the framework, for example:

Service discovery protocols, as well as transport protocols, will be supported by framework plug-ins, they are not part of the framework code itself, and they can be developed by 3rd parties. Note that existing discovery protocols define term “service” differently - as an independent communication endpoint (usually a TCP/IP port). In this document it is called “peer” (host, target, communication endpoint), and a peer can provide multiple services over single communication channel.

Use of standard discovery protocols should be optional, because it can potentially cause conflict or interference between development tools and application being developed over a use of same standard protocol – device software often includes implementation of service discovery protocols as part of the application code to support their main functions.

Communication Protocol

The communication protocol defines data packet properties and roles common for all services. The communication protocol API provides functions for opening and closing the communication channel for a particular peer, and for sending and receiving data packets. The protocol defines contents of a part of a packet, the rest of the packet is treated as array of bytes at this level. The communication protocol implementation also provides:

The protocol defines three packet types: commands (requests), results (responses), and events. Each packet consists of several protocol defined control fields followed by a byte array of data. Binary representation of control fields is a sequence of zero-terminated ASCII strings. The format of the data depends on the service. We recommend using the framework preferred marshaling for data formatting.

Syntax:


<message><command><result><event><flow control message>

Commands


<command>
    ⇒ C • <token> <service name> <command name> <byte array: arguments>

Command packets start with the string “C”.


<token><chars>

A token is a unique string generated by the framework for each command. It is used to match results to commands.


<service name><chars>

A service name is used to identify a service that handles the command. It is the same string as that returned by Service.getName().


<command name><chars>

The interpretation of the command name depends on the service.

A command should always be answered with result packed. The result does not have to be positive – it can include an error code, or it can be special "N" result that indicates that command was not recognized, but there always must be one. Since the client cannot detect that a response is missing, if for some reason the peer is not able to answer a command, it should consider such situation a fatal communication error and it must shutdown the communication channel. It is not necessary to wait for a result before sending the next command. In fact, sending multiple commands in a burst can greatly improve performance, especially when the connection has a high latency. At the same time, clients should be carefully designed to avoid flooding the communication channel with unlimited number of requests, since this will use resources in forms of memory to store the requests and time to process them.

Results


<result>
    ⇒ N • <token> •
    ⇒ R • <token><byte array: result data>
    ⇒ P • <token><byte array: progress data>

Result packets start with the string “P” for intermediate result, “R” for final result, and “N” if the command is not recognized. Receiving a “R” or “N” result concludes execution of corresponding command. There should be exactly one “R” or “N” result for each command. In addition, command execution can produce any number of intermediate “P” results. “P” results can be sent before “R”, and it can serve, for example, as command execution progress report when execution takes long time.


<token><chars>

The token should match the token field of one of the pending commands that produced the result.

Events


<event>
    ⇒ E • <service name><event name><byte array: event data>

Event packets start with string “E”.


<service name><chars>

The service name identifies the service that fired the event. It is the same string as that returned by Service.getName().


<event name><chars>

The interpretation of the event name depends on the service.

Events are used to notify clients about changes in peer state. Services should provide sufficient variety of events for clients to track remote peer state without too much of polling. Clients, interested in a particular aspect of the target state, should have a “reflection” (or “model”) of that state and update the reflection by listening for relevant events. If a service implements a command that changes a particular aspect of peers state, then, normally, it should also generate notification events when that same part of the state changes and it should provide a command to retrieve the current value of the state – to be used by clients to initialize the reflection. Service events are defined statically, together with commands. The framework does not do any event processing besides delivering them to clients, however a service can define additional event-related functionality if necessary, for example commands for event filtering, enabling, disabling, registration, etc. Care should be taken when designing events for a service – if events are sent too frequently, they will cause flooding of the communication channels and degrade performance. However, too few events will force clients to poll for changes and can also degrade performance. A balanced approach is the best.

Flow Control

It often happens that one side of communication channel produces messages faster then they can be transmitted over the channel or can be consumed by another side. This will cause channel traffic congestion (flooding). The framework will deal with the problem and slow down transmitting side by blocking execution inside sendEvent(), sendCommand() and sendResult() functions when message buffers are full. However, in many cases, it is not the best way to handle congestion. For example, it can make a tool UI appear locked for prolonged time or it can break target software if it is designed to work in real time. Clients can use flow control events to implement advanced techniques to handle traffic congestion, for example, message coalescing, switching to less detailed messages, etc.


<flow control message>
    ⇒ F • <int: traffic congestion level>

The traffic congestion level value is in the range -100..100, where -100 means no pending messages (no traffic), 0 means optimal load, and positive numbers indicate level of congestion. When a peer receives flow control message with congestion level > 0 it should try to reduce its transmition speed.

Message Examples

These examples use simplified command arguments and result data. See the service description for the actual data formats.

Executing suspend command from RunControl service:

 

Send   :      C 1 RunControl suspend “Thread1”
Receive:      E RunControl suspended “Thread1”
Receive:      R 1 “Success”

Same command, but target was already suspended:

Receive:      E RunControl suspended “Thread1”
…
Send   :      C 2 RunControl suspend “Thread1”
Receive:      R 2 “Already suspended”

Same command, but target was suspended (by another client) after sending the command, but before command was executed:

Receive:      E RunControl running “Thread1”
…
Send   :      C 3 RunControl suspend “Thread1”
Receive:      E RunControl suspended “Thread1”
Receive:      R 3 “Already suspended”

Framework API

/**
 *
 * Class Protocol provides static methods to access Target Communication Framework root objects:
 * 1. the framework event queue and dispatch thread;
 * 2. local instance of Locator service, which maintains a list of available targets;
 * 3. list of open communication channels.
 */
public class Protocol {

    private static IEventQueue event_queue;

    /**
     * Before TCF can be used it should be given an object implementing IEventQueue interface.
     * The implementation maintains a queue of objects implementing Runnable interface and
     * executes run methods of that objects in a sequence by a single thread.
     * The thread in referred as TCF event dispatch thread. Objects in the queue are called TCF events.
     * Executing run method of an event is also called dispatching of event.
     *
     * Only few methods in TCF APIs are thread safe - can be invoked from any thread.
     * If a method description does not say "can be invoked from any thread" explicitly -
     * the method must be invoked from TCF event dispatch thread. All TCF listeners are
     * invoked from the dispatch thread.
     *
     * @param event_queue - IEventQueue implementation.
     */
    public static void setEventQueue(IEventQueue event_queue);

    /**
     * @return instance of IEventQueue that should be used for TCF events.
     */
    public static IEventQueue getEventQueue();

    /**
     * Returns true if the calling thread is TCF dispatch thread.
     * Use this call to ensure that a given task is being executed (or not being)
     * on dispatch thread.
     * This method is thread-safe.
     *
     * @return true if running on the dispatch thread.
     */
    public static boolean isDispatchThread();

    /**
     * Causes runnable to have its run
     * method called in the dispatch thread of the framework.
     * Runnables are dispatched in same order as queued.
     * If invokeLater is called from the dispatching thread
     * the runnable.run() will still be deferred until
     * all pending events have been processed.
     *
     * This method can be invoked from any thread.
     *
     * @param runnable the Runnable whose run
     * method should be executed asynchronously.
     */
    public static void invokeLater(Runnable runnable);

    /**
     * Causes runnable to have its run
     * method called in the dispatch thread of the framework.
     * Calling thread is suspended util the method is executed.
     * This method is thread-safe.
     *
     * @param runnable the Runnable whose run
     * method should be executed on dispatch thread.
     */
    public static void invokeAndWait(Runnable runnable)
        throws InterruptedException;

    /**
     * Get instance of the framework locator service.
     * The service can be used to discover available remote peers.
     *
     * @return instance of ILocator.
     */
    public static ILocator getLocator();

    /**
     * Return an array of all open channels.
     * @return an array of IChannel
     */
    public static IChannel[] getOpenChannels();

    /**
     * Interface to be implemented by clients willing to be notified when
     * new TCF communication channel is opened.
     */
    public interface ChannelOpenListener {
        public void onChannelOpen(IChannel channel);
    }

    /**
     * Add a listener that will be notified when new channel is opened.
     * @param listener
     */
    public static void addChannelOpenListener(ChannelOpenListener listener);

    /**
     * Remove channel opening listener.
     * @param listener
     */
    public static void removeChannelOpenListener(ChannelOpenListener listener);

    /**
     * Transmit TCF event message.
     * The message is sent to all open communication channels – broadcasted.
     */
    public static void sendEvent(String service, String name, byte[] data);

    /**
     * Call back after TCF messages sent by this host up to this moment are delivered
     * to their intended target. This method is intended for synchronization of messages
     * across multiple channels.
     *
     * Note: Cross channel synchronization can reduce performance and throughput.
     * Most clients don't need cross channel synchronization and should not call this method.
     *
     * @param done will be executed by dispatch thread after communication
     * messages are delivered to corresponding targets.
     */
    public static void sync(Runnable done);
}

/**
 * IChannel represents communication link connecting two endpoints (peers).
 * The channel asynchroniously transmits messages: commands, results and events.
 * A single channel may be used to communicate with multiple services.
 * Multiple channels may be used to connect the same peers, however no command or event
 * ordering is guaranteed across channels.
 */
public interface IChannel {

    /**
     * Channel state IDs
     */
    static final int
        STATE_OPENNING = 0,
        STATE_OPEN = 1,
        STATE_CLOSED = 2;

    /**
     * @return channel current state, see STATE_*
     */
    int getState();

    /**
     * Send command message to remote peer for execution. Commands can be queued
     * locally before transmission. Sending commands too fast can fill up
     * communication channel buffers. Calling thread will be blocked until
     * enough buffer space is freed up by transmitting pending messages.
     * @param service - a remote service that will be sent the command
     * @param name - command name
     * @param args - command arguments encoded into array of bytes
     * @param done - call back object
     * @return pending command handle
     */
    IToken sendCommand(IService service, String name, byte[] args,
        ICommandListener done);

    /**
     * Command listener interface. Clients implement this interface
     * to receive command results.
     */
    interface ICommandListener {

        /**
         * Called when progress message (intermediate result) is received
         * from remote peer.
         * @param token - command handle
         * @param data - progress message arguments encoded into array of bytes
         */
        void progress(byte[] data);

        /**
         * Called when command result received from remote peer.
         * @param token - command handle
         * @param data - command result message arguments encoded into array of bytes
         */
        void result(byte[] data);

        /**
         * Called when communication channel was closed while command was waiting for result.
         * @param token - command handle
         * @param error - exception that forced the channel to close
         */
        void terminated(IToken token, Exception error);
    }

    /**
     * Send result message to remote peer. Messages can be queued locally before
     * transmission. Sending messages too fast can fill up communication channel
     * buffers. Calling thread will be blocked until enough buffer space is
     * freed up by transmitting pending messages.
     * @param token - command handle
     * @param results - result message arguments encoded into array of bytes
     */
    void sendResult(IToken token, byte[] results);

    /**
     * Get current level of outbound traffic congestion.
     *
     * @return integer value in range –100..100, where –100 means no pending
     * messages (no traffic), 0 means optimal load, and positive numbers
     * indicate level of congestion.
     *
     * Note: in-bound traffic congestion is detected by framework and reported to
     * remote peer without client needed to be involved. Clients willing to provide
     * additional data about local congestion should register itself using
     * Protocol.addCongestionMonitor().
     */
    int getCongestion();

    /**
     * Channel listener interface.
     */
    interface IChannelListener {

        /**
         * Called when a channel is opened.
         */
        void onChannelOpened();

        /**
         * Called when channel closed. If it is closed because of an error,
         * ‘error’ parameter will describe the error. ‘error’ is null if channel
         * is closed normally by calling Channel.close().
         * @param error - channel exception or null
         */
        void onChannelClosed(Throwable error);

        /**
         * Notifies listeners about congestion level changes. When level > 0
         * client should delay sending more messages.
         * @param level - current congestion level
         */
        void congestionLevel(int level);
    }

    /**
     * Subscribe a channel listener. The listener will be notified about changes of
     * outbound traffic congestion level.
     * @param listener - channel listener implementation
     */
    void addChannelListener(IChannelListener listener);

    /**
     * Remove a channel listener.
     * @param listener - channel listener implementation
     */
    void removeChannelListener(IChannelListener listener);

    /**
     * Command server interface.
     * This interface is to be implemented by service providers.
     */
    interface ICommandServer {

        /**
         * Called every time a command is received from remote peer.
         * @param token - command handle
         * @param name - command name
         * @param data - command arguments encoded into array of bytes
         */
        void command(IToken token, String name, byte[] data);
    }

    /**
     * Subscribe a command server. The server will be notified about command
     * messages received through this channel for given service.
     * @param service - local service implementation
     * @param server - implementation of service commands listener
     */
    void addCommandServer(IService service, ICommandServer listener);

    /**
     * Remove a command server.
     * @param service - local service implementation
     * @param server - implementation of service commands listener
     */
    void removeCommandServer(IService service, ICommandServer listener);

    /**
     * A generic interface for service event listener.
     * Services usually define a service specific event listener interface,
     * which is implemented using this generic listener.
     * Service clients should use service specific listener interface,
     * unless no such interface is defined.
     */
    interface IEventListener {
        /**
         * Called when service event message is received
         * @param name - event name
         * @param data - event arguments encode as array of bytes
         */
        void event(String name, byte[] data);
    }

    /**
     * Subscribe an event message listener for given service.
     * @param service - remote service proxy
     * @param server - implementation of service event listener
     */
    void addEventListener(IService service, IEventListener listener);

    /**
     * Unsubscribe an event message listener for given service.
     * @param service - remote service proxy
     * @param server - implementation of service event listener
     */
    void removeEventListener(IService service, IEventListener listener);

    /**
     * @return IPeer object representing local endpoint of communication channel.
     */
    IPeer getLocalPeer();

    /**
     * @return IPeer object representing remote endpoint of communication channel.
     */
    IPeer getRemotePeer();

    /**
     * @return collection of services available on local peer.
     */
    Collection<String> getLocalServices();

    /**
     * @return an object representing a service from local peer.
     * Return null if the service is not available.
     */
    IService getLocalService(String service_name);

    /**
     * @return an object representing a service from local peer.
     * Service object should implement given interface.
     * Return null if implementation of the interface is not available.
     */
    <V extends IService> V getLocalService(Class<V> service_interface);

    /**
     * @return collection of services available on remote peer.
     */
    Collection<String> getRemoteServices();

    /**
     * @return an object (proxy) representing a service from remote peer.
     * Return null if the service is not available.
     * Return an instance of GenericProxy if 'service_name' is not a standard TCF service.
     */
    IService getRemoteService(String service_name);

    /**
     * @return an object (proxy) representing a service from remote peer,
     * which implements given interface.
     * Return null if implementation of the interface is not available.
     */
    <V extends IService> V getRemoteService(Class<V> service_interface);

    /**
     * Install a service proxy object on this channel.
     * This method can be called only from channel open call-back.
     * It allows a client to extends TCF by adding proxy objects for non-standard services.
     * Client, wishing to become service proxy provider, should register itself
     * using either Protocol.addChannelOpenListener() or IChannel.addChannelListener().
     * It is not allowed to register more then one proxy for a given service interface.
     */
    <V extends IService> void setServiceProxy(Class<V> service_interface, IService service_proxy);

    /**
     * Close communication channel.
     */
    void close();

    /**
     * Close channel in case of communication error.
     * @param error - cause of channel termination
     */
    void terminate(Throwable error);

    /**
     * Redirect this channel to given peer using this channel remote peer locator service as a proxy.
     * @param peer_ - peer that will become new remote communication endpoint of this channel
     */
    void redirect(IPeer peer);
}


/**
 * Object implemeting IToken interface is created by framework for every
 * command sent over communication channel. It is used to match command to its
 * results, and also can be used to cancel commands.
 */
public interface IToken {

    /**
     * Try to cancel a command associated with given token. A command can be
     * canceled by this method only if it was not transmitted yet to remote peer
     * for execution. Successfully canceled command does not produce any result
     * messages.
     *
     * @return true if successful.
     */
    boolean cancel();
}

Preferred Marshaling

The TCF message data format is service specific. Since service specifications are separate from the protocol specification, a service designer can choose any data format that suits the service requirements best. However, to promote better compatibility and to simplify service design and implementation, we recommend using JSON for data formatting.

Locator Service

The Locator service uses the transport layer to search for peers and to collect data about the peer's attributes and capabilities (services), see Locator Service.