SB747 -- System Architecture

Hoppie Home

SB747

 Last updated on 2008-04-02 19:32z

TCAS codes

SB747 itself won't be developed further, but the existing TCAS broker key conventions will be helpful to other programmers for a while. The TCAS system is simple. Every time a position report of a plane in the general neighbourhood is received, SB747 drops a TCAS message to the Broker. These messages have the following format:

tcas {callsign engines model lat lon alt spd pbh}

where the fields mean the following:

callsign The callsign of the plane, guaranteed unique and usually extremely constant.

engines Number of engines on the plane, for visual generators.

model Basic plane model, such as "helicopter" or "jet".

lat Latitude in degrees.decimal_degrees.

lon Longitude in degrees.decimal_degrees.

alt True altitude MSL in feet.

spd Ground speed in knots.

pbh Pitch, bank, and heading encoded in a single 32-bits integer. The fields are encoded in Microsoft Flight Simulator units, 10 bits pitch, then 10 bits bank, then 10 bits heading, and the two least significant bits are zero.

When a TCAS traffic blip goes out of range, the very last update is:

tcas {callsign 0 0 0 0 0 0 0}

However, you should take care that updates of a blip may cease without any warning at all. Some kind of timeout is required.

Architecture re-thinking (now obsolete)

Until February 2005, SB747 could boast a common code base for both the VATSIM and IVAO networks (and any other network based on the FSD server software and protocol). But by that time, VATSIM and IVAO each started to roll out clients and servers that used specific, proprietary authentication methods which essentially made sure that only "approved" software could log into the servers. With these network-specific authentication schemes also came large changes in the original protocol, obviously moving both networks farther and farther apart.

Rationale

In order for SB747 to follow these protocol changes while still conforming to the various network's licenses, it became unfeasible to maintain one common code base that was completely shared by all networks. At that moment it was not even clear whether VATSIM would ever allow the existance of a client that could also log in to IVAO servers. So I began work on splitting off the required code for each network into separate modules, for three reasons:
  1. Each network protocol module could be independently changed as the network moved further away from the old common SBPC protocol.
  2. SB747 could still be maintained as full OpenSource software, while the protocol module for a network that insisted on closed source could remain closed (and was distributed in binary-only format).
  3. If some network would actively insist on a monolithic application which could not be used to log in to another network, not even by including two separate protocol modules, it would be easy to separately compile two or more applications, each with only one protocol module inside.

Shared code versus split code

For a while, I played with the idea of having one single "classic" FSD module with network-specific plugins, but this approach quickly turned out to be unnecessarily complex.

True, there always was the possibility that different networks would implement the same idea, and the opportunity for code sharing was definitely present. But each line of shared code meant a possible trap door in case somebody insisted on a license that prohibited (network) code sharing between networks. And the networks explicitly had split up, so they would not be interested to keep a common base in the first place. I could simply copy the existing single network module to the two networks, and start modifying them as the networks moved. Lastly, with common code there was always the possibility that a change in one network meant an update in both the network module and the common code, and therefore in all other network modules as well.

So I came to the conclusion that in this particular situation, a complete separation between network modules with no common network code was the safest and most practical option.

The split code modules should behave like true loadable modules, i.e., when a connection to network A would be made, module A would load; and when the same user subsequently would move to network B, module A would unload and module B would load. I kept a spare system around "just in case", where network modules would be loaded when required, but could not be unloaded. The same discussion surfaced in the Linux kernel development group, so I felt in good company. And after all, just needing to restart SB747 when switching networks did not seem such a big deal.

Cleanly separating code from interface

Of course, SB747 also contains a lot of non-network code (actually the vast majority of the code base is non-network-related). I needed a well-defined, relatively narrow API to connect the SB747 core system with the network module of the day. Luckily, Kees Leune and I designed SB747 back in 1999 with this in mind, and developed the World Database. This is a simple central data structure that can be accessed from both sides with a standard key-value system, extended with timeouts and FIFOs and triggers. It is very well suitable for a fully asynchronous interface. If we had been studying marketing, we would have called it the ActiveRegistry™.

By defining fixed keys and values from the SB747 core point of view, it would be possible to map each individual network event to the appropriate SB747 events. The network modules would be responsible for the correct data translation, but the actual event would be the same. Although this system was not ideal, it looked like it would work pretty well. After all, we did not have to make one single change to it for five years!

The only significant change that SB747 needed to undergo was the notion of more than one network module, of which at most one would be active. This meant that de-registering event triggers became a necessity. This had not been implemented before because there had never been a reason to de-register an event handler. It did not look like a big deal.

The World Database had 13 API calls, of which only one was relevant: world::trigger. This was not more than a wrapper around a plain Tcl lappend statement, and could easily be complemented with a world::untrigger counterpart. With Tcl's way of using namespaces, it was even simple to write a variant that accepted wild cards, so that all triggers with callbacks to a single namespace (module) could be unset at once, without the risk of forgetting one and running into callback trouble later on. It got implemented in ten minutes.

With most synchronous function calls out of the way already, and a clean way to unsubscribe triggers, it looked like making the current "satco" module unloadable and reloadable was going to be rather easy. A few difficulties were encountered with the wind probe hack (that warps a probe ahead over the intended flight plan to measure the winds in the virtual world, as no real wind charts exist), but that whole hack should really go away in the first place. Disabling the .probe command took only one hash mark. This left only two direct satco:: calls in place, to init and close up.

Network operations

All the asynchronous key handling is fine, but there are some things that should still be done by a classic function call. Loading the module was one, initialising it (could be linked to loading), and of course the counterparts cleaning up and unloading.

In order for the correct module to be loaded, a way was required to know which protocol to speak to which server. This wasn't really a big deal, as both relevant networks (IVAO and VATSIM) both had their own server network. A simple adaptation of the hosts.txt syntax would do. An additional advantage of splitting up the networks was that I now got a clean way to store the individual network's usernames (PID/CID) and passwords. This was already possible with the current hosts.txt syntax, though cumbersome to say the least.

Some investigation had to take place to see whether a large part of the hosts.txt file could be replaced by once-per-session data retrieval from the "ServInfo" database. This semi-realtime database can be polled without being in the way of the realtime network servers, yet while holding data that is well enough up to date to serve its role properly. This feature did not receive top priority, as the semi-realtime data was not necessarily more extensive than the fixed file, and would add considerable complexity to a now simple mechanism. A typical network connection flow would be:

  1. SB747 user selects the network and server from File/Setup (persistant between sessions). With network selection, PID/password data would be asked or retrieved from a previous session.
  2. SB747 user clicks on the "Online" button in the GUI.
  3. SB747 loads the required network module for the chosen network, initialises it, and attempts to establish a connection with the network.
  4. If successful, the normal event mechanism via the World Database would take over and the synchronous load call would return with an OK.
  5. If unsuccessful, an error message would be returned and the network module would unload.
  6. When the user indicated his wish to go offline, the connection would be cleanly ended and the network module would be unloaded. This could be either a synchronous call or a "go offline" event.
  7. When significant network trouble would be encountered, an error message event be generated via the World Database, the connection would be cleanly ended if possible, and the network module would be unloaded.
A clear distinction between synchronous calls (always with a return value) and asynchronous events (via the World Database) could this way be maintained. Module load and unload times were well-defined, and lots of diagnostics could be generated by the specific steps in the flow.

Voice Operations

Voice is a tremendously complex subject, that I luckily do not need to touch as I can use the AVC and TeamSpeak programs to do the dirty work for me. Both VATSIM and IVAO currently use the "first ATIS line" hack to get the host name and channel of the controller's voice link to the pilots. Using standard ATIS requests, and with existing SB747 infrastructure, I could for years reliably get all station's voice data into a cache list, ready for use. Test flights of SB747 2.0.0 already indicated that this mechanism is still working correctly.

However I now need to get this data to two separate voice programs, not at the same time though. This requires a breakdown into basic operations instead into program-specific features. One central "call voice program" routine should to the dirty work of calling the program via the published command line interface. In all known cases, these programs have one of those interfaces where you may call up a second copy of the program, which will see that another copy already runs, and passes new information to that first copy before terminating itself. Ideal!

There are only two basic operations: connect to a voice channel, and disconnect from a voice channel. That's easy. So with two generic calls, and one call that knows which program is currently active, it should work.

To be continued...

Actual Work to Do

On March 13, SB747 successfully logged into IVAO using the new authentication scheme. The ToDo list still is considerable, to get SB747 ready for a true 2.0 rollout with as many features operational as possible. I've ordered this list in approximate order of urgency, but the actual processing order will definitely vary according to circumstances. I aim to get a playable SB747 as soon as possible, and will take into account that some things have workarounds (such as weather via SimWX).
  • Implement ATIS transactions (#Ax). ATIS belongs to the core infrastructure and should be supported ASAP. One evening due to infrastructural changes in SB747. DONE
  • Use IVAO voice server/channel protocol instead of brain-dead ATIS line. Playability requirement. There is no specific voice server packet in the IVAO protocol, so likely this still runs via ATIS. One evening at least, probably more as there is SB747 infrastructure to be adapted. Leave it for a weekend. DONE
  • Implement relevant Plane Info packets (-xx). To get a correct visual appearance to the other planes around. A few hours.
  • Use IVAO weather protocol. Pretty important, requires also SB747 infrastructure changes. A weekend at least.
  • Review status of Wind Variation (#DL). 15 minutes, maybe plus one hour for implementation if it does wind shears.
  • Integrate Luca's TeamSpeak interface directly into SB747. An evening, likely together with an update to the RogerWilco Daemon. HALFLY DONE.
  • Fix the livery selection using the Multiplayer Traffic Library IDs. A few hours, need people to watch "me" from outside.
  • Review implementation status of SB peer-to-peer messages (#SB). Likely nothing, 10 minutes.
  • Implement Change Flightplan logic (changes pushed from ATC). Very important for IVAO. Evening.
  • Re-implement METAR requests (not via ACARS #Ax any more). It depends on the weather system whether METAR requests are as important as they used to be in the past. If so, then need to increase this priority. One evening at least.

to be continued...

© 2010 Jeroen Hoppenbrouwers For more information, mail to hoppie@hoppie.nl