criteria and an objective approach to selecting commercial real-time operating systems based on published information




старонка1/2
Дата канвертавання25.04.2016
Памер165.43 Kb.
  1   2



 CRITERIA AND AN OBJECTIVE APPROACH TO SELECTING COMMERCIAL REAL-TIME OPERATING SYSTEMS BASED ON PUBLISHED INFORMATION
Phillip A. Laplante1

Penn State University



Abstract
Matching a commercial real-time operating system to a particular application is a problem for which there is no obvious solution strategy. While factors such as estimated production volume, likely processor involved, mission criticality, cost, reliability, software support, ease-of-use, and maintainability need to be considered, oftentimes the selection is merely based on the preferences of the design team, management, or intensity of vendor marketing.
In this paper the selection of real-time operating systems is examined. A set of criteria and an associated metric are presented that rely only on published marketing information. Then, to demonstrate the application of this metric, information from several existing commercial real-time operating systems is collected and analyzed. Five hypothetical applications are then matched with the most appropriate real-time operating system, using the criteria and metric.
The contribution of this paper is a set of criteria, a metric, and a methodology for matching real-time operating systems to applications based on marketing information, without the need for intensive performance testing.
Keywords: metrics, real-time operating systems, selection criteria
1. Introduction
There is surprisingly little work on metrics for evaluating real-time systems. A literature search turned up nothing for evaluating commercial real-time operating systems (RTOS2) in journals. A number of reports are available either from workshops or trade magazines but these were difficult to find.
The dearth of published work on evaluation of RTOS is surprising because there is both an engineering and business advantage in some formalized methodology for evaluating candidate systems. In any case, some metrics that are documented for RTOS only involve scheduling features. For example, Buttazzo gives average response time, total completion time, weighted sum of completion times, maximum lateness and maximum number of late tasks [1]. Other metrics are based on extensive experimental procedures, [2], [3], [4], [5], [6]. Unfortunately, these are fairly limited in scope and the results of a given test cannot be necessarily imputed beyond the experimental platform. Hence, they provide only partial guidance in the selection of a commercial RTOS solution.
Unfortunately, unless a comprehensive experience base exists using several alternative commercial real-time operating systems in multiple, identical application domains, there are only two ways to objectively determine the fitness of a product for a given application. The first is to rely on third party reports of success or failure. These abound and are published widely on the Web (e.g. [3]). The second is to compare alternatives based on manufacturer’s published information from brochures, technical reports, and Web sites. Therefore, there should be some way to incorporate manufacturer’s published data into the decision-making process, especially when independent performance tests are unavailable and impractical to conduct.
The contribution of this paper, then, is an objective and flexible technique for comparing commercial real-time operating systems based on marketing information. This technique can be used, however, in conjunction with supplemental information from actual experience and third party reports.


    1. Build Versus Buy

A common question that is asked at the time of systems requirements specification is “should a commercial real-time solution be used or should one be built from scratch?” While the answer depends on the situation, commercial kernels3 are frequently chosen because they generally provide robust services, are easy to use, and may be portable.


While commercial RTOS provide flexibility in the number of tasks supported and scheduling discipline, there are drawbacks in their use. For example, they are usually slower than the hard-wired interrupt driven model because tremendous overhead is incurred in implementing the task-control block model, which is the typical architecture for Commercial RTOS [7], [8]. Furthermore, commercial solutions tend to suffer from featureitis – too many unneeded features are incorporated in order for the product to have the widest appeal. The run-time and storage costs of these features may be excessive. Finally, manufacturers may be tempted to make misleading claims, or give best case performance figures. The worst-case response times, which are the most informative, can generally not be known. If they are known, they are typically not published because they would place the product in an unfavorable light.
For embedded systems, when the per-unit charge for commercial products is too high, when desired features are unavailable, or when the overhead is too high, the only alternative is to write the real-time kernel. But this is not a trivial task [7], [8]. Therefore, commercial RTOS should be considered wherever possible.
1.2 Commercial Real-Time Operating Systems
There are many commercial solutions available for real-time systems, but deciding which one is most suitable for a given application is difficult. Many features of embedded real-time operating systems must be considered including cost, reliability, and speed. But there are many other characteristics that may be as important or more important, depending on the application. For example, the real-time operating system largely resides in some form of ROM and usually controls hardware that will not tolerate many faults; therefore, the RTOS should be fault-tolerant [9]. Also, the hardware needs to be able to react to different events in the system very quickly; therefore, the OS should be able to handle multiple processes in an efficient manner [10]. Finally, because the hardware on which the operating system has limited memory, the operating system must also require a reasonable amount of memory in which to run [7].
In fact, there are so many functional and non-functional attributes of any commercial RTOS, that evaluation and comparison must be a subjective endeavor. Some structure, however, should be used in the heuristic decision-making. Using a standard set of criteria provides such structure.
2. Criteria for selecting real-time systems
What makes a good RTOS? In short, it depends on the standard definition of a real-time system, namely, a system in which requirement satisfaction (logical correctness) is based on the correctness of the system’s behavior in terms of data and time [7]. In essence, the RTOS must have correct and bounded behavior under all system load scenarios.
From a business perspective, there is concern about compatibility, especially in an environment where there are heterogeneous and legacy systems. Furthermore, it is critical that the RTOS support newer or standard network technologies for flexibility of product development and maintenance.
Mainstream real-time researchers and engineering practitioners seem to agree on the following desirable characteristics for real-time systems, these are:


  • timeliness

  • design for survival under peak load (schedulability)

  • predictability

  • fault-tolerance

  • maintainability [1]

Therefore the selection criteria should reflect these desiderata.


Consider thirteen selection criteria, , each having a range where unity represents the highest possible satisfaction of the criterion and zero represents complete non-satisfaction. Recognizing that the importance of individual criterion will differ depending on the application, a weighting factor, , will be used for each criterion , where unity is assigned if the criterion has highest importance, and zero if the criterion is unimportant for a particular application. Then a fitness metric, , is formed as

. (1)
Clearly, a higher value of means that the RTOS is well suited to the application, while a lower value means that the RTOS is not well suited for the application.
While selection of the values for and will be subjective for any given RTOS and any given application, the availability of this heuristic metric, provides a handle for objective comparison, historical perspective and other uses. The following sections describe the criteria.
2.1 Minimum Interrupt Latency
The minimum interrupt latency, , measures the time between the occurrence of hardware interrupt and when the interrupt’s service routine begins executing. A low value of represents relatively high interrupt latency, while a high value of represents low latency.
This criterion is important because if the minimum latency is greater than that required by the embedded system, a different operating system must be selected.
2.2 Total Number of Tasks Supported

This criterion, , defines the most concurrent processes the operating system can simultaneously support. Even though the operating system may support a large number of tasks, this metric may be further limited by available memory. This criterion is important for systems that need numerous simultaneous processes. A relatively high number of tasks supported would result in, while few task supported would suggest a lower value for .



2.3 Memory Requirements

Criterion specifies the system memory required to support the OS. It does not include the amount of additional memory required to run the system’s application software. suggests a minimal memory requirement, while would represent a larger memory requirement.



2.4 Scheduling Mechanism
The scheduling mechanism criterion, , enumerates whether preemptive, round-robin, or some other task scheduling mechanism is used by the operating system. Round-robin scheduling is used to allow tasks at the same priority level to share the resources of the system. Preemptive scheduling allows high priority tasks to begin immediately by preempting the current task running. Most PC operating systems use preemptive task scheduling because an indeterminate amount of processes will be running, each with their own unique CPU requirements. However, when the number of processes is fixed, as in the case of many real-time systems, preemptive task scheduling is not always a more efficient algorithm than round-robin scheduling [10].
Kernel preemption is another feature that reduces response times and increases system schedulability. If the kernel is not preemptable, then a low priority task that is in the middle of a kernel service call would inherit the highest priority to the low priority task, creating a priority inversion. Kernel preemptability alleviates this situation.
If many scheduling mechanisms are supported and if the kernel is preemptable, then a high value would be assigned to .
2.5 Intertask Synchronization Mechanism

Criterion refers to the available methods the operating system has to allow processes to communicate with each other. Among possible choices are mutual exclusion (mutexes), binary and counting semaphores, POSIX pipes, message queues, shared memory, FIFO buffers, control sockets, and signals and scheduling. Each mechanism has advantages and disadvantages [11], [12]. Let if the RTOS provides all desired scheduling mechanisms. A lower value for implies that less synchronization mechanisms are available.



2.6 Context Switch time

Criterion refers to the time it takes for the kernel to save the context when it needs to switch from running one task to another. The context may include one or more of the following: PC pointer, stack pointer, heap pointer, I/O pointer, floating point registers, temporary storage registers, and any MMU registers. This information is needed so that the OS can return to the point of interruption without loss of information [13]. A relatively short context switch time would result in a higher value for .



2.7 Programming Environment

Application availability, , refers to the amount of software available (either that ships with the operating system or is available elsewhere) to develop applications to run on the operating system. For example, RTLinux is supported by the GNU suite of software, which includes the gcc C compiler and many freely available software debuggers, and other supporting software. This is an important consideration, especially when using an unfamiliar operating system. This criterion also takes into consideration the number of other operating systems that are compatible with the given RTOS.

Let if a large amount of software were available and compatible, while 0 would mean that little or none was available or compatible.

2.8 Portability

Criterion refers to the different processors that the OS supports. This is important in terms of portability and compatibility with off-the-shelf hardware and software. This criterion also encompasses the range of peripherals that the OS can support, such as video, audio, SCSI, and such. A high value for the criterion represents a highly portable and compatible RTOS.



2.9 Source Code Availability

Criterion refers to whether the code of the operating system will be available to the developer, for tweaking or changes. The source also gives insight to the RTOS architecture, which is quite useful for debugging purposes and systems integration. Settingwould suggest open source code or free source code, while a lower value might be assigned in proportion to the purchase price of the source code. Let if the source code were unavailable.



2.10 Software Support (warranty)

Criterion refers to the after-sale support a company puts behind its product. Most vendors offer some sort of free technical support for a short period of time after the sale, with the option of purchasing additional support if required. Some even offer on-site consultation. A high value might be assigned to a strong support program, while if no support is provided.



2.11 Cost

This criterion is directly related to the cost of the RTOS including the development license cost as well as the per target license costs. This consideration is critical because for some systems, the RTOS cost may be disproportionately high. In any case, a relatively high cost would be assigned a very low value, while a low cost would merit a higher value for .



2.12 Royalty Fee

Frequently, a per target license or royalty fee is charged for each delivered system. In the case where few targets are intended, this can be inconsequential. However, when a large number of targets are intended then the per target licensing fee can be considerable. This criterion, , is high if a relatively low per unit royalty fee is charged.



2.13 Networking Support

This criterion, , is based on a listing of what networks and network protocols are supported by the given RTOS. A high value for the criterion represents a relatively large number of networks supported.


3. Evaluation of Commercial Real-Time Operating Systems

A number of commercial RTOS are now examined based on the criteria introduced. Although the data is real, the manufacturer names are omitted as the intention is not to imply a recommendation of any product. These systems were selected based on both the availability of published information on the Web, and the knowledge that these systems have fairly widespread use. Of course, the scope of the study could have been expanded to consider many other viable commercial products. But the intent of this work is to provide a methodology and some examples of its use – not to completely assess the RTOS marketplace.


Since this work was done, clearly some of the systems performance characteristics may have changed. One must realize that any analysis is only as accurate as the data provided by the manufacturers at the time of the study4. This is typical of a real-world situation where there is little opportunity to test the accuracy of a manufacturer’s claim prior to making a buy decision. However, this constraint does not diminish the value of this work, which is to provide a methodology for assessment of goodness of fit for commercial real-time solutions and a particular application.
In the sections that follow, the following assumptions are made:


  • For all the sample RTOS, assume that the calculations for the number of interrupt, the minimum time that it takes, and other system analysis based on the metrics chosen are performed under the same conditions, i.e. sampling, time constraints, number of processors, and etc.




  • Maximum or minimum of tasks refers to the OS object, such as the MMU, device drivers, and other system tasks.




  • Assume that interrupt refers to “hardware interrupt”. “Software interrupts”, together with hardware interrupts and other vectoring mechanisms provided by the processor, are referred to as “ exception handling”.




  • Thread switching time is equivalent to the measurement of context switching time.

In the cases where a criterion value can be assigned, this is done. Where the criteria are “processor dependent” or indeterminate absent a real-application, assignment of a rating is postponed, and a value of * is given. This “uncertain” value is fixed at the time of application analysis. Note too that the values between tables need to be consistent. So, for example, if a 6 microsecond latency yields for RTOS X, the same 6 microsecond latency should yield for RTOS Y.


Consider commercial RTOS A. Table 1 summarizes the criteria and ratings. The rationale for the ratings are described in the following paragraphs.

.
Table 1: Summary data for real-time operating system A.




Criterion

Description

Rating

Comment

m1

Minimum Interrupt Latency

*

CPU dependent

m2

Total Number of Tasks Supported

0.5

32 Thread Priority Levels

m3

Memory Requirements

0.7

ROM: 60k

m4

Scheduling Mechanism

0.25

Preemptive

m5

Intertask Synchronization Mechanism

0.5

Direct Message Passing

m6

Context Switch Time

*

na

m7

Programming Environment

1

Compilers: Greenhills, ADS, ARM, Diab, GNU, Tools: Mentor Graphics, Greenhills, SDS, ARM, ADS

m8

Portability

0.8

PowerPC, ARM, MIPS, DSP’s, Network Processors

m9

Source Code Availability

1

yes

m10

Software Support (Warranty)

0.5

Paid phone support

m11

Cost

0.7

approximately $2500

m12

Royalty Fee

0

no

m13

Networking Support

1

TCP/IP, FTP, SMTP, SNMP, PPP, ATM, ISDN, X25, Telnet, Bootp, http-server

The product literature indicated that the minimum interrupt latency is CPU dependent, therefore a * value is assigned here (which will be later resolved as 0.5 for the purposes of evaluating the metric). Context switch time is not given and so a * is also indicated. The RTOS supports 32 thread priority levels, but it is not known if there is a limit on the total number of tasks, so a value of 0.5 is assigned. The RTOS itself requires 60K of memory, which is somewhat more than some of the alternatives so a value of 0.7 is assigned. The system provides only one form of scheduling, preemptive priority, so a lower value, 0.25, is assigned here than if other forms such as round-robin were available. Intertask synchronization and communication is available only through message passing so a relatively low is assigned.


The company provides paid phone support, which is not as “generous” as other companies, so a value of is assigned. There is a royalty cost for each unit so a zero was assigned. Finally, there is a wide range of software support for the product, including network protocols, and the source is available so values of one are given for these three criteria.
Using the same kind of reasoning, ratings tables are obtained for commercial RTOS B, C, D and E, shown in tables 2 through 5 respectively.
Table 2: Summary data for commercial RTOS B.


Criterion

Description

Rating

Comment

m1

Minimum Interrupt Latency

0.8

Less then 15 us (Typical)

m2

Number of Tasks Supported

*

CPU Dependent

m3

Memory Requirements

0.2

Less then 4 MB

m4

Scheduling Mechanism

0.5

Preemptive and round-robin

m5

Intertask Synchronization Mechanism

1

Mutual exclusion, POSIX pipes, message queues, shared memory, signals, sockets, fifos

m6

Context Switch Time

*

Dependent on CPU













m7

Programming Environment

0.75

GNU Development Tools (gcc)

m8

Portability

0.5

X86, PowerPC, MIPS, Alpha

m9

Source Code Availability

1

Open source available

m10

Software Support (Warranty)

0.5

Support available for additional cost

m11

Cost

0.5

$5000

m12

Royalty Fee

1

Yes

m13

Networking Support

1.0

Full TCP/IP Suite


Table 3: Summary data for commercial RTOS C.


Criterion

Description

Rating

Comment

m1

Minimum Interrupt Latency

1

8 microsecond

m2

Number of Tasks Supported

*

Multiple cooperating tasks

m3

Memory Requirements

*

POSIX Memory mapping, memory locking

m4

Scheduling Mechanism

0.1

Preemptive task scheduling

m5

Intertask Synchronization Mechanism

0.5

Inter-task communication through shared memory space

m6

Context Switch Time

0.5

“fast”

m7

Programming Environment

1

POSIX compliant software supported

m8

Portability

0.2

PowerPC

m9

Source Code Availability

0

Source not available

m10

Software Support (Warranty)

1

Full support available

m11

Cost

0.2

$20000

m12

Royalty Fee

1

Yes

m13

Networking Support

1

All POSIX Compliant



Table 4: Summary data for Commercial RTOS D.


Criterion

Description

Rating

Comment

m1

Minimum Interrupt Latency

*

CPU Dependent

m2

Number of Tasks Supported

1

“Unlimited “

m3

Memory Requirements

1

2K memory required

m4

Scheduling Mechanism

1

Preemptive, round-robin and time slicing

m5

Intertask Synchronization Mechanism

1

Binary, counting, and mutual exclusion semaphores with priority inheritance, message queues, POSIX pipes, counting semaphores, signals and scheduling, control sockets and shared memory

m6

Context Switch Time

0.5

14 micro second switch time

m7

Programming Environment

1

Tornado II, more then 1800 APIs available, powerful file system and I/O management, C++ and other standard run-time support

m8

Portability

1

x86, PowerPC, ARM, MIPS, 68K, CPU 32, ColdFire, MCORE, Pentium,i960,SH, SPARC, NEC V8xx, M32 R/D, RAD6000, ST 20, TriCore

m9

Source Code Availability

0.25

Source available for $120,000

m10

Software Support (Warranty)

0.8

Phone Tech support available

m11

Cost

0.1

$23,000

m12

Royalty Fee

1

Yes

m13

Networking Support

1

Full TCP/IP Suite


Table 5: Summary data for commercial RTOS E.


Criterion

Description

Rating

Comment

m1

Minimum Interrupt Latency

1

6 microseconds

m2

Number of Tasks Supported

1

32,000

m3

Memory Requirements

0.9

kernel = 6.2Kb

file management = + 32Kb

peripherals = + 200Kb


m4

Scheduling Mechanism

0.25

Prioritized, preemptive scheduling.

Equal priority tasks are scheduled on a time-sliced, round-robin basis



m5

Inter-task Synchronization Mechanism

1

Semaphore event flags, queuing libraries, binary semaphores, timing semaphores, physical semaphores, and counting semaphores.

m6

Context Switch Time

*

Dependent on Processor

m7

Programming Environment

0.5

Editor, disk utilities, file utilities, and a virtual port that allows the user to run, monitor, and switch between multiple task output screens.

m8

Portability

0.2

 68k architecture family

m9

Source Code Availability

1

Yes

m10

Software Support (Warranty)

1

Full support

m11

Cost

0.7

 $2,880 – 5,850

m12

Royalty Fee

1

Yes

m13

Networking Support

0.6

Provided by BSDnetTM, which provides Ethernet, and TCP/IP support, telnet, and ftp.

Admittedly, these assignments were somewhat subjective. Numerous arguments can be made that a higher or lower number should be assigned in almost every case. But, the idea is that there is some consistency in how the ratings are assigned so that like attributes can be compared objectively. This represents the power of this methodology over heuristic or even emotionally based approaches.



4. Matching the Application to the Operating System Using the Criteria

In order to demonstrate the use of the criteria and associated metric, a set of five hypothetical systems are analyzed to demonstrate how comparative metrics can be used to match an application to the appropriate real-time operating system. These systems are loosely described as




  • a hand-held game application,

  • a controller application for the fuel injection system of typical passenger car,

  • an animatronic robot used in the filming of a science fiction movie,

  • a navigation system for a fighter aircraft, and

  • a medical device that reduces the time needed for a medical imaging scan.

These systems were selected because they are representative of a large band of the real-time embedded application spectrum. Moreover, pedestrian familiarity with the basics of what these systems do, avert the need for complex systems requirements specifications.


To do the comparison, a tabular format shown in tables 6 through 10 is used. All uncertain are set to 0.5 where information is still unavailable for the purposes of evaluating the fitness metric . This is not the ideal manner for dealing with uncertain information [14], but consideration of this aspect is left for further study.
The result is a set of values that reflect the weighted importance of each criterion for a given application, and the known information about the candidate operating system. A quantitative selection of the appropriate RTOS can then be made.
In the forgoing discussions, tables 6 through 10 were obtained by setting up a set of spreadsheets in which the weighting factors are input in appropriately named column. Then the objective metric component values are input for the various candidate operating systems. In this way, metric M can be computed rapidly and correctly, and the weighting factors can be adjusted as necessary for sensitivity analysis.
  1   2


База данных защищена авторским правом ©shkola.of.by 2016
звярнуцца да адміністрацыі

    Галоўная старонка