An Introduction to Safety-Critical Systems

References

Definitions: Dependability

• Safety is one aspect of the more general concept of dependability.
• The standard reference to definitions in the context of dependability is J.C. Laprie et al.: Dependability: Basic Concepts and Terminology. Springer-Verlag, Berlin Heidelberg New York (1992).
  
• A short summary of the most important terms is given in the following set of slides J. Peleska: Development of Safe Systems.
  
• A more detailed discussion of terms and concepts is given in J. Peleska: Formal Methods and the Development of Dependable Systems
  
• Note that the concept of dependability evolved from computer science communities. In more traditional engineering disciplines, some of the notions defined by Laprie and his group are used in a different way. For example, engineers may use the term ``reliability'' with the semantics of ``dependability''.
  
• Furthermore, it is a fairly new observation that security is an important aspect of dependability: Traditionally, security was investigated in other communities than those dealing with safe, reliable and available systems. This is discussed in more depth in J. Peleska: Formal Methods and the Development of Dependable Systems, Chapter 1
  

Concepts: Safety Requirements Versus Customer Requirements

• Customer requirements specify capabilities of a system that are desired from the (paying) customer's perspective. They may be functional (requirements referring to data items, data transformations and behavioural aspects) and/or non-functional (e.g., requirements referring to maintainability, availability, reliability).
  
• Safety requirements specify capabilities of a system aiming at the prevention of hazards. A hazard is a situation in which there is actual or potential danger to people or the environment. In other words, a hazard is a state with a high potential for the occurrence of an accident. For example, two trains approaching or following each other on the same track segment (block) is considered to be a hazardous state of the railway network, because it is highly probable that the trains will not be able to brake in time to avoid a collision. An interlocking system shall therefore fulfil the safety requirement that at most one train is allowed to enter a track segment.
  
• System Requirements contain the customer requirements and the safety requirements. (N. Storey uses slightly different terminology, where safety requirements form an entity separate from the system requirements. This is not such a good idea, since it is generally accepted that the system requirements contain everything required for the system: It should not be necessary to consult another document in order to view the full set of things that have to be provided by the system.)
  
• Depending on the type of application, customer requirements may be
  • identical to safety requirements (e.g. requirements for a computer performing safety checks for an interlocking system),
    
  • have a non-empty intersection with safety requirements (e.g. a computer system controlling optimal throughput in a production line and checking safety conditions in parallel),
    
  • have an empty intersection with safety requirements (safety requirements are enforced by law or a separate authority, but do not contribute to the "productive value" of the system)
    
• It is good specification practice to strictly separate the safety requirements from customer requirements not related to safety: In many cases, the verification of safety requirements will be performed by authorities other than the customer, and they will not be interested in requirements without safety relevance. Also, separation of concerns during the specification process leads to better system design.
  
• Safety requirements are derived along the following lines:
  1. Hazards are classified according to their severity (e.g., neglectable, minor, critical or catastrophic hazards; see section Methods: Risk Analysis = Hazard Analysis + Risk Assessment)
     
  2. In the hazard analysis (also called threat analysis ) it is investigated how malfunction or even some functional or non-functional preliminary customer requirements of the system under consideration may lead directly or through a causal chain of possible events to a (neglectable, minor, critical or catastrophic) hazard.
  3. Safety requirements list the necessary system-specific capabilities, such that
     • catastrophic hazards are highly improbable,
       
     • critical hazards are improbable,
       
     • minor hazards occur only with low probability,
       
     • neglectable hazards occur at most occasionally.
  
  
• Remarks and Observations:
  • Complex customer requirements may be conflicting with safety requirements: They may implicitly specify that the system should perform a transition into an unsafe state. Therefore it is a major verification objective to show that the customer requirements are consistent with the safety requirements. Formally speaking, the safety requirements specify boundary conditions for the implementation of functional/non-functional requirements.
    
  • The severity of hazards is specified more precisely for specific safety-critical applications. For example, in avionic control systems, a hazard is classified as critical, if it may lead to severe damage of an aircraft or to accidents where single persons may be involved. A catastrophic hazard is one where the complete aircraft is lost and potentially all passengers lose their lives.
    
  • The qualitative probability attributes in the above list are usually quantified for a specific safety-critical application. For example, for railways a "highly improbable" event is one occurring with probability of less than 10^-12 during a operational period of 20 hours.
    
  • In general, it won't suffice just to quote in the system requirements the safety standards to be fulfilled. Instead, it will be necessary to specify how the general safety standards applicable are instantiated and proven for the specific system.
    
  • The classification of neglectable, minor, critical or catastrophic hazards is not an absolute measure; it depends on what the society is (or seems to be) willing to accept. For example, the death of five astronauts in a space shuttle explosion was considered as catastrophic, while thousands of deaths caused by car accidents per year are considered as minor hazards.
    
  • Observe that laws and standards usually do not state other dependability requirements than those safety and sometimes security: As long as the required safety conditions are not impaired, your system may be as unreliable and unavailable as you want (and can afford). The reason for this phenomenon is that governments are hesitant to provide laws and standards to prevent economic losses.
    
  
  

Concepts: Safety-related Responsibilities for PM, QA, CM, SD