INTERNAL EVALUATION PROGRAM |
(Last Revision: August 6,2010)
Introduction
The concept of the Internal Evaluation
Program is based on the premise of verifying compliance with
safety regulations in accordance with FAA HBAT 99-19 and
Advisory Circular 120-59.
A good IEP also ensures that safe
operating practices are a responsibility shared by each
functional department within the airline. The incorporation
of internal evaluation techniques into functional department
operating practices will enhance the ability to detect
system weaknesses that can result in an unsafe condition. In
addition, the internal evaluation concept fosters
improvement on a continuing basis, rather than in response
to an inspection finding from an outside source. Better to
find and correct deficiencies before noticed by others,
particularly the FAA.
The Internal Evaluation Program is not
intended to replace other regularly scheduled company audit
functions or result in a reduction of normal FAA
surveillance programs.
Scope and
Approach
The internal evaluation program includes
all major functional departments involved in the airlines
airworthiness, flight and ground operations programs. These
departments are Flight Operations Line, Flight Operations
Technical, In-flight, Field Services, Training Departments,
Flight Safety Department, System Operations Control,
(Dispatch), and Crew Scheduling, among possible others.
Specific topics covered include applicable Federal Aviation
Regulations (FARs), company policies and FAA mandated
publications and operations specifications.
The internal evaluation method focuses on
allowing the functional manager to detect and correct
deficiencies that can affect safety or result in FAR
compliance problems. All departments should complete the
internal evaluation annually, note any findings, and take
corrective action. At the specified interval, an Internal
Evaluation Finding Report should be made which provides a
summary of departmental findings and progress in their
resolution. This also provides evidence of due diligence in
correcting deficiencies.
Evaluation
Benefits
A thorough and well supported internal
evaluation program provides several benefits, some of which
are: It enables management at the various levels within the
company to assess the effectiveness of their particular
departments
-
When accomplished on a consistent
basis it provides an excellent barometer of change in
operational performance
-
It helps build and maintain employee
awareness and interest in how their particular
function/duty effects safety
-
Effectively conducted, it promotes
improved operational performance through detailed
feedback and corrective action time tables
Key Terms
and Definitions
Objective Evidence Documented
statement of fact, prepared by the functional department.
Material may be quantitative or qualitative and is based on
observations, measurements, or tests, (questions), which can
be verified. This data is necessary to substantiate findings
or concerns and to enable management or evaluators to
determine the root causes of reported findings. Sources are:
Finding Objective evidence which
demonstrates instances of noncompliance or nonconformance
with a specific standard (such as a FAR).
Noncompliance A finding in which
objective evidence shows that established policies and
procedures are not consistent with regulatory requirements
and standards, thereby indicating potential systemic
deficiency.
Nonconformance A finding in
which objective evidence shows that a particular action was
not performed in accordance with regulatory requirements and
standards. Does not indicate a systemic deficiency.
Concern Objective evidence that
does not produce a finding, but rather, a condition that may
become a finding. Emphasizes the anticipatory aspect of the
internal evaluation program.
Classes of Findings -
-
Class A ANY
non-compliance with any FAR.
-
Class B Failure of the
certificate holder to adhere to documented company
procedures, related to specific regulatory requirements
or safety, that have been developed by the certificate
holder and approved or accepted by the FAA.
-
Class C Findings are
Opinions and Recommendations concerning safety of
flight, passenger and crew safety, safety related to
aircraft operations on the ground, or airworthiness
items.
NOTE: It is the findings that
are categorized. Category determinations do not reflect the
severity or corrective action taken to resolve the finding.
ATOS
(AIR
TRANSPORTATION SYSTEM OVERSIGHT)
The Air Transportation Oversight System
(ATOS) was implemented in 1998 as a new approach to FAA
certification and surveillance oversight, using system
safety principles and systematic processes to assure that
air carriers are in compliance with the Federal Aviation
Administration regulations and have safety built into their
operating systems. While some safety managers feel that ATOS
is being replaced by the implementation of SMS, it is still
with us and will be prominently embedded into the FAA
mandated SMS. Having an IEP address all the elements
and attributes of ATOS will go a long way in building
confidence in your FAA Certificate Management Office
Inspector team that you have a good handle on safety risk
management.
Unlike the traditional oversight methods,
ATOS incorporates the structured application of new
inspection tasks, analytical processes, and data collection
techniques to the oversight of individual air carriers. This
approach enables Flight Standards inspectors to be more
effective in the oversight of air carriers by focusing on
the most critical safety aspects of an air carrier's
operation. As currently applied, ATOS provides a systematic
process for conducting surveillance, identifying and dealing
with risks, and providing data and analysis to guide the
oversight of each carrier.
Under ATOS, an air carrier's operations
have been separated into 7 systems, 14 subsystems, and 88
underlying component "elements" which provide the structure
for conducting surveillance, collecting data, and
identifying risks or areas of concern. A model may be
referenced at
http://www.faa.gov/about/initiatives/atos/
Surveillance is effectively implemented
through two distinct types of inspection, the Safety
Attribute Inspection (SAI) and the Element Performance
Inspection (EPI). An SAI is planned for at the subsystem
level and conducted at the "element" level by a team of
inspectors to determine if the air carrier has the safety
attributes of Responsibility, Authority, Procedures,
Controls, Process Measurement, and Interfaces adequately
designed into their system element processes.
EPIs are also conducted at the "element" level, but are
accomplished by individual inspectors to determine if the
carrier's system element processes meet established
performance requirements, if the air carrier's procedures
and controls are adhered to, and if proper records are
maintained. In addition, over 2200 specific regulatory
requirements have been incorporated into the SAIs and EPIs
to ensure that air carriers are in full compliance with all
applicable CFRs.
Automated
Safety Audit Tools
A good audit management software tool
should:
- Organize all organization audit
requirements and findings into one database
- Organize and schedule audits of all
operational departments
- Provide ATOS-compliant audit
checklists that are user-controlled and easy to change
- Allow identification and data
collection on all safety issues
- Provide a central clearing house for
all audit findings and corrective actions
- Allow tracking of all corrective
actions and Safety Risk Management risk mitigations
- Provide formal work processes for
consistency throughout organization
- Provide tracking of organizational
safety due diligence on all safety issues
- Reduce audit program manpower
- Reduce the time it takes to complete
and report audits
- Provide aggregate organizational
audit and safety data analysis in customizable reports
- Provide trend analysis and real time
data reports
- Be intuitive and easy to use
requiring minimal training
- Present easy navigation paths from
the opening screen and promote easy travel to each
function
For more information on how your company
can quickly gain these advantages, please
email one of our Senior Consultants to find out more
Next Article- LOSA