John Koskinen's Responses to
Questions
from Paula Gordon
Concerning National and Global
Aspects of Y2K
Introduction
In March, 2000, approximately two weeks before his tenure as
head
of the President's Council on Year 2000 Conversion came to an end, John
Koskinen agreed to respond to a written set of questions concerning Y2K.
I submitted the following list of questions to him on March 20. I
received
his responses on March 22. I later asked for and received his
permission
to quote his responses.
My thanks to John Koskinen for so graciously agreeing to respond to
this set of questions and for allowing his responses to be quoted.
I share
this material in the hope that these questions and responses will help
focus
light on Y2K, on what happened prior to the rollover and what has been
occurring to date.
The Questions
The list includes 25 different topics and approximately 60 questions.
The questions were sent by e-mail. The following is a sample of the
form in
which the questions were sent:
1)
Incident Reports
I understand that there were 6000 incident reports received by ICC in the
first five days of the year.
~ First, is this true? Were there 6000 incident reports received
by ICC in the
first five days of the year?
~ How many more reports have been received to date?
~ Has this data been made available?
~ If it has not been made available, will it be made available?
~ If it cannot be made available because of the names of companies and
business that are mentioned, could the incident reports be made public
if
these names were deleted?
The
Responses
Mr. Koskinen's responses to the questions are interspersed
throughout each section. I mention this in order to clarify that
what follows
is not a transcript of a "live dialogue", but rather responses to one list
of
pre-submitted e-mailed questions.
What is reprinted here includes every single one of his e-mailed
responses. All of these responses appear in their entirety with the
exception of a few edits indicated by "........." These few
deletions do not
alter the meaning of his responses to the questions that were asked.
I did not correct any typos in the responses that he e-mailed me.
Other
Modifications
I have modified several of my own questions in the list that appear
below. I replaced two specific references with more general descriptors
owing to the potentially sensitive nature of the information included in
the
original question. I have also added a few specific references and
made
some references more precise. All modifications appear in brackets.
I have added "PG NOTES" after some of the questions and responses.
Some of these notes contain background material concerning the basis for
the
questions. Some of the notes include comments in response to Mr.
Koskinen's
answers. In some instances, I have also added information gathered after
Mr.
Koskinen sent me his responses.
Some Recommended
References
The reader may find it helpful to review or be aware of the following
references. These references should help put the questions and answers
in
a broader context. I have also attached appendices that include materials
that are not readily accessible.
Reference A ~ The February 29, 2000 Senate Report (see
http://www.senate.gov/~y2k/documents/final.pdf)
For a list of reported
incidents involving Y2K, see the appendix in this pdf document or
see
http://hv.greenspun.com/bboard/q-and-a-fetch-msg.tcl?msg_id=002iY6
Reference B ~ The March 29, 2000 Final Report of the President's
Council
on Year 2000 Conversion (http://www.y2k.gov/docs/LASTREP3.htm
)
Reference
C ~ January 17, 2000 Comments and Impact Ratings (Paula
Gordon) (http://users.rcn.com/pgordon/y2k/.
Click on "Comments,
Essays, and Op-Ed Pieces")
Reference D ~ Presentations by Olivia Bosch, Rosanne Hynes, and others
at the January 24 -25, 2000 Conference in Livermore,
California sponsored by Lawrence Livermore
National Laboratory (http://cgsr.llnl.gov
Click on January 24 - 25, 2000
Conference on Y2K, then click on Agenda)
Reference E ~ Grassroots Information Coordination Center Web site: This
Web site focuses on Y2K and infrastructure concerns. It serves as
a
repository of media coverage, wire service reports, government reports
and
the like. The Grassroots Information Coordination Center Web site
at
http://www.greenspun.com/bboard/q-and-a.tcl?topic=Grassroots%20Informa
tion%20Coordination%20Center%20%28GICC%29
Reference F ~ Glitch Central at http://www.ciaosystems.com/glitchcentral.htm
is a Web site
that has been tracking reports of Y2K-related problems.
Explanatory Note Concerning the Use of Some Unnamed Sources
There are many people who are in a position to provide needed information
or expertise concerning Y2K who feel little, if any inclination to do so.
In
their minds, the disincentives to being forthcoming concerning problems
or
issues relating to Y2K may far outweigh any possible incentives.
The
following is a list of some major reasons that individuals may refrain
from
speaking out or may require that they remain anonymous when they do:
~ fear of losing a job, jeopardizing a contract, or otherwise adversely
affecting one's career or business,
~ liability concerns or fear of litigation,
~ fear of other possible consequences or reprisals
~ Non Disclosure Agreements preventing the sharing of proprietary
information,
~ organizational or peer pressure,
~ a desire to avoid controversy,
~ a climate that can prove hostile to people who are candid about sensitive
or controversial Y2K-related concerns.
For these and other reasons, it has been necessary for me to promise
anonymity to a number of sources who have provided me information.
*******
John Koskinen's Responses to Questions from Paula Gordon
Concerning National and Global Aspects of Y2K
March 22, 2000
(With Notes Added 4/4/2000)
1) Incident Reports
Paula Gordon: I understand that there were 6000 incident reports
received
by the Information Coordination Center (ICC) in the first five days of
the
year.
First, is this true? Were there 6000 incident reports received by
ICC in
the first five days of the year?
John Koskinen: I love rumors. While I don't know the final
number of
events, there were nothing like 6,000. If there were, we would have
advised
people of that fact to counter the media claims that Y2K really had not
been
much of a problem.
PG NOTE (4/4/2000): Since receiving Mr. Koskinen's response, I checked
again
with my primary source regarding the extraordinarily large number of
incident reports that were received in the first days following the
rollover. This source assures me that there were many thousands of
incident
reports within the first four to five days of the year. (The previous
mention of "6000" had been a ballpark figure.) It may be possible
that Mr.
Koskinen has discounted any reports of incidents that he felt were of "no
significance". It may also be that only the incidents that
were thought by
ICC staff to be significant were brought to his attention.
(PG): How many more reports have been received to date?
Has this data
been made available?
JK: Every glitch of any significance was made public since we needed
to
establish that the world really had met and conquered a significant
challenge.
(PG): If it has not been made available, will it be made available?
If it
cannot be made available because of the names of companies and
business that are mentioned, could the incident reports be made public
if
these names were deleted?
JK: The information will be available in the Presidential records.
PG NOTE 4/4/2000 It is good to know that these reports will
be made
available in "the Presidential records".
2)
The "Powering Down" of the National and Global Infrastructure
PG: I have heard a wide range of figures concerning the extent to
which
national and global infrastructure were "powered down".
Do you have any
data on this?
JK: No
(PG): Did the President's Council urge the "powering down"
of the
infrastructure?
JK: Absolutely not. We said, in the face of rumors that some
pipelines were
going to shut, that each company needed to make that decision on its own.
All we asked was that they coordinate their plans with their local
utilities and emergency managers.
Several transit authorities noted that they would stop for a few minutes
over the rollover, but, with our encouragement, they all made those
announcements to the public.
(PG): If so, did you have reason to believe that there
would be such a
widespread effort to "comply"?
JK: There is no information that there was a widespread effort to
comply.....
(PG): Or did "powering down" efforts come as a surprise to the President's
Council? I recall at the October 7 [1999]
briefing on the chemical industry
that there did not seem to be a call to "power down" chemical plants.
As I
recall, there was concern that shutting down production could have
its own
problems.
JK: We were not surprised because there's no evidence that a significant
powering down occurred.
PG NOTE 4/4/2000: There have been numerous reports by reputable
individuals in and out of government concerning the "powering down" of
specific sectors and regions. It is something of a mystery
that others close
to what was happening should have reported "powering down" efforts, while
Mr. Koskinen knows of "no evidence that a significant powering down
occurred."
3) The ICC
PG: Did the ICC fulfill its stated purpose?
JK: Yes
(PG): Did the purpose of the ICC change after the first week or so
of
January?
JK: No
(PG): To what extent did the ICC take a proactive stance?
For instance,
we had exchanged e-mail prior to the rollover concerning how the ICC
(or
IY2KCC?) would be alerting others of problems that might be triggered a
few
hours later in another part of the world. The case in point was a
Chinese
nuclear power plant that was nearly identical to ones in France.
The idea
was that if the facility in China had problems, the similar facilities
in
France would be immediately notified. Was this done?
By the same
token, was such action taken concerning the nine or so problems in
the first
hours at Japanese nuclear power plants?
JK: There was no occasion to warn others of glitches that occurred
that
might affect them since there were no such incidents. We publicized
fully
every glitch, including those with the monitoring systems at the Japanese
plants, as they occurred, but it was fairly clear that those were
local events
--which turned out to be the case.
PG NOTE 4/4/2000: Perhaps, the question was not
as clear as it might
have been. Prior to the rollover, it had been my understanding
that when a
specific instrument, system, or piece of machinery, etc. failed,
that
potentially interested parties in other parts of the world would be notified
so
that they would be forewarned of the possibility of similar failures.
4) The (December 13, 1999) Fact Sheet on Baseline Sector Data
[See
attached appendices]
PG: Does additional material exist on baseline data beyond this fact
sheet
and did ICC make use of additional baseline data beyond the data in the
fact sheet?
JK: Anything of interest or significance was published. Our
goals were to
get that information out, not keep it to ourselves.
(PG): What use was made of the baseline data that were compiled prior
to
the rollover?
JK: We educated the press and the public about the fact that things
go
wrong everyday and they should not assume that a failure on January 1 was
necessarily a Y2K failure.
Since there were not a significant number of Y2K failures, the benchmarks
were not needed or used to make judgments about whether failures
reported were Y2K or normal occurrences.
(PG): Why in some instances on the fact sheet is the focus
solely on a few
days period of time right around the rollover? Many IT consequences
as
well as embedded systems failures can take weeks, if not months to
manifest as problems.
JK: The focus generally was on the what happens every day -- not
just
December 31. If a failure occurred later, the benchmarks were still
relevant.
As you know, the fear that glitches would occur later and would be serious
turned out to be unfounded.
(PG): Comparative data that I have seen in all high hazard sectors
show an
incidence of problems in all of these sectors at record highs when compared
with the same time frame (January and/or February) in prior years.
Has
the ICC tracked the uncommonly high number of problems involving the oil
and gas sector, the chemical sector, nuclear power plants, planes,
and
trains?
JK: No.
(PG): Has the ICC been aware of the comparisons with prior year incidents
that others have worked on? I thought that this had been the reason
for ICC
collecting baseline data, so that such comparisons could be done.
For instance, I do not recall the ICC reporting anything about problems
with refinery or pipeline problems. According to some sources, the
problems have been at an all time high since the first of the year.
Indeed I
know of a report that shows that during the first part of January,
the number
of explosions involving natural gas, methane, and propane was over 1000%
higher than the same period in prior years. (This particular report
was
based on OSHA data, Product Safety Lists, and the UN's OSHA-like data
base.)
JK: The problems you cite have not been identified by anyone
with
knowledge of the events as having anything to do with Y2K.
(PG): Did the ICC gather information concerning refinery explosions
or
unscheduled maintenance and pipeline ruptures or explosions?
JK: We collected information that anyone responsibly could establish
was a
Y2K failure. While you've been focused on these explosions, they
have not
been identified as being Y2K related, nor have they created any significant
problems for the public or the economy.
(PG): If such information was not gathered by ICC, was it gathered
by any
government agency?
JK: Not for Y2K purposes. I don't know of any other collections
or analyses,
but all the agencies reported to us -- and we to the public -- Y2K
glitches. If we could have found a Y2K problem in a pipeline or refinery,
we would have been delighted to tell the press and the public about it
to
remind them of the significance of all the work that had been done in
safely moving the world into the 21st century.
(PG): If it was not gathered, why wasn't it gathered?
PG NOTE 4/4/2000: The responses to Topic 4 are most interesting.
Mr.
Koskinen is not denying that there were an excessive number of problems
of the kinds that I have noted. He is saying that he was not personally
aware of anyone "with knowledge of the events" who was making the
connection between Y2K and the excessive number of problems that were
occurring.
5)
Concerns Regarding Possible Electric Power Problems and Actual
Problems
PG: Joe Weiss [of the Electric Power Research Institute] stated at
the
Lawrence Livermore Conference in January that the industry did not
know
how things would turn out. One person I shared this with was
quite upset
when I shared this with him. He felt that had the public been told
about this
uncertainty that there would have been more effort to prepare just in case
something did go wrong. He feels that instead the public was subjected
to a
round of Russian Roulette that we luckily won. He does not feel that
this is
the way that public policy should be conducted. Do you have any
comments?
JK: Calling this Russian Roulette is silly. I'm sorry your
friend is upset,
but if you look back at the industry and government pronouncements you'll
see that the public was, in fact, warned to prepare for much more than
actually happened anywhere.
(PG): What is the reason that the ICC has made no mention of
the reports
around the time of the rollover of the thirty some problems with electric
utilities in the U.S.?
JK: I don't know where you're getting your information. No
one has
established that any problems with utilities were Y2K. The press
was all
over the place and they did not raise them this issue either. Your
sources
seem to be operating in the dark, so to speak.
(PG): What is the reason that the ICC has made no mention of the
reports
around the time of the rollover of the scores of problems with electric
utilities that occurred around the world?
JK: Because there were no such reports other than those we reported.
You
keep assuming that we were trying not to report problems. If you
reread my
briefings, you will see that we reported issues that were so minor people
normally would never mention them, because we were concerned that the
public and the press would conclude that Y2K was nothing but hype.
(PG): It appears to be the case that a problem or a failure was not
reported
as a "reportable failure" if the problem or failure was quickly worked
around
of if there was no disruption of service owing to "workarounds" or contingency
plans. It this correct?
JK: This was not our policy. I am sure that organizations here
and around
the world did fix some problems quickly and not report them and I so advised
the press during my briefings during the rollover.
(PG): It would be helpful if all the "non reported failures" could
now be
reported so that the public would understand how extensive the problems
had been and how great the possibility of disruptions had been. Also the
public would be apt to have a far better appreciation of the extraordinary
efforts that went into contingency planning and crisis management here
and
around the world. Can that story be told now?
JK: ..... I can assure you that we've been putting out information
-- as Bruce
McConnell did at the IYCC and the Senate report did -- on any reports we
have had about Y2K failures.
PG NOTE (4/4/2000): Several comments:
~ There is an untold story concerning the extent to which contingency
planning and crisis management efforts helped ensure that there were a
minimum of problems around the time of the rollover.
~ Failures or malfunctions were not always reported as "problems" if they
were "worked around" or other steps were taken that prevented larger
problems.
~ It is interesting to note the different problems that were reported by
different sources.
~ It should also be noted that there has been inadequate attention given
to
date to the disincentives to report problems in both the public and the
private sectors.
~ More will become known concerning private sector failures that were not
publicly reported during the first three months of the year with the filing
of quarterly reports, SEC reports, insurance claims, and law suits.
6)
Nuclear Power Plants Massive and Last Minute Turning Back of
Clocks
PG: It is my understanding that considerable effort was made in December
of 1999 to convince the nuclear power industry to roll back clocks to 1972
and that this was indeed done. Is this the case?
JK: Not to my knowledge. I have heard nothing about this.
(PG) To what extent did such a roll back take place?
JK: I have no information on this.
PG NOTE (4/4/2000): It is interesting that Mr. Koskinen did
not know about
these efforts that took place in December of 1999.
(PG) If it did occur, what are the implications for future remediation
efforts
that will now be needed to turn short term fixes into long term or permanent
fixes?
7) The Chemical Safety Board's CIRC Reports
PG: Were the CIRC reports a part of the ICC's collected data?
JK: The ICC received reports on Y2K failures from all agencies of
the
government. CIRC reports are not reports of Y2K failures.
(PG): It is my understanding that the CIRC reports are not
intended to track
all problems, that they represent only a portion of problems that occur
involving the chemical sector. If that is so, was ICC relying primarily
on
industry sources for reports?
JK: We relied upon industries, state and local governments and regulators,
and all Federal agencies for reports -- along with a monitoring of all
press
reports here and around the world.
PG NOTE (4/4/2000): The CIRC reports are not intended to capture
all
problems. Owing to litigation and liability issues, insurance
claims, and
bottomline concerns, it would be more surprising than not if the reports
of
incidents that have occurred since the first of the year included any
specific or detailed mention of suspected or proven Y2K-related IT systems
problems or failures or embedded systems, PLC-related, or SCADA system
problems or failures. It appears from what Mr. Koskinen was saying that
the
ICC was relying on industries and regulators among others to suggest any
possible connections with the problems that occurred and Y2K. If
no
connection to Y2K was identified or no possible connection was suspected
or
acknowledged, then it sounds as if no report of the problem would
have been
forwarded to the ICC in the first place.
8) EPA Reports
PG: Was the ICC getting reports from EPA?
JK: Yes.
(PG) Were those reports made public?
JK: Yes, to the extent that they provided any information on Y2K
glitches.
Otherwise, they were incorporated in our generaly conclusions of no
reported problems.
PG NOTE (4/4/2000) Again, owing to litigation and liability issues,
insurance claims, and bottomline concerns, it would be more surprising
than
not if the reports of incidents that have occurred since the first of the
year included any specific or detailed mention of suspected or proven
Y2K-related IT systems problems or failures or embedded systems,
PLC-related, or SCADA system problems or failures. It appears from what
Mr.
Koskinen was saying that the ICC was relying on industries and regulators
among others to suggest any possible connections with the problems that
occurred and Y2K. If no connection to Y2K was identified or no possible
connection was suspected or acknowledged, then it sounds as if no
report of
the problem would have been forwarded to the ICC in the first place.
9) DOT's Office of Pipeline Safety
PG: Was ICC getting reports from the Office of Pipeline Safety?
JK: Yes, through the Department of Transportation which monitored
that
information.
(PG): Were all such reports being tracked?
JK: Yes, the deparments compiled all Y2K information and forwarded
it to
experts in the ICC
PG NOTE (4/4/2000) As noted before, owing to litigation and liability
issues, insurance claims, and bottomline concerns, it would be more
surprising than not if the reports of incidents that have occurred since
the
first of the year included any specific or detailed mention of suspected
or
proven Y2K-related IT systems problems or failures or embedded systems,
PLC-related, or SCADA system problems or failures. It appears from what
Mr.
Koskinen was saying that the ICC was relying on industries and regulators
among others to suggest any possible connections with the problems that
occurred and Y2K. If no connection to Y2K was identified or no possible
connection was suspected or acknowledged, then it sounds as if no
report of
the problem would have been forwarded to the ICC in the first place.
10) Sectors in the US and Abroad That Were Not Fully Remediated
PG: Little has been said concerning those sectors that did not fully
remediate. If Iraq did in fact take a fix on failure approach to
its oil
and gas sector, then it should be in dire straits and in desperate
need of
replacement parts. Indeed, some are saying that Iraq is in desperate
need
of replacement parts.
Some in the oil and gas sector in the U.S. had stated in 1999 in SEC
filings and elsewhere that they were planning to fix on failure.
Could the
increase in problems in the U.S. be related to Y2K and embedded
systems-related problems?
JK: No one in authority has established this -- or even maintained
it.
PG NOTE (4/4/2000): For coverage concerning the UN and Iraq's
requests
for equipment, see the attached appendices. The International
Energy Agency
(IEA) in its report of May 1999 (also in the attached appendices)
anticipated that non-remediated embedded systems could result in refinery
problems.
11)
Force Majeures in the Oil and Gas Sector Since the Rollover
PG: Has the ICC made any mention of the four oil and gas sector-related
"Force Majeures" that were declared in various parts of the world
since the
rollover? One of these has been in the U.S. Was no connection
seen
between these and Y2K?
JK: No one with responsibility has maintained that these had anything
to do
with Y2K.
PG NOTE (4/4/2000) Force Majeures are extremely rare. To have
four
declared within such a short period of time would seem to be worthy of
investigation.
If no one at the Department of Energy, the Federal Energy Regulatory
Commission (FERC) or the IEA was actively looking into the possible
connection between Y2K and these Force Majeures, then there would have
been
no report forwarded to the ICC in the first place. It seems quite
unlikely
that ICC would have made such a determination independently. Again,
owing
to liability and litigation issues, insurance claims, and bottomline
concerns, it would be more surprising than not if the reports of the four
Force Majeures included any specific or detailed mention of suspected or
proven embedded systems, PLC-related, or SCADA system failures. It
appears
from what Mr. Koskinen was saying was that the ICC was relying on industries
and regulators among others to suggest any possible connections with the
problems that occurred and Y2K.
12).... Problems Being Reported [in One of the Largest Departments of
the Federal Government]
There have been reports concerning large numbers of problems.... [in one
of the largest departments of the government]. Some of these problems
appear
to have fallen into the category of "non-reportable failures".
Unless
there is a security concern, it would seem helpful to bring such problems
to
the attention of policymakers, since [the problems] appear to [have
significant] implications. The government's intelligence agencies
should
surely be aware as well.
JK: We received no such reports and are not aware of any such failures.
PG NOTE (4/4/2000): In the original version of this question
that I sent to
Mr. Koskinen 3/20/2000, I had identified the department in question.
This
may well be an instance of the failure of "bad news" to travel upward and
reach those in key roles of responsibility. I am puzzled that in
this
instance he expressed no interest in checking out the information I provided
him.
13)
Ramping Down and Apparent Phasing Out of Y2K-Related Federal Efforts
PG: At the present time, it is my understanding that all Federal
agency Y2K
efforts, including intelligence agencies and oversight efforts of
Congressional Committees, have been or are in the process of being phased
out. I don't know the extent to which the General Accounting
Office will
keep a focus on Y2K. The Manufacturing Extension Partnership Program
of
NIST is scheduled to be phased out in June. The only exception that
I know
of at this time is the "informal" effort that is to continue under the
aegis
of a Deputy or Assistant Director of OMB. Is this correct?
JK: I don't know that any of this, other than the ICC and the President's
Council are closing down, is correct.
PG NOTE (4/4/2000): I find this response puzzling as well.
(PG): If it is correct, who will be making sure that short
term fixes will be
replaced by permanent fixes? Who will be taking a long range view
and
considering the lessons that are being learned from Y2K and embedded systems
challenges.
JK: Those running organizations, in the public and private sectors,
including
CEO's and CIOs, are responsible for maintaining the integrity of their
operations.
PG NOTE (4/4/2000): To the best of my knowledge, the resources and
the
assemblage of talent focused on Y2K have been or are being dismantled
throughout the Federal Government, including the Congress. There
seems
to be little possibility that there will be support for any significant
ongoing efforts. Unless the General Accounting Office continues to
focus
on Y2K, there seems to be little or no chance that there will be any ongoing
assessment of Y2K efforts and impacts or any monitoring and assessment
of
current or future Y2K-related problems. This throws into question
the
ongoing efforts that are needed to identify and address current and future
problems. It also throws into question the completion of ongoing
efforts
that are needed to complete the remediation of partially or temporarily
remediated systems.
14) Challenges
in Addressing Continuing Y2K and Embedded Systems
Concerns within Agencies and Departments
PG: It has come to my attention that in one of the largest department
of the
Federal government, it has become politically incorrect to talk about Y2K.
Yet in this same Department, many non mission critical systems have yet
to
be remediated. Problems are talked about as "computer problems".
How is
it going to be possible to continue to dedicate expertise and resources
to
ongoing remediation problems in such a climate? Also, if those who
have
the expertise are not long with the agency or have been scattered
to
different parts of the agency, how will follow up work get done and how
will on going challenges be met?
JK: I am not aware of any agency with such a problem.
PG NOTE (4/4/2000): This comment is quite perplexing.
15) The Ongoing Role of OMB in Tracking Y2K and Embedded
Systems Problems
PG: Who will be in charge? Will this be a part time responsibility?
What will be the nature of these efforts? Will ongoing efforts
include the
tracking of what is currently an abnormally high incidence of problems,
explosions, and accidents, etc. involving the following sectors: the oil
and
gas sector, the chemical sector, nuclear power plants, planes, and trains?
Will problems involving water and sewage systems also be tracked?
Will
the progress be tracked of those sectors that did not remediate prior
to the
rollover?
JK: No one has found any indication that embedded chip problems resulted
in any significant problems thus far. Going forward, organizations
remain
responsible for their operations.
PG NOTE (4/4/2000): To the best of my knowledge, two persons from
the
Federal government with embedded systems expertise participated in the
November 9, 1999 meeting on embedded systems convened the
President's Council. A half dozen or so individuals from the
private sector
who had significant hands-on embedded systems expertise were also at that
meeting. When Mr. Koskinen says that "no one has found any
indication
that embedded chip problems resulted in any significant problems thus far",
several questions come to mind: Does "no one" mean no one in
government? Does "no one" means "no person with expertise in
embedded
systems"? If it means "no person with expertise in embedded systems",
I
know persons with embedded systems expertise who would disagree.
The
meaning of his statements may also turn on what he means by "any
significant problems". There have been numerous problems involving
loss
of life, property damage, and major ecological damage, all of which
have
either been connected to Y2K or have been strongly suspected of being
Y2K-related.
16)
Expertise in Government Regarding Embedded Systems
PG: Now that Gary Fisher of the National Institute for Standards
and
Technology is no longer working on embedded systems concerns at NIST,
who at NIST or elsewhere is going to be providing expertise on this matter
to
OMB?
Who else in the Federal government can OMB call on for expertise in
following these problems and conferring regarding policy?
I have not as yet been able to identify anyone as yet at the Department
of
Energy who is familiar with the prediction that embedded systems failures
could lead to an increase in refinery outages. The IEA report
of May 1999
for a discussion of the fact that the failure of embedded systems could
indeed result in refinery problems.
PG NOTE (4/4/2000) The IEA report is quoted in the attached appendices.
JK: As noted, you and a few others are the only remaining people
who still
are waiting for the fabled embedded chip disasters to occur. The
consensus
is that, fortunately, the problem was overstated and has not resulted in
any
major problems.
PG NOTE (4/4/2000): Embedded systems failures have occurred.
They
have not been widely acknowledged as yet as being problems relating to
Y2K. This was true before the rollover and it continues to be true
now. I
am not "waiting for embedded chip disasters to occur."
From my vantage
point, embedded systems failures are continuing to occur. The
number of
"predicted coincidences" grows daily. The increasing
number of incidents
of reported problems should not come as a surprise since many embedded
systems in a wide range of sectors were either not remediated or not
adequately remediated.
Mr. Koskinen formally conferred with a group of embedded systems experts
was on November 9, 1999. The intent of the meeting was to see
if
consensus could be reached regarding a number of issues involving
embedded systems, issues that had been vigorously debated for well over
a
year. The subsequent statement released by Mr. Koskinen, along with
a
related press release are included in the attached appendices. Additional
material on embedded systems is also included in the appendices.
If embedded systems experts helped guide post-rollover assessments, it
would be helpful to find out about their perspectives and their areas of
expertise.
17) Plane-Related Issues
PG: Is there anyone at National Transportation Safety Board (NTSB)
who
is looking into the possible connection between problems with the
automated systems of the MD 80 and 90 series of planes and the problems
that this series of planes has experienced? I spoke with the Chairman
of
the NTSB in early March and he indicated that this was not an angle that
was being explored by the NTSB at that time. I do not know if that
has
subsequently changed.
JK: Y2K has not been implicated in any of those issues -- no one
with any
real knowledge ever could find a Y2K problem in airplanes that threatened
their ability to fly.
PG NOTE (4/4/2000): A very long list of known and suspect
problems
could be cited. Some relevant new reports are noted here.
These are
from main stream media sources that are posted on the Grassroots
Information Coordination Center Web site at
http://www.greenspun.com/bboard/q-and-a.tcl?topic=Grassroots%20Informa
tion%20Coordination%20Center%20%28GICC%29
~ In the early hours of the rollover two planes in Europe developed
the
same problem within a half hour of each other. Both were grounded
as a
result until the problem was corrected.
~ Problems in the MD 80 and 90 series have occurred in abnormally
high
numbers during the first months of the year. Problems
with this series of
planes continue to occur to this day. The number of MD 80 and 90
series
problems during this time period far exceeds the number of problems
occurring in comparable time periods in previous years.
~ A back up computer system problem was involved in the serious problem
that another aircraft experienced in early March.
~ Another series of planes had electrical problems as of 4/4/2000.
It would not surprise me if no one at the NTSB or the FAA has as yet begun
to look seriously into a possible Y2K connection and the exceedingly high
number of problems involving planes that have occurred since the beginning
of the year. If no one has been looking into such a connection,
it may only
mean that no software engineers with the necessary expertise have as yet
come forward or none have as yet been called on to confer concerning the
possible connection between Y2K, electrical and automated systems
problems or failures, and the problems that have been occurring.
As with
other matters that involve complex technical issues, it can be hard
for
officials who do not have that particular technical expertise to identify
those
who do, seek their counsel, benefit from their assessments and advice,
and
incorporate what they learn into decisionmaking, policymaking, and problem
solving processes.
18)
Short Term Fixes in General
PG: Is anyone in the Federal government looking at the issue
of what
needs to be done to ensure that all the various kinds of short term fixes
that
were implemented will be replaced by permanent fixes?
JK: CIOs. [Chief Information Officers]
(PG): With the disbanding of offices and teams that addressed
Y2K issues,
who will be overseeing and driving the remediation on the non mission
critical systems that were not remediated prior to the rollover.
JK: CIOs.
19)
Your Comments Concerning the Valued Role Played by Those
Who Raised the Concern of the Public Regarding Y2K
PG: In late January in a [State Department press release
dated January
27] you were quoted as saying that you were of the opinion two years ago
that it would be "the end of the world as we knew it" if the necessary
remediation was not done in time. You gave credit to those who raised
the
alarm for helping to focus attention on the problem.
Is this an accurate
restatement of what you said? If not, would you clarify
what you meant?
PG NOTE 4/4/2000 His words in that interview were as
follows:
"It was clear two years ago to me after talking with a lot of experts,
if
nobody did anything else beyond what they had already done up until two
years ago, that the world as we knew it would end."
The January 27, 2000
transcript of the interview is in the attached appendices.)
JK: .....I said that many times and continue to believe it.
However, I
think it is now time to declare victory and move on. Claiming
that the
world is still about to end casts doubt on the good work done before.
PG
NOTE (4/4/2000) During the months prior to the rollover,
my impact
ratings regarding the possible long term impact of Y2K were provisionally
between a 5.5 and 9.5. This estimate was contingent upon the extent
to
which the public sector, as well as the private sector succeeded in
efforts to
minimize impacts. In my January 17 Comments piece,
I offered my
impact rating for the first quarter of the year only. That rating
was
provisionally between a 2.5 and 5.5, this time depending on several
factors,
including the severity of fuel shortfalls. (http://users.rcn.com/pgordon/y2k/,
click on "Comments, Essays, and Op-Ed Pieces"). (The 10 point
impact
survey scale referred to here is described in Part 1 of my White Paper
on
Y2K at the same URL just cited.)
I have come to understand that much remains to be publicly disclosed
concerning all of the efforts that went into minimizing Y2K impacts.
This
includes unanticipated as well as unpublicized actions that were taken
prior
to the rollover. In some instances such actions were taken as late
as
December. It also includes the extensive contingency planning
and crisis
management efforts that helped minimize problems here and abroad.
I
hope that far more become known concerning the efforts that were made.
My intention is certainly not to cast doubt on the extraordinary
accomplishments of the thousands of people here and around the world to
avert major problems.
I have come to understand that, for a variety of reasons, the problems
that
have occurred and are still occurring are not being acknowledged as being
Y2K-related or possibly Y2K-related. Since official government monitoring
efforts of the sort needed are not in place at this time, it will not be
an easy
task to assess what has happened and what is happening. In addition,
there is no assurance that ongoing problems are being adequately addressed
now. There is no assurance that they will be adequately addressed in the
future. I would also like to see the continuation of efforts
that are
still needed.
I think that there is a chance that the evidence concerning the nature
and
scope of the problems that have occurred and of problems that are ongoing
will become more widely acknowledged as time goes by. There
appears to
be a whole host of legal, business, social/psychological, organizational,
economic, and even political reasons that to date have been serving as
barriers to such acknowledgment.
(PG): If this is an accurate rendering of what you said in January
of this
year, I wonder why you didn't say it earlier? [The reference
here goes back
to Mr. Koskinen's statements quoted in the January 27 State Department
press release that were noted at the beginning of this section.]
JK: ....... I made comments like that from the start. I continually
noted
that my disagreement with those thinking the world would come to an end
was
not a disagreement about the magnitude of the problem but with their view
that the problem could not be solved -- or could not be solved without
a
national or world-wide declaration of emergency with the public advised
to
be prepared for weeks or months of problems. I thought then -- and
was
proved correct -- that we could and would solve this problem if we could
organize the government, the US economy, and countries around the
world to deal effectively with the issue. That was done.
PG NOTE (4/4/2000): Worst case Y2K scenarios involving a simultaneous
convergence of infrastructure disruptions and technological disasters,
now
or in the foreseeable future, have been averted. This
is certainly a great
and most welcome blessing. The fact that worst case scenarios have
been
averted is not however tantamount to "solving the problem".
Problems of
a lesser magnitude and less daunting character remain. Not
all of the wide
range of problems that remain are being openly acknowledged and
addressed. Some in fact are not being acknowledged or addressed at
all.
It seems to me that a scenario is unfolding in which neither the public
nor
the private sector is doing all that needs to be done to complete the work
that was begun. Neither sector appears to be taking steps to forestall
problems that could yet emerge over the next year or more.
Neither appear
to be focusing adequate attention on what is happening now.
Problems
continue to occur. They are not receiving adequate attention.
They are not
being adequately addressed. A scenario is continuing to unfold
that is
costly in terms of loss of life, public health and safety consequences,
environmental impacts, and socioeconomic impacts, certainly not as costly
as the worst case scenarios, but nonetheless costly. The public
and private
sectors appear to have declared victory prematurely. In doing
so, support
has been withdrawn from ongoing efforts that are needed.
Challenges
remain even though the dimensions of the problem have been greatly
reduced. If these Y2K-related challenges that remain are ignored
or fail to
be adequately addressed, needless losses will continue to occur and
needless harm will result. The apparent reluctance to deal
with ongoing
problems augurs poorly for our ability to deal with future challenges and
threats that may prove even more daunting than Y2K.
20)
Ongoing and Future Problems Involving Technology
PG: As corroborated by the notes of some closed door meetings involving
the President's Council in December 1998 and January 1999, it has been
concluded by some that the Council was consciously trying to keep the full
potential seriousness of Y2K from the public and to do so through a
public
relations campaign that was designed with that purpose in mind.
If that
is an accurate description of what actually happened, then these questions
follow:
JK: ....... you know that is not an accurate description of
those meetings or
our policies......
PG NOTE (4/4/2000): My perception of the direction that the
Council's
public information efforts were taking was corroborated by my reading of
the
official notes of the December 16, 1998 meeting of the Council. These
notes can be requested under the Freedom of Information Act. The
document is entitled President's Council on Year 2000 Conversion
Meeting
Minutes. The meeting was convened by the Chair of the President's
Council at 2:05 p.m. on December 16, 1998 in Meeting Room E of the
Federal Reserve Building, 20th and C Streets, NW, Washington, DC.
Perhaps these minutes will also be made available in "Presidential papers".
This meeting included a discussion concerning "ways to work with the
media". The meeting minutes were widely distributed in the
early part of
1999. In this way the substance of that discussion became
known to many
people outside of government. An attendee of the meeting on
realizing that
I had seen the minutes confirmed the substance of this discussion.
My perceptions were also confirmed when I heard some plenary panel
presentations at the Second Global Y2K National Coordinators Meeting at
the United Nations in New York on June 22, 1999. These presentations
were by representatives from two extremely prestigious media
organizations. The presenters openly noted their adoption of
the approach
to Y2K and public information that the President's Council had been urging.
Additional perspectives concerning the nature of the Council's approach
to
public information efforts can be found in a paper that focuses on the
health
sector and the pharmaceutical industry issued by the Center for Y2K &
Society in March, 2000. (Http://www.y2kcenter.org).
(PG): If and when the public comes to realize the nature of
the public
relations campaign that was carried out, will they loose faith in their
government?
Will the public believe their government in the future if they come to
believe that in the recent past their government chose to only partially
inform them concerning the possible threats that we were facing?
What will the implications of such "managing of perceptions" be for
addressing future problems that constitute a threat to the public?
If you had it to do all over again, what would you change, if anything,
concerning the Council's apparent efforts to shape public perceptions
concerning the nature and scope of the problem?
JK: Absolutely not, since your assumption that we were somehow
nefariously shaping perceptions is incorrect. We spent significant
resources
trying to bring facts -- not assertions -- to the public, comfortable that
they
would respond appropriately to those facts. We did and they did.
In the
end, the polls all showed that they believed the reports from federal,
state
and local governments and individual critical infrastructure companies
that
they were ready for Y2K. And, in the end, to the dismay of some diehards,
those reports and facts turned out to be absolutely correct. I think
that's what the public will remember about Y2K.
21) Small Size of the Staff of the Council (11 as of December 1999?)
PG: If you had it to do all over again, what would you change?
JK: Not one body.
(PG): If you had it to do all over again, would you include
technical experts
on your staff?
JK: No.
22) General Overall Approach
PG: If you had it to do all over again, what would you change,
if anything?
JK: Nothing, including all the time I've spent discussing the issue
with those
who disagreed with our approach and criticized us vigorously, even though,
after the fact, we were right and they were wrong. It was an important
dialogue and I'd do it again, as shown by the time I'm taking to respond
to
these questions.
23) Insurance Claims and Law Suits
PG: Has the Council or the ICC been tracking insurance claims and
law suits
relating to Y2K? I heard in January that a reinsurer for oil
refineries
had an unprecedented number of claims in the first weeks of the year.
Have
you heard similar information?
JK: No.
PG NOTE (4/42000) I based my question in part on a communication
that I
received on January 23 from the head of a software engineering company.
He told of a long time associate of his who is a reinsurer who had over
100
incidents reported in the first 20 days of the year that the reinsurer
expected
would likely "end up being claims". The reinsurer noted that
this was
"atypical in the extreme." Normally this company had only one or
two claims
for the month of January with annual totals running near 40 only "in a
bad
year". According to my contact, the firm reinsures industrial policies,
primarily "manufacturing, fuel, distillates, and transportation".
It seems
important to track publicly available information concerning insurance
claims
along with related litigation, as a means of increasing understanding
concerning the nature and extent of Y2K-related impacts. Such understanding
is needed in order to assess past efforts to address Y2K and
inform current and future efforts.
24) Specific Report of the Potential for a Chemical Plant Explosion with
Major Environmental Consequences in [Another Country]
Months ago [August 3, 1999], a copy of an e-mail was sent to me in error
by
someone [in the Federal Government]. I received an e-mail from you
the
same day asking me not to pass that e-mail on to others owing to the
sensitivity of the contents. I understand that millions of
dollars went into
doing the necessary remediation in [several plants] and averting
a major
problem. I wonder if it is now possible to talk about this problem
publicly. If the name of the country should not be mentioned, can
the part
of the world
be mentioned and some specifics about the seriousness of problem that
had been averted? I think that information concerning such problems
can
be very convincing to those who doubt that the work that was done was a
worthwhile expenditure of time and money. I think that information
about
such problems is also important for the public, the media, and public officials
to know, so that they will more fully appreciate Y2K related threats and
challenges.
JK: It was not clear then and is not now whether there was a major
Y2K
threat in that plant.
PG NOTE (4/4/2000): The problem involved several plants located
in a
heavily populated area. The problem was acknowledged prior
to the
rollover as being Y2K-related. This statement concerning
the situation is
not in keeping with my reading of the "sensitive" e-mail that was
inadvertently sent to me or with other information that came to my attention
several months ago.
25)
The Role that Multinational Corporations Played in Minimizing
Infrastructure Problems in Other Parts of the World
PG: There seems to be an untold story here. I hope that the
role that
corporations played can be made public. They certainly deserve a
great
deal of credit. Will there be some focus on this in your final
report?
JK: You are very correct here. Not only do they deserve credit,
but their
work around the world with their own facilities, with host governments
and
through information sharing with competitors was a significant part of
the
reason why the rest of the world did so well. We'll try to bring
that
point to people's attention in our report and I appreciate your focus on
it.
PG NOTE: On page 22 of the Council's March 29, 2000 Final Report,
the
following brief statement is found: "And in many industries,
large
multi-national companies actually worked directly with their local
counterparts and host countries to fix basic systems."
ENDNOTE
Our viewpoints concerning Y2K and the past two years obviously differ
greatly in many ways. Because of these differences, I appreciate
even more
the time that John Koskinen took to respond to the questions I sent him.
This exchange has been helpful to me and I hope it will be of help to others
who are continuing to try to make sense of what has happened and what is
happening now. I hope as a consequence that more positive energy
might be
directed toward ongoing efforts that are needed. I hope that people
will
become acquainted with sources of information and other materials that
will
help advance their understanding. Increasing understanding on all
fronts
seems key to increasing our capacity to address current as well as future
challenges.
**************************************************************************
*******
APPENDICES
1) Transcript of State Department Interview with John Koskinen
27 January 2000
Transcript: What Happened to Y2K? Koskinen Speaks Out
http://www.usia.gov/cgi-bin/washfile/display.pl?p=/products/washfile/latest&
f=00012704.glt&t=/products/washfile/newsitem.shtml
(Administration Y2K coordinator assesses global remediation) (4,650)
The costly effort undertaken in the past two years to deal with the
Year 2000 computer problem prevented massive disruptions in systems
and services during the date rollover into the new millennium,
according to White House Y2K coordinator John Koskinen.
Koskinen, Chair of the President's Council on Year 2000 Conversion,
said in a January 18 interview in Washington that the relatively
problem-free date change that occurred is an indication not that the
Y2K problem was not serious, but that the work devoted to fixing
thousands of computer systems worldwide was successful.
Koskinen said the absence of serious Y2K disruptions in developing
countries, where remediation efforts had lagged behind those in
industrial countries, is explained by the less intense reliance in
those countries on digital technology, and by the fact that they were
able to apply the lessons learned from dealing with the problem
elsewhere.
Koskinen spoke with the Office of International Information Program's
Paul Malamud about the smooth transition into the year 2000, and the
work that made it possible.
Following is a transcript of the interview. In the transcript,
"billion" equals 1,000 million.
(begin transcript)
Q: January 1 has come and gone, and reports show that there were fewer
disruptions of computer operations and infrastructure, on a global
basis, than some had feared. In retrospect, do you feel the advance
publicity and the large amount of money that went into fixing computer
systems worldwide was overblown? Could this have been handled by
smaller "fixes" performed on an ad-hoc basis after January 1?
A: I think a lot of people did do it in an ad hoc way, at the end, and
seem to have gotten through it well. However, for organizations using
large information technology structures there was no way they could do
it at the last minute.
The major banks around the world worked on this for several years
together, because you are talking about organizations that have
millions of lines of software in code that had to be fixed. In fact,
one of the reasons that people thought the world, as a whole, was
going to have difficulty was that it takes so long to work through
those big systems.
You have to distinguish governmental organizations and private-sector
companies that had major software problems from organizations that had
more straightforward information technology challenges. I think what
happened was that some smaller organizations and governments have less
reliance on complicated systems, and therefore, a lot of their systems
either were not significantly affected by Y2K or they could take care
of those in a relatively short period of time for relatively little
money.
When people started working on Y2K no one knew exactly the full impact
of potential failures involving large networks of computers. In
addition, no one knew where in power plants, telephones systems,
chemical plants, date-sensitive "embedded processors" might have a Y2K
problem or not. My favorite example is elevators. Two or three years
ago, the assumption was that elevators were at risk. There was concern
that some elevators -- if they were dependent on date-sensitive
computer chips -- might malfunction. But after about a year of
testing, it turned out elevators did not have a problem. This meant
that if you were a country or company that started your Y2K
remediation efforts late in the game, you learned from the experience
of others that you didn't need to be very concerned about elevators.
And the same in chemical plants. It turned out there are only
relatively a small number of critical systems in a chemical plant.
The U.S. Chemical Manufacturer's Association and the Environmental
Protection Agency issued a brochure in the middle of 1999 that said
"These are the systems that are at risk. If you are using these, this
is how to fix them; if you are not using these, you are probably in
pretty good shape." So what happened was that as a result of a lot of
good work, the countries and organizations that started later had the
benefit of all that background and that research and information which
was fairly freely exchanged; so that as they moved into late 1999,
they could actually focus on things greatly at risk.
But then, turning it around, if everybody had waited until early 1999,
I think the people who run the major banks around the world and
similar large institutions would tell you the Y2K fix would never have
gotten done. In the case of the federal government, for instance, we
started in 1995 in a coordinated way -- some U.S. government agencies
began their Y2K remediation efforts even before that -- and people
were working into the middle of 1999; four years later still working
on their systems as fast as they could. So the reason a lot of serious
computer programmers thought the world would never make it was because
of the magnitude of the challenge.
Now could there have been less hype around the edges of the issue with
some people saying the world was going to come to an end because of
Y2K? We had a lot of difficulty over the last year and a half
convincing people that progress was being made. The federal government
prediction was that, in fact, there would be no major failures here or
around the world, failures impacting entire nations. We also felt
there would only be scattered outages in the United States; but that
was seen as a minority view by some.
So there was a certain amount of press coverage and hype about whether
or not the problem could be solved that probably we could have done
without. Fortunately, however, the public did not overreact, which was
our concern. And to the extent that publicity about the Y2K issue got
more people in the last six to nine months to really focus on the
problem, I think it probably helped us come to a very successful
conclusion. I don't think there is anyone who worked anywhere around
the world on the problem who thinks that it was not a major problem.
There is no bank I know, there's no power company I know, there's no
telephone company I know -- I talked to a lot of them -- who feel that
they wasted their time or their money, or if they had spent just fifty
percent less they could have done just as well. I think all of them
looking back on it are very pleased that they got through without any
difficulties.
Q: It may be true that the time and financial resources spent
reprogramming computer systems were well worth the sacrifice. However,
there was also concern about "embedded chips" -- that is those
computer chips that direct the operations of machines and consumer
appliances. There was an assumption they might be date-sensitive and
malfunction on January 1, 2000. Yet, there have not been many reports
of problems. Why not?
A: Well, what happened fortunately is most embedded chips turned out
not to be date sensitive. There are 30-50 billion out there. When I
started this job a couple of years ago, I fondly referred to them as
the growth industry of the problem, because people had begun to worry
about them, yet there was no way you could get anybody to tell you the
answer. I met with manufacturers of various parts of the chips, the
chip manufacturers, the people that put them together, power
companies, telephone companies -- nobody knew the extent of the
potential problem.
The upshot was that (a) a lot of work had to be done investigating
embedded chips, and (b) a lot of people became concerned that this
would be a major issue. The advantage of the issue, however, was it
got people to look beyond pure information processing. Everybody knew
that banks, insurance companies, financial institutions, payroll
systems were date sensitive, because they calculated how old you were,
how long you had been working, what day of the year it was. People had
not spent as much time taking a look at what went on in other kinds of
operations: oil refineries, power companies, power plants, etc.
Fortunately for the world -- and I think one of the reasons you did
not see major infrastructure failures -- is the chips themselves
generally turned out not to care what date it was. But what we did do,
because of the focus on embedded chips, was look at control systems,
which are basically software or computers that run operations. So if
you go onto a plant floor, you go onto a ship, you go into an oil
refinery, what you see increasingly is people sitting at computers
running the place. They are getting information from all those
embedded chips and it's all coming into then a computerized process.
So the reason, for instance, that airports had a problem with runway
lights was not because the lights themselves had embedded chips in
them that had to care about what date it was, but the chips in the
lights fed into a control system that set the cycling for the lights,
and that control system cared what date it was. So the bottom line
was, embedded chips turned out to be much less of an issue than people
worried about: once you could find the control panels, you needed
simply to update or check those. And, of course, these issues only are
relevant when sophisticated control systems are in use.
As we became familiar with the issue, we began to appreciate the
extent to which technological development varies throughout the world.
A lot of operations crucial to the functioning of industrial
infrastructure turned out to rely on manual or analog, rather than
digital, controls. It turned out a lot of the power companies and
telephone processes around the world were, in fact, not affected by
the embedded chip problem, which is why those countries had to spend
less and also why they had less difficulty.
But even in the United States and England and places where they have
very complicated systems, because they paid attention to them early
on, they were able to replace the switches, replace the control
systems wherever they needed to, to make sure they could continue to
run them. I think we got lucky in the sense that it turned out the
potential for the chip itself to stop the operation was relatively
minor. The risk turned out to be again back in the software control
processes, but it was important to find those to make sure that smart
building systems, card access systems, plant control systems in those
computers were checked. Because up until that time people were only
looking at their financial management systems.
Q: Some press reports estimate $200 billion was spent worldwide on
preparing for Y2K. Do you believe that is an accurate figure?
A: I think that's liable to be a more accurate estimate than the $600
billion number you see. This problem has been unique. It has been
global. The early estimates were that $300-600 billion would have to
be spent. That range itself gives you an idea that those are pretty
much guesses.
We are very confident we know how much the federal government spent,
which was $8.5 billion. The Commerce Department last fall did an
analysis of all the available reports of actual expenditures, and
estimated that in the United States the federal government and others
spent about $100 billion to remedy the Y2K problem. We estimate that
that's probably close to half of what the world spent, so that's where
the $200 billion comes from. That's the lowest number you'll hear.
Everybody's still talking about $3- 4- 500 billion. I think those
numbers do not correspond to reality. But even if it is only $200
billion, that's a lot of money.
Q: Did the Y2K remediation process turn out to be a financial bonanza
for computer engineers, consulting firms, etc., who were called in?
Some have suggested they may have had a stake in emphasizing the
seriousness of the problem.
A: No, I think actually if you look at it, at least in the United
States, a lot of corporations and certain federal agencies did the
work themselves, with their own staffs. There clearly were consultants
and people willing to work on the outside, and one of the concerns
when I started this job was there wouldn't be enough programmers
available anywhere to be able to deal with the problem. The shortage
of programmers never turned up. This was, in part, because people got
better at figuring out how to fix these systems with windowing
techniques and other technical fixes and partially because as work got
done, people doing that work were freed up to work on other systems.
Although it's hard to pin down the statistics, I think a significant
amount of the work was done internally, in many places.
A significant amount of the money spent to remedy the problem went for
upgraded equipment. Some people say that this was all a plot for all
the information technology companies to sell more stuff. The truth is
more subtle. Many of the companies that produce information technology
over time provided free computer software "patches" designed to thwart
the Y2K bug, or other kinds of free upgrades or information. When
questioned about Y2K, the answer from these companies wasn't
necessarily "Buy a new one of our things." The answer was in three
categories: either "It's okay," or "It's okay with a fix that we'll
provide to you -- either sell it to you or give it to you," or "It's
too old and we are not servicing it anymore and it doesn't work and
you have to get a new one."
I think what happened with a lot of companies, and where a lot of the
money was spent, was they looked at old legacy systems and decided
that since they were going to replace those systems sometime in the
next two to three years anyway, they might just as well replace them
now, rather then fiddle around and try to figure out how to fix them.
I think part of the reason people are talking about a productivity
gain in the global economy in recent years is that, prompted by fears
about Y2K, a substantial amount of the money went for consolidating
and getting rid of old legacy systems and developing and buying new,
more productive and more efficient systems. Around the edges, I am
sure there were some consultants trying to sell people a lot of fancy
new things for no particular good reason. But I think that is a very
minor part of the process. The $100 billion in the United States was
spent by thousands of different organizations, each one making its own
judgments. The major Fortune 500 companies in the United States are
not naive. They are not run by people who are bamboozled by sales
people, either internally or externally. I think they ultimately are
people who spend their money carefully.
If you look at their information technology budgets, most of them went
up over the last two or three years. They went up not because somebody
was doing a good sales job. They went up because people were
discovering how difficult it was to solve this problem. The federal
government was the same way. We started with a Y2K budget under $3
billion and the number kept getting larger because it took more and
more time, people discovered, to actually fix the problem. And so the
indication of the magnitude of the problem is that in most cases
people found it took longer and it cost more and was more complicated
than they estimated. And these are people who are experts. They aren't
naive managers employing 25 people. These are large organizations with
their own in-house staff and very sophisticated managers who
discovered that, in fact, in many cases it took hundreds of millions
of dollars to solve the problem.
Q: Don't mainframe computer systems tend to get replaced anyway, due
to rapid advances in technology and speed?
A: Yes. I think for those people that was their judgment. In many
cases they did not realize how old and inefficient their legacy
systems were or how many they had; when they looked at it, they said
"Why don't we just get rid of all this stuff?" In fact, our view five
years ago in the federal government was that this would be a great
time to inventory our own systems and get rid of the ones that were
inefficient or complicated to run or always breaking down, and to
procure more modern, standardized off-the-shelf equipment. I think you
can find that in 20-25 percent of the cases in the federal government
that's what happened.
Q: Looking at developing nations, what was the extent of the problem
there, as it finally manifested itself?
A: It is always difficult to know what is going on in other nations.
What we do know is that when we assembled and invited the Y2K
coordinators from around the world to meet with us in December of 1998
at the United Nations, we had about 120 countries there, and probably
half of them weren't sure exactly what this problem meant. But they
all agreed to work together and share information on a regional basis
and on all the continents around world. When we had them back to the
U.N. in June 1999, we had a 173 countries represented -- the largest
meeting in history of the United Nations. And it was clear that all
173 of those delegates knew that this was a problem of some degree in
their country that they needed to deal with. Our advice to them, as to
smaller businesses in the United States, was not that they go buy
everything new. We advised them that some things would be just fine,
but that they should take advantage of the information available,
assess each situation, find out what's actually at risk, and deal with
that.
Increasingly, it became clear that most developing nations didn't have
much digital information technology: their power systems, their
telephone systems, a lot of their systems were analog. They were
automated, but their analog devices had gauges instead of digital
readouts and, therefore, they didn't really have any major risks. Our
concerns, I think theirs, were primarily wherever they had gone into
the digital area, particularly in financial transactions. You can take
your credit card around the world and get cash almost everywhere these
days. All of that depends upon financial and telecommunications
systems that are interconnected between nations and continents. These
were what were most at risk, it turned out. But what was going on at
the same time was the central bankers of the world, out of Basel, were
working with all central banks in the world and all market regulators
to share information and to try to make sure there wouldn't be serious
problems come January 1, 2000, with the international flow of
financial transactions.
I think because of the kind of international effort and the fact
individual nations paid attention to the issue where they needed to,
we've only seen a few glitches -- some, but just a handful of glitches
in financial systems or similar telecommunications networks.
Q: Suppose no attention had been paid to the problem and no efforts
made to fix the Y2K bug in advance of January 1. What would have
happened?
*****[The quote cited in the 3/22/2000 Questions & Responses is as
follows:]
A: It was clear two years ago to me after talking with a lot of
experts, if nobody did anything else beyond what they had already done
up until two years ago, that the world as we knew it would end. The
New York Stock Exchange would not have been able to open on Jan 3, the
financial markets would have closed, the banks would have had very
great difficulty calculating accurately the money they were owed, or
the money they owed to others. Payroll systems and other basic
complicated financial systems in the U.S. would not have functioned.
And over time we would have had a clear degradation in
telecommunications and some power systems. I think that we wouldn't
have had to wait very long, if we had done nothing. As systems started
to operate, they would have stopped. In fact, in spite of our largely
successful remediation efforts, I have seen a list of about 90
glitches and failures around the world due to Y2K problems. This list
is an indication where we were headed if we didn't do anything.
My disagreement with the doomsayers was the view that we could never
fix it. Some believed that it was such a complicated problem and it
infected everything potentially and that we'd never get enough
cooperation, enough work done together, enough information sharing, to
be able to get it done in time.
My view was that if we mobilized all possible resources, we could, in
fact, make a significant impact on minimizing the risks. If you talk
to major financial institutions in this country, major banks, major
telephone companies, they will all tell you that they are delighted
and breathing a great sigh of relief that their systems are running
today. They are confident that they wouldn't have run if they hadn't
done all this work in advance. In the State of California, Los Angeles
County, an enormous jurisdiction, estimates that about 60 percent of
their intelligent systems would have stopped. They'd looked at,
literally, thousands of systems -- they went through them all -- and
the vast majority of them had problems that if they hadn't corrected
them would have stopped them cold -- they would not have been able to
pay benefits to local people, they would not have been able to pay
their payroll.
So the irony is that because people worked at it in such a consistent
way, and there was effective information-sharing, and because people
got better at it as we went through it, people are now questioning
whether it was a big problem in the first place. Historically, in
information technology the world hasn't done well with big problems.
Major projects usually cost too much. They take a long time to get
done, and they usually don't work well, which is why a lot of the
doomsayers were information-technology programmers. They weren't
people off the street -- they were people who looked like they should
know. Some of them said it would be impossible. So one of the great
ironies is, the world having pulled together to meet this challenge
and deal with a major information technology problem, having done it
not a hundred percent perfectly, but pretty well, close to
ninety-eight percent perfectly, we now confront the other side of the
coin -- "Could you have spent less"? Oh, that's a good question to
pursue, but when you're running one of those companies, if you had a
major failure in the first week of January, in the year 2000, the
acceptable answer wouldn't be "I didn't quite get it done," but "Look
how much money I saved by not fixing it right."
Q: Does the Y2K experience hold any long-term implications for the
global information infrastructure?
A: There are a number of possible implications. Many organizations
worldwide now have a better inventory of their information technology,
and a better understanding about the critical nature of it. In the
future, they'll manage these systems better.
In addition, I think focusing on the Y2K risk will help us with
understanding issues of information security as we go forward.
Information security has not received the attention it deserves, just
as information technology itself in some places has been seen by top
managers as peripheral to the function of an organization: "Well those
are the geeks, those are the techie guys, I don't know what they're
talking about."
I think what happened with Y2K is chief executives, national leaders,
top managers, discovered that you don't need to know about "bits" and
"bytes," the technical language of information technology, to
understand that if it doesn't work you are out of business. People
running organizations understand that the operations of information
technology and the security of information technology go to the core
of their ability to run their systems and run their businesses. So I
think that that will help us as we go forward, insuring that, in fact,
we provide the appropriate protections for those systems in the
future.
And as we've said, I think most people will have better systems when
they get done with it. They will have upgraded; they will have
replaced their legacy systems. Finally, in terms of national and
international cooperation, it's not quite clear where it goes into the
future. Within the United States, you've seen a tremendous amount of
information-sharing and cooperation within industry groups and across
industry groups trying to deal with this problem. In addition, there
are better lines of communication between the private sector and the
government sector in a lot of countries. Then we had this kind of
unique cooperation on an international organizational basis with
national coordinators representing individual nations, and so we have
a list now of 173 national coordinators that we've been sharing
information with back and forth who have been holding regional
meetings.
There have been at least two regional meetings in every continent of
the world in the last year, sharing information, working together.
What you're most likely to see in the future is that, on a regional
basis, countries that have worked together on information technology
for Y2K are likely to continue to do that. South America is now
talking about how they can continue this kind of informal
information-sharing, to do a better job with electric power, and oil
and gas development now that they see how it all relates for the first
time throughout the continent. We've had some discussion with the
national coordinators at their request. Is there a way to continue
this informal, non-bureaucratic approach to sharing information? It's
not quite clear where that'll go. There are a lot of different
initiatives for improving the use of information technology in the
world and nobody wants to duplicate those efforts. But on the other
hand, one of the unique things about Y2K was it was dealt with
generally very effectively by ad hoc coalitions.
The International Y2K Cooperation Center was funded by the World Bank
with contributions from the United States. It had an affiliation with
the U.N., but it was really a freestanding organization. And the Joint
Year 2000 council, which functioned under the Bank for International
Settlements, with market regulators and insurance regulators as well
as bank regulators, was pulled together as an ad-hoc group. Over 200
major financial institutions in countries around the world cooperated
in way they never had before.
They all had a goal, which was we had to deal with Y2K. So there was a
common enemy that people could deal with. Now that we've dealt with
that, there's a common goal of everyone being more efficient in using
information technology and taking advantage of it. Whether we'll be
able to figure out how to capture that experience and that momentum
going forward into the future is still not clear. Groups won't do well
just meeting for the sake of meeting. I think there is, at a minimum,
a great interest in developed as well as developing countries to find
a way to continue to share information about what's going on with
electronic commerce, what's going on with information security, but
it's still open as to what will come of this.
(end transcript)
(Distributed by the Office of International Information Programs, U.S.
Department of State)
2) Sector
Benchmarks Fact Sheet
December 13, 1999
(The following are benchmarks for some key sectors compiled by the President's
Council on Year 2000 Conversion.)
FINANCIAL
SERVICES
Banks
and ATMs
o According to second quarter 1999 reporting, there were 10,350 banks and
savings institutions in the United States. In 1998, these institutions
served
consumers at 83,963 bank branches. (Source: American Bankers Association)
o As of Fiscal Year 1999, there were 227,000 ATMs in the U.S. (Source:
American Bankers Association)
o Under normal circumstances, 1 to 2 percent of all ATMs are "down" because
of
mechanical breakdowns or because they simply run out of cash. (Source:
EFT
Report: "Business Strategies for the Debit, POS, EBT and ATM Marketplaces"
9/8/99)
o About 8 to 10 percent of the time, customers experience failure on their
first attempt at an ATM. This is typically because of user error: entering
the
wrong PIN, trying to withdraw unavailable funds, or accessing the wrong
account. (Source: EFT Report: "Business Strategies for the Debit, POS,
EBT and
ATM Marketplaces" 9/8/99)
Credit Transactions
o About 10 percent of all credit transactions fail routinely. Reasons
for
failure include: equipment break downs, consumers over credit limits, or
user
error. (Source: Star Systems, Inc.)
ENERGY
Electric Power
o There are 3,108 electric power companies that supply services to American
consumers (not including Canadian and Mexican companies that are part of
the
interconnected power grid):
-- 9 Federal utilities (including 4 DOE Power Marketing Adminstrations
and
TVA),
-- 239 Investor-owned utilities,
-- 858 Electric cooperatives, and
-- 2,102 Municipal electric utilities.
o Each year, customers nationally in the U.S. experience about 13 hours
of
power outages, not including the effects of major storms.
o The average length (nationally) of a power outage caused by a major storm
is
72 hours.
o In 1998, a year of particularly destructive weather (hurricanes, ice
storms,
etc.), the average national reliability for the year was 99.18 percent.
o Major system failures do not always affect customers. The
interconnectedness of the grid allows system operators the flexibility,
in
some cases, to switch instantly to alternate sources of power.
o On average, winter is a time of low electric demand, so demand-related
stresses on system reliability - such as experienced during the hottest
parts
of the summer - seldom occur during the winter holidays.
o Causes of localized outages during the holiday season typically include
weather-related incidents such as tree branches or ice falling on power
lines,
and other physical problems such as traffic accidents and vandalism.
o Examples of electric transmission outages that have occurred during the
Christmas/New Year holiday season:
-- A severe ice storm in January 1998 knocked out power for 1.5 million
customers in Canada and New England.
-- Freezing rain and sleet in early January 1997 caused outages for 95,000
customers in the Carolinas.
-- On Christmas Day in 1996, 75,000 customers in British Columbia lost
power
when a connector failed.
-- Ice, sleet and snow in Virginia and West Virginia disrupted power to
122,000 customers in early January 1994.
(Sources: Edison Electric Institute and North American Electric Reliability
Council)
Power Plant Outages
o A Forced Outage is defined as an unplanned component failure or other
condition that requires the unit to be removed from service.
o The Forced Outage Rate is the percentage of time that capacity is lost
due
to forced outage.
o The Equivalent Forced Outage Rate (EFOR) reflects both forced outages
and
forced de-rating that reduces available capacity. (For example, EFOR
would
count the loss of 10 percent of a plant's capacity for 10 hours in a forced
de-rating as "equivalent" to a forced outage of 1 hour.)
o For 1994 to 1998, the EFOR data shows:
-- Forced outages are not uncommon; they range from about 5 percent EFOR
for
Hydro units to 13 percent EFOR for nuclear units.
-- Forced outages are routinely accommodated without loss of service to
electric customers.
Summary data on 1994-1998 average EFOR
(Note that coal, nuclear and hydro provide roughly 50 percent, 20 percent,
and
10 percent of generation, respectively. Natural gas provides most
of the
rest.)
Plant Type # units (in 1998)
EFOR (avg. over 1994-1998)
Coal
856
7.06%
Oil
132
12.37%
Gas
382
9.85%
Nuclear
110
13.05%
Hydro
747
5.02%
[Sources: Generation Availability Data System (GADS), Generating Availability
Report (GAR)]
Nuclear Power
o In the last five years, during the interval December 28 - January 3:
-- The average number of event reports (nuclear power plant and materials):
22.
-- The average number of nuclear power plant event reports: 17
-- The average number of emergency declarations: 1 "Unusual Event" (the
lowest
of four possible emergency classifications)
o Based on information dating back to 1985, there have been nine
weather-related events that resulted in notifications to NRC in the December
28 - January 3 period.
-- 3 involved inoperable sirens (all caused by severe cold/icing)
-- 2 involved loss of offsite power (causes: 1 high winds and 1 lightning
strike)
-- 2 involved low water levels at the intake - possible loss of heat sink
(causes: 1 extreme tide and 1 high winds affecting lake level)
-- 2 involved restricted plant access (causes: 1 mudslide and 1 icy roads)
(Source: Nuclear Regulatory Commission)
Oil
o Residences -- including multi-family -- heating with home heating oil
or
heavy fuel oil: 9.3 million in 1997. (Source: Department of Energy
- Energy
Information Agency)
o In 1997, 5,000 establishments were engaged in retail supply of heating
oil
to customers and there were nearly 6,000 liquid propane dealers.
(Source:
Census Bureau)
o Industry sources estimate that "automatic deliveries" (contracts where
dealers take into account usage patterns and degree days and automatically
re-supply the customer) account for about 80 percent of the market with
"will
calls" making up the remaining 20 percent. (Source: American Petroleum
Institute)
o Industry sources estimate that most "will call" customers refill before
reaching the last quarter of their stocks. "Will call" customers
who request
supplies on short notice know they face some competition for a limited
amount
of extra trucks and drivers. (Source: American Petroleum Institute)
Gasoline
o There are approximately 180,000 gas stations in the United States.
o Depending on the company branded outlets, the number of gasoline stations
normally closed on New Year's Day ranges from zero to 15 percent.
o Unplanned closings on New Year's Day are typically weather related.
o Stations that request to be closed in advance are those not typically
busy
on New Year's Day because they are not located on a major travel route.
o Temporary supply outages at individual stations range from 1 to 15 percent,
with a very short duration time for replenishment (approximately one hour).
o Point of Sale operation (pay at the pump) is approximately 99 percent
reliable on a daily basis, with easily implemented manual back-up systems.
(Sources: American Petroleum Institute and Federal Energy Regulatory
Commission)
Natural Gas
o There are about 55 million residential natural gas customers (metered)
nationwide. (Source: American Gas Association)
o Natural gas is supplied by 1,400 LDCs, Municipals and Combination Utilities,
as well as by hundreds of marketers that supply natural gas. (Source: American
Gas Association)
o Loss of service because of weather is not typical. Disruptions
are
typically caused by a third party (e.g., construction crew accidentally
breaks
into a natural gas line). (Source: American Gas Association)
o On a typical winter day, 1 percent of compressors,.08 percent of
measurement facilities, and 5 percent of communications/data devices may
encounter problems. None of these malfunctions are sufficiently significant
to impact customer service or natural gas delivery. (Source: Interstate
Natural Gas Association of America)
TRANSPORTATION
Roads and Highways
o On a normal day, less than 1 percent of traffic signals turn to flashing.
On a bad day, the number of flashing signals may increase to 1 percent.
o The rate of fatalities during the New Year's holiday period is less than
half the rate during the rest of the year (4.6 fatalities per 100 million
miles of long-distance automobile travel compared with 9.3 fatalities at
other
times).
o The percentage of alcohol-related fatalities during the New Year's holiday
period decreased from 67 percent in 1997 to 51 percent in 1998.
o In 1997, during the New Year's holiday period, 192 people were killed.
Of
these, 129 were killed in alcohol-related crashes.
(Source: Department of Transportation)
o More than 196 million registered vehicles and 176 million licensed drivers
are on record in the United States. (Source: National Transportation Safety
Board)
o There are about 2.6 million licensed commercial truck drivers in the
United
States. (Source: National Transportation Safety Board)
Transit
o Number of systems operating nationwide, including Dial-A-Ride, ferryboats
and public vanpools: Approximately 6,000. (Source: Department of
Transportation)
o Large system operations are typically on a significantly reduced schedule
during the Christmas and New Year's periods, such as a holiday or Sunday
schedule. Some exceptions may apply. (Source: Department of Transportation)
o Rail rapid transit systems carry almost two billion passengers annually.
(Source: National Transportation Safety Board)
o Ferry boats, most prominently in New York City and Seattle, carry more
than
270 million passenger miles annually. (Source: National Transportation
Safety
Board)
Aviation
o Average number of commercial flights during the last five three-day New
Year's holidays: 84,560.
o Average number of commercial flights on the last five New Year's Days:
25,726.
o Average number of commercial flights delayed 15 minutes or more on the
last
five New Year's Days: 424.
o Percentage of New Year's Day commercial flights delayed: 1.6 percent.
o Average number of all flights, including commercial, military and general
aviation, delayed on the last five New Year's Days: 430.
o Average number of delays of all flights due to weather: 390, or 91 percent.
o For the entire year 1997, the number of flights delayed 15 minutes or
more:
approximately 245,000, a decrease of 9.6 percent from 1996.
o Percentage of 1997 delays due to weather: 68 percent or 16.9 percent
less
than in 1996.
(Source: Department of Transportation)
Railroads
o For the last six New Year's Days, there have been a total of 26 reported
accidents on the railroads, an average of 4.33 per year. (Source: Department
of Transportation)
o For the entire year 1998, there was an average of 7.05 accidents per
day.
(Source: Department of Transportation)
o Amtrak carries about 21 million intercity passengers annually. (Source:
National Transportation Safety Board)
o The railroad industry transported more than $32 billion in freight in
1997,
and amassed more than 1.36 trillion revenue ton-miles. (Source: National
Transportation Safety Board)
Pipelines
o Pipelines carry more hazardous materials in the United States than any
other
form of transportation. (Source: National Transportation Safety Board)
o Annually, almost 600 billion ton-miles are carried in 177,000 miles of
pipe,
and 21 billion cubic feet of natural gas are delivered through 1.2 million
miles of pipe. (Source: National Transportation Safety Board)
o Total pipeline mileage: 2,182,000 miles.
o Total hazardous liquid shipped annually: 592.9 billion ton-miles (1996).
o Total number of pipeline operators: 2,424.
o Average notices of failures for the last three years:
Dec. 31 - 5
Jan. 01 - 3
Jan. 02 - 6
Jan. 03 - 2
Jan. 04 - 2
o Average number of reportable incidents for the last three years.
Liquid Pipeline Transmission Pipeline
Distribution Pipeline
Dec. 31 2
0
0
Jan. 1
1
0
1
Jan. 2
4
2
3
Jan. 3
3
1
1
Jan. 4
1
0
0
(Source: Department of Transportation)
U.S. Coast Guard Operations
o Average daily number of search-and-rescue requests in last five Decembers:
21.
o Average daily number of search-and-rescue requests in last five Januarys:
36.
o Average daily imports of crude oil in the last five Januarys: 260,000
barrels.
o Average daily number of pollution cases responded to by the Coast Guard
in
the 20-day period surrounding New Year's: 25.
o Average daily number of marine casualty cases responded to by the Coast
Guard in the 20-day period surrounding New Year's: 19.
(Source: Department of Transportation)
EMERGENCY
SERVICES
EMS
o Gaps in service can be experienced when low batteries, power outages,
problems with satellites, microwave systems, towers, sun spots, terrain
issues
or weather affect radios, portables, cell phones and related communications
equipment.
o Mechanical breakdowns or problems with EMS vehicles and related responding
units (like extrication units, rescue, air medical, etc.) are regular
occurrences.
o Air transport, both rotor wing and fixed wing, are very dependent on
weather
conditions any time of the year.
o In areas of the country with extreme weather, certain equipment can be
rendered inoperable by the extremely cold temperatures (e.g., Alaska).
o In most natural disasters, communications are the first thing to fail
and
radio compatibility among multiple provider agencies often becomes a problem.
o Wireless communications can be and often are compromised in any major
event
or major emergency because of over use of the system by the public and
media.
o EMS Staffing is higher during holidays and other major events, but typically
is severely strained when a second event occurs during the period (such
as a
tornado or other weather-related event).
o Increased prevalence of a particular disease in a community (such as
influenza) can reduce the availability of EMS crews.
(Source: Federal Emergency Management Agency)
"9-1-1"
o A failure at a 9-1-1 Center is not an unusual occurrence, but typically
does
not reduce service levels.
o The average Emergency Communications District can experience a disruption
weekly. "Manual switchover" to non-computer operations is dependent
on
working phone lines to allow calls for emergency assistance. The
typical
reasons for failures:
-- A problem at the dispatching site. A 9-1-1 switch could fail within
a
dispatch center. The more complicated the dispatching system, the
greater the
exposure.
-- The failure of the public switch telephone network. It is not
unusual for
this to occur on a weekly basis. It can happen as a result of erroneous
programming.
-- Lightning strikes are listed as primary causes for Public Safety Answering
Point (PSAP) equipment damage and failure. These weekly incidents
occur when
underground or overhead telecommunication lines are damaged in an ice storm
or
auto accident.
-- Telephone lines are frequently jammed by call overload. Mother's
Day, New
Year's Day, radio station give-aways, and the first day of major concert
and
event ticket sales contribute to the overloads.
(Source: Federal Emergency Management Agency)
MAIL
DELIVERY
o Extreme weather and long-term, widespread power outages can force delays
or
a reduced level of service. This is not typical. However, it does
happen
occasionally.
o With 38,000 locations nationwide, a few post offices, at any given time,
are
without communications or energy. This can force a reduction in service
or,
less commonly, temporary closure of one or more post offices.
o Severe hurricanes, floods, fires and winter storms are the most common
challenges to postal operations. Weather problems can cause delays
in air and
surface transportation that will, in turn, delay mail.
(Source: United States Postal Service)
3) Press Coverage
Concerning Iraq’s Oil Industry and the Need for Spare Parts
and Equipment
UNITED NATIONS, 1/19/2000 (AFP) - UN Secretary General Kofi Annan has warned
of
a possible "major breakdown" of the Iraqi oil industry if the Security
Council continues to withhold spare parts and equipment. In a letter to
the
council president, US ambassador Richard Holbrooke, Annan said Iraq's oil
industry was "in a lamentable state" requiring "prompt remedial action."
The
letter was dated January 14 and made public on Tuesday. Annan repeated
a
previous recommendation that the council increase from 300 million dollars
to
600 million dollars the amount Iraq could spend on rehabilitating its
industry in the six-month phase of the oil-for-food programme which ended
on
December 9. He said he would await the report of a group of six UN experts
who arrived in Iraq on Monday before deciding whether or not to recommend
a
similar increase of the allocation under the current six-month phase. The
experts -- two from Britain, and one each from the Netherlands, Jordan,
Norway and Russia -- are due to stay in Iraq until January 31, Annan said
in
a separate note. The Security Council removed the ceiling on Iraq's exports
of crude oil when it revised its nine-year-old sanctions regime on December
17. But restrictions remain on the use which Iraq may make of its oil
revenues, and all contracts for imports must be approved by the council's
sanctions committee. On Wednesday the United Nations said that Iraq had
submitted a total of 2,003 contracts worth 1.11 billion dollars for oil
parts
and equipment under the three most recent complete phases of the oil-for-food
programme. The sanctions committee had approved 907 of these, worth 453.3
million dollars, and put another 448, worth 224.6 million dollars, on hold,
the Office of the Iraq Programme said. The other contracts are either pending
the committee's decision or have not yet been processed due to insufficient
information provided by Iraq.
In his letter, Annan said the deterioration of Iraq's oil facilities was
affecting the health of workers and causing serious environmental damage
as
well as damaging oil wells, some of them permanently. If it continued,
he
said, it "may also cause a major breakdown in Iraq's oil production and
export capacity." Annan appended to his letter a detailed list of spare
parts
and equipment drawn up by an independent expert who visited Iraq under
contract from the UN from December 15 to 21. The expert, from the Dutch
firm
Saybolt, quoted the ministry of oil in Baghdad as saying that Iraq had
averaged production of 2.75 million barrels a day in the six months to
December 9. Of this, it exported 2.16 million barrels per day. The expert
quoted the ministry as saying this level of output had been achieved "under
a
regime of severe risk management" rather than a "planned programme of good
reservoir management." He forecast output of between 2.5 million and 2.6
million bpd in the current six-month phase of the oil-for-food programme,
with exports of between 1.95 million and 2.0 million bpd.
In his letter Annan said that "unless applications for contracts for key
items of oil spare parts and equipment are approved expeditiously and are
made available and commissioned within a short time frame, the production
of
oil is likely to drop, even under a regime of 'severe risk management'."
Copyright 2000 by Agence France-Presse (via ClariNet)
4) May 1999 International Energy Agency (IEA) Report Mention of Embedded
Systems Problems and Refineries
(http://www.iea.org/ieay2k/html/refine.htm)
“Refineries are by design highly complex relying heavily on computers
for smooth operation. An extensive survey of a refinery in the UK
identified 94 systems requiring investigation for Y2K compliance. Of the
systems assessed it was found that three would fail and that two of these
three failures would cause a shutdown.
Attempting to trace even a small number of potential Y2K problems at a
refinery is undeniably a major undertaking.
Refining is but a part of the general problem facing oil companies
trying to address Y2K issues. It is a technologically intensive industry
and
companies are likely to operate myriad date sensitive integrated systems.
Embedded processors are the main source of this sensitivity and are
found in devices such as flow meters, transmitters and smart valves. They
are found throughout the oil industry and in all sectors, including drilling
platforms, production platforms, pipelines and process plants. In the case
of process plants, the devices containing embedded chips are interconnected,
making the
problem even more complex and increasing the possibility of Y2K failure.
A pilot inventory and assessment of a catalytic cracker and
co-generation plant in the US revealed 1,035 systems of which 21% were
not
Y2K compliant and 6% that would lead to serious plant shutdowns or reduced
production capabilities. The catalytic cracker would fail, rendering the
refinery incapable of making gasoline. Given the widespread use of
catalytic crackers in
modern refineries, questions must inevitably be raised about their
reliability in other refineries. For the co-generation plant 19% of the
hardware, 36% of the software and 24% of the custom code was found to be
non-compliant.
In late 1997 one oil company’s engineers testing valve control
equipment in their refineries discovered thousands of terminals controlling
the dispensation of oil to have microchips with Y2K problems. All of the
chips required replacement, however it was discovered that the replacement
chips would not fit on the existing motherboards. It was therefore necessary
to order both new
chips and motherboards. Worse still, the replacement motherboards were
found
not to fit the old valves so the valves themselves had to be replaced.
This
example demonstrates how a Y2K problem can escalate beyond the original
fault to include systems that may actually be compliant. An item’s Y2K
compliancy is therefore no guarantee that its replacement will not be
necessitated by problems arising in other equipment. “
5) Embedded Systems: What is the Nature of the Threat They Pose?
The following definition is taken from the United Kingdom's Action 2000
Web site:
http:///www.open.gov.uk/bg2000/whattodo/embsys2.html :
"Embedded systems contain programmed instructions running via processor
chips...They perform control, protection, and monitoring tasks...In broad
terms embedded systems are programmable devices or systems which are
generally used to control or monitor things like processes, machinery,
environments, equipment, and communications."
When
embedded systems fail, they can fail in a variety of unpredictable
ways. Small, seemingly insignificant failures can trigger other
system
failures. The timing of the triggering of other system failures
cannot be
readily predicted since the environment in which the failures are taking
place is dynamically changing. Once the failures have occurred
and have
triggered other failures,
the root causes of the initial failure can be hard if not impossible
to
determine. (See Part 2 of "A Call to Action: National and
Global
Implications of the Year 2000 and Embedded Systems Crisis ~
A Working White Paper on Y2K at http://users.rcn.com/pgordon/y2k/
for a discussion of
these matters.
There are major differences in perspectives concerning those who have some
familiarity with date-sensitive embedded systems. There is a dispute
over
the threats that they pose. On November 9, a meeting was held involving
some embedded systems experts and some members of the President's Council.
A result of that meeting has been that the head of the Council has
acknowledged that some embedded systems that do not appear to track the
date
may nonetheless have date-sensitive microchips in them and that these
systems also have to be tested and plans made to handle breakdowns.
6)
Statement Concerning Embedded Systems Issued by John Koskinen Circa 11/29/99
[Comments of Paula Gordon are added in all caps in the text of the list
of
"final statements". The "final statements" have been numbered in this text.
They are not numbered in the original text.]
PRESIDENT¹S COUNCIL ON YEAR 2000 CONVERSIONS
MEETING ON Y2K EMBEDDED SYSTEMS
Tuesday, November 9, 1999
American Society of Association Executives Building
1575 I Street, Washington, DC
Participants in the meeting included technicians that had done work in
the
bio-medical, defense, electric power, gas, manufacturing, oil, shipping,
and
telecommunications industries. To help with the discussion, an agenda was
provided with discussion statements concerning the types of embedded systems
potentially atY2K risk, difficulties in testing for such embedded systems
and fixes for problems found. Those statements were revised during
the
meeting and the agreed upon final statements are presented below, along
with
a brief summary of the discussion that led to the final statement.
Types
of embedded systems found to have a Y2K risk:
[1] Final Statement: Embedded systems are at risk of problems during Y2K
rollover if they conduct a calculation that depends on a representation
of
the date. The date could be in "relative" or "absolute" form.
The participants presented a number of specific cases where they had found
Y2K problems in embedded systems. Several of these involve calculations
of
time increments inside an embedded system without the date being displayed
or apparently used. In these instances an embedded system calculates the
time interval by subtracting seconds from seconds, minutes from minutes,
hours from hours, and calendar dates from calendar dates.
All except one of the examples were large, complex processes where embedded
systems inter-relate with each other and, in some cases, with external
computer systems. The one example was of a stand-alone embedded system
that
was unconnected to others that did not apparently involve dates.
That
example lead to a discussion about the need for a continuous power source
being available for any such devices to function, and it was pointed out
that in some sectors there are many such devices, but that few problems
had
been found in them.
There was considerable discussion of potential failure rates of embedded
systems. Estimates ranged from a 1 - 2% potential failure rate of processes
containing embedded systems in some sectors to 4 - 6% in others, but no
conclusion was reached.
COMMENT: IN SOME SECTORS, THE POTENTIAL FAILURE RATE IS FAR HIGHER.
SOME OF THOSE SECTORS INCLUDE THE SECTORS THAT POSE THE GREATEST
RISK TO LIFE, PUBLIC HEALTH AND SAFETY, AND ENVIRONMENTAL
SUSTAINABILITY. PG
.....An important distinction was made between failure
of an embedded system, which may not cause a process or device to fail
in
operation, and failure of a process or device due to an embedded system.
The
former represents the estimates above, and the latter is much less
prevalent.
The remainder of the discussion during the meeting focussed on large,
complex processes that contain embedded systems. The question
of having a
real time clock or access to a clock was discussed and examples were
presented where the time was set by a process controller and transmitted
to
other embedded processors involved in the process. Other examples
of
problems were discussed where time was used apparently to calculate relative
increments (e.g. day of the week) as opposed to absolute dates.
When
embedded systems will fail:
[2] Final Statement: Where possible, all mission critical systems should
be
tested end-to-end, whether or not the systems appear to have date sensitive
functions. Failure to do so means a small level of risk has been
assumed
that, at minimum, should be addressed with a contingency plan.
COMMENT: THE USE OF "SMALL" TO CHARACTERIZE THE LEVEL OF RISK
MINIMIZES THE SERIOUSNESS OF THE FACT THAT THERE IS RISK. THE RISK
MAY
INCLUDE SOME HIGHLY SENSITIVE "SAFETY CRITICAL" SYSTEMS IN NUCLEAR
POWER PLANTS, CHEMICAL FACILITIES, REFINERIES, HAZARDOUS MATERIALS
SITES OR FACILITIES, PIPELINES, WATER SUPPLY SYSTEMS, AND SEWERAGE
DISPOSAL PLANTS. PG
The discussion that lead to this statement began with a presumption that
embedded systems involved in calculating time increments, as well as those
that apparently computed dates, are at Y2K risk. During the discussion
the
statement to "test mission critical systems whether they have a date
function or not" was almost agreed to, until it was pointed out one can
only
test those types of devices with end-to-end testing.
This statement was focussed on mission critical systems because it is
difficult and expensive to conduct such testing. The term mission
critical
systems was used to include safety critical systems as well as other systems
where the cost of failure would be high. Therefore, while the statement
says the risk of failure is low, the impact of any such failure would be
high.
COMMENTS: THIS GETS US INTO THE STAKES VS PROBABILITIES ARGUMENT.
THE STATEMENT REFLECTS A FOCUS ON THE PROBABILITIES SIDE OF THE
ARGUMENT. PG
..The statement also recommends a contingency plan to help mitigate risk
-- such a plan should not be viewed as an alternative to testing because
detection of a failure may be difficult and a failure could cause
substantial collateral damage before it is detected.
[3] Final Statement: The majority of failures of embedded systems are expected
to occur on or about December 31st through January 1st. However,
simply
turning a system off during that time frame is generally not a solution.
COMMENT: THIS STATEMENT SEEMS TO ME TO BE HIGHLY QUESTIONABLE AND
ALSO MISLEADING. THIS STATEMENT DOES NOT REFLECT AN UNDERSTANDING
OF
~ THE FACT THAT THE TIMING OF FAILURES IS FAR MORE COMPLICATED THAN
THAT (SEE MARK FRAUTSCHI'S PAPER ON EMBEDDED SYSTEMS. SOME
ASSESSMENTS, INCLUDING GARTNER GROUP ASSESSMENTS, HAVE PROJECTED
PROBLEMS OVER TIME, NOT ALL AT ONE TIME.)
~ THE RESULTS OF FAILURES, INCLUDING MULTIPLE CASCADING FAILURES,
WILL NOT NECESSARILY BE IMMEDIATELY APPARENT.
~ THERE COULD INDEED BE ROLLING WAVES OF DISRUPTIONS AND DISASTERS
THAT GO ON FOR MONTHS OR YEARS OWING TO EMBEDDED SYSTEMS FAILURES
THAT OCCUR ON TOP OF IT-RELATED DISRUPTIONS. IT MAY BE IMPOSSIBLE
TO
TRACE THE ORIGIN OR THE SEQUENCE OF THE FAILURES AND DISRUPTIONS AND
ALL THE MORE DIFFICULT TO UNDERSTAND HOW TO PROCEED WITH REPAIRS.
PG
The discussion explored the question of whether the time of primary risk
of
failure was during the rollover time. It was generally agreed that the
vast
majority of failures in embedded systems are likely to occur over that
period. On the specific question of whether Greenwich Mean Time would
be a
time of high failure, it was stated that most failures would likely
occur
at 12:00 local time, although some would also occur on Greenwich time.
During the discussion, there was a concern raised that the statement may
lead to the ineffective solution of turning off systems during the rollover
period. Therefore, the specific admonition not to rely on that work-around
was included in the statement.
[4] Final Statement: One can have two apparently identical systems of which
one
will not have a Y2K problem but the other will have operating difficulties.
However, the chances of this are small.
The likelihood of failure of one of two identical systems, as described
in
this statement was considered to be very small, but, again, it was agreed
that all mission critical systems needed to be tested.
Difficulties in testing for embedded systems at risk:
[5] Final Statement: Organizations that have relied on a device manufacturer¹s
declaration of Y2K compliance are at risk if they do not keep up with the
most recent manufacturers¹ statements.
COMMENT: VENDOR CERTIFICATION IS OF QUESTIONABLE SIGNIFICANCE IF AN
EMBEDDED SYSTEMS IS LINKED TO ANOTHER SYSTEM OR SYSTEMS THAT ARE
NOT COMPLIANT. PG
The discussion concerned cases where testing had brought into question
manufacturers¹ statements of the readiness of their products.
A number of
instances were cited where problems had been found both externally by users
that had tested and by manufacturers themselves. While the changes
needed
to remedy such problems have normally been made quickly available, the
concern was expressed that many organizations were not aware of or taking
advantage of those fixes.
[6]
Final Statement: Some interconnection problems among embedded systems can
only be revealed by end-to-end testing.
The discussion concerned how to test for problems in embedded systems.
There was considerable discussion of difficulties of testing in operational
environments and the risks and complexities of end-to-end testing.
However,
a number of examples were cited to show that one could not find all
potential problems in complex, interconnected embedded processes without
end-to-end testing.
Fixes:
[7] Final Statement: Anyone taking a fix-on-failure approach for Y2K,
particularly with embedded systems, runs a significant risk of collateral
damage and a difficult recovery.
There was little discussion leading to this statement. Remedying
the kinds
of Y2K problems participants had found in embedded systems was difficult
and
time-consuming.
[8] Final Statement: After a full and careful technical assessment, there
may be
administrative or operational workarounds to many Y2K problems involving
embedded systems.
COMMENT: IT WOULD BE USEFUL FOR ANY FUTURE ITERATIONS OF THESE
STATEMENTS TO INCLUDE AT LEAST SOME REFERENCES INVOLVING HIGH RISK
EXAMPLES.
While simply turning a system off during the rollover is not normally an
effective administrative work-around, in some instance it could be.
Similarly, setting the year back so that Y2K does not occur may be a
work-around in some instances. However, before using these or any
other
ways to work-around the Y2K problem, all agreed that a thorough assessment
of the full implications of the work-around was necessary.
[9] Final Statement: Even those that have conducted thorough testing need
to
develop contingency plans for mission critical processes and exercise them.
There was little discussion of this statement, in light of the earlier
statements that indicate the risk of Y2K problems.
COMMENT: THE IMPLICATIONS OF FAILURES AND THE NEED FOR NOT JUST
DEVELOPING AND IMPLEMENTING CONTINGENCY PLANS, BUT DOING THE SAME
FOR RESPONSE AND RECOVERY PLANS AND ACTIONS ARE NOT ADDRESSED AS
THEY SHOULD BE IN THESE STATEMENTS. THESE NEED TO BE CARRIED OUT
WHILE ALSO CONTINUING TO ADDRESS ASSESSMENT, REMEDIATION, AND
TESTING CONCERNS. PG
[END OF Mr. Koskinen's statement and my comments in CAPS regarding his
statement. PG]
7) Press Release Concerning Embedded Systems, and NIST and Century Corporation
Collaboration
(Fair Use for Educational and Research Purposes Only)
http://www.newsalert.com/bin/clipstry?StoryId=CoesRubKbyte2nJK&TopicName=top
ic1 from Businesswire December 01, 1999
Y2K Report Says Time is Running Out
UPPER SADDLE RIVER, N.J.--(BUSINESS WIRE)--Dec. 1, 1999--Century Corp,
in
collaboration with The US Commerce Department's National Institute of
Standards and Technology (NIST), has published EMBEDDED SYSTEMS AND THE
YEAR
2000 PROBLEM. The November 22, 1999 technical report disputes the myth
that
embedded systems can be ignored if they don't appear to use a date. This
misconception has caused testing procedures to miss many critical time
dependent functions that should have been tested.
The report also concludes the vast majority of Y2K embedded failures will
occur on January 1, 2000 and therefore very little time is remaining to
test
and correct.
Reactions to the report include those of Secretary of Commerce William
M.
Daley as stated in his November 22, 1999, press release "urging American
businesses to redouble their efforts to test for year 2000 computer problems
that are hidden away in a variety of machines other than computers."
"Ferreting out all the Y2K connections in the systems that run manufacturing
plants, provide services to consumers, and control a host of operations
that
we all rely on is a tough job. We urge businesses to be especially vigilant
in testing embedded systems."
"The Commerce Department's National Institute of Standards and Technology
and Century Corp., a computer consulting firm, have assessed the range
of
testing methods industry is using. They conclude that it is possible that
many
important systems have not been tested adequately. NIST strongly recommends
that all critical systems be tested literally from end to end."
Michael Cherry. President of Century Corp. has stated, "In my opinion,
the
most vulnerable industries include utilities, chemical, energy,
manufacturing and defense."
As a non-regulatory agency of the U.S. Department of Commerce's Technology
Administration, NIST strengthens the U.S. economy and improves the quality
of life by working with industry to develop and apply technology,
measurements and standards through four partnerships: the Measurement and
Standards Laboratories, the Advanced Technology Program, the Manufacturing
Extension Partnership and the Baldrige National Quality Program.
Century Corp. is a consulting firm that provides testing, expert witness
and
remediation services to solve the Y2K embedded systems problem. Recently,
they co-authored the U.S. Department of Commerce, NIST, Year 2000 embedded
system test guidelines, available on the NIST Web site
at http://www.nist.gov/y2k/embedded.htm and the U.S. Department Of Commerce,
Nist Embedded Systems and the Year 2000 Problem, available on the NIST
Web site at http://www.nist.gov/y2k/embeddedarticle.htm
CONTACT: Century Corp.
Conrad Macina, 201 934-4242
mc@centurycorp.com
8)
Some Ways in Which Embedded Systems’ Failures Can Occur: Comments
on Buffer
Overflow, Function Overflow, and Annual Maintenance Scheduling
Some observations from two different software engineers concerning ways
in
which embedded systems' failures can occur. All are posted anonymously
by
request:
Buffers
and Buffer Overflow - 1
“..(A) buffer is a designated area in RAM. It is designated to hold some
data element. Its designation may be at the hardware level, wherein
there is a chip with X amount of RAM space wholely dedicated to
providing buffer space for a particular operation; or is designation
may be purely implemented in software. In this case, the operation
is assigned a RAM address which is no different than any other
RAM address available for a program.
Now, the former (hardware buffers) are the minority of situations for
embedded systems. But, when these fail there are a limited
number of possible failure paths. THere is also far less probability
of failure with these kinds of systems due to date datatype size as
the whole chip/system is working off a limited input. In this sort of
embedded, everything involved has been integrated including the
clock system. Ironically, this kind of system is much more likely to
suffer catestrophically from remediation induced errors. In that
case, a remediation team fixes the date length in one spot and
does not check for the ability of the hardware in other spots to
accept the now larger by two digits data type size. A rare
occurrance most likely. These sorts of embedded systems are just
far less likely to have overflow problems as the buffer is hardware
limited and so the programmers plan for that when coding.
It is ironic that the area most vulnerable to a hard overflow from a
buffer is least likely to experience it.
Now the other sorts of buffer over flow comes from the more recent
occurances where generic mother boards running something like
DOS are used as embedded systems controlers. Sort of like taking
a cheap PC because it is cheap and using it with some add on
specialty boards to report and control stuff. In this case the buffer
overflow is much more likely to fail as there are many more points
of failure. However, the results of such failure would ironically be far
less catestrophic as the programmers had to develop error
handling. In these sorts of systems the programmer assigns a
chunk of memory as his buffer. Say a space for the datetime
expressed as milliseconds since jan 1 1980. As a programmer,
she could care less as to the size of this integer (a real number
held as the closest binary construct). She just gets it from the
operating system. She then takes this number and manipulates it.
The MOST likely point of buffer failure for any and all embedded
systems is here. This is where the programmer has assigned
space for say the results of a date calc as two digits and gets the
wrong two because the divisor or the set date is now 4 digits. Or it
goes into negative numbers as a result of dividing by 1900. At this
point there is the question of software failure existing. But this is
just a regular sort of failure.
Now buffer overflow is where the result of the date calc is assigned
a finite amount of RAM and the result will not physically fit in the
space. The results of this failure would be defined by the hardiness
of the OS and any additional operating systems within any
additonal addon boards, and any exception handling built into the
program in control at the time.
There are other more insideously hidden overflow issues in such
things as relational database management programs (RDBMS) like
sybase, et al. Programs built around these could and probabaly will
have buffer overflow problems which are not shut down situations
but which will corrupt things.
The issue of data corruption stems from the data type of date time
on most platforms. Mostly as an integer marking the number of
milliseconds from some arbitrary starting point, this number is just
that, a number. So if we get a screwup, it is most likely to be at
the point where the programmers are taking this valure and doing
stuff with it. It is there that data corruption starts. Either we do not
allow for the size of this critter and drop some digits or we have
some other mis interpretation of that this value range should be.
Actually using too much RAM (the prototypical buffer overflow)
occurs here. I take the date as an long or double integer and use
OS routines to convert to a string to display. Only I did not allocate
enough RAM. So when I splat the new string into its spot through
place by address, I end up corrupting some other area in RAM by
the extra two digits, which just happen to be the string
representation for 00. No problem, unless my program (most likely,
in fact almost guaranteed) or some other program happens to read
that area of now populated RAM. It would then get spiurious values.
>From then on, all behaviour is unpredicatable.
But basically my take on the buffer problem is the following.
All y2k problems are software problems. All software will exhibt
undefined behavior when the data upon which the software operates
falls outside the range allowed by the programmer. Therefore, the
y2k problem(s) are, and always were, and will continue to be, a
data problem.
When the data be bad, the results (may) be bad.
I do not know of any calculation in the world that benefits by having
bad inputs, do you?
Now, we are rapidly about to discover that our world has changed
far more profoundly that the end of the cold war. We are in the
process of discovering that we cannot ASSUME that we can trust
our machines, their results, their calculations, or the underlying
data.
In our estimation undefined behavior will be still popping up over the
next three to four years and that we will still be scrubbing data and
repairing/replacing erroneous inputs for the decade...."
Buffers
and Buffer Overflow - 2
January 10, 2000
"...This might help some folks:
Sorry I didn't get this out earlier. To define it is easier terms here
is the information you are requiring.
First embedded systems do not have a standardization program in place.
In
essence there can be several ways programs are written. Now to the Y2K
question. There are buffers in most embedded system programs. This buffers
can be varying in size. When a command is registered with an embedded system
and it is improper or not accepted it can place the command in the
buffer.
It could be doing this by the hour, day, week etc...
Now if I have a program that does not recognize year 2000 it will search
for the date (Year) for a period of time. If it does not find it
then a
loop is created and placed in the buffer. Now comes the problem. When the
buffer is full the system can shutdown, be degraded or begin to act up.
The concern is that it could take hours, days, even weeks before the buffer
is full.
Simple Example: A fire alarm panel each day at midnight registers the date
in this format- Month Day Year- Now on Dec 31, 1999 at midnight it does
not
recognize year 2000. So it attempts to complete the command "store date".
When this fails over a period of time a loop is created in the buffer.
This
continues for two weeks. At the end of two weeks the buffer is full. No
place to send the command so the system shutdown, becomes degraded, or
begins to send out erroneous commands.
Concerns
Shutdown the system and restarting could clear the buffer and then the
process restarts again.
Degraded systems could fail when need the most.
Programs that are running and erroneous commands are sent out. I.E.......
open value to 30 percent rather than 10 percent at the temperature of 70
degrees.
Hope this helps explains things that can go bump in the night..."
Functional
Overflow
"Functional overflow is the name given to either of two conditions. In
one case, the function (procedure or basic 'task' that a program
might perform) overflows the memory space allocated to it. This
occurs because it retrieves some data (such as a date) from an
area in RAM which is larger that it can accomodate within the
variable size within the function. In other words, the function has
allocated 26 bytes and the date comes back as 28 bytes. This
overflow could cause a problem in a number of circumstances such
as, if that variable is used in a calculation, if that variable space is
passed back from the function, if adjacent variable space is
accessed (it would be corrupted in a pass-by-reference function).
As with other software, when the data steps outside the range
allowed by the programmer, things aft gang aglay (get snafu'ed).
The other basic functional overflow occurs when spaces allocated
to a function cause an overflow error within the OS of the device. In
this case, the device's operation as a whole is threatened. The
limiting factor is how well crafted might be the exception handling
within the OS of the device. As a general rule of thumb, you can
figure that the smaller the device (chips, systems, et cetera) the
less room there is for robust error handling, so......."
Annual
Maintenance Scheduling
Annual maintenance scheduling triggers a diagnostic self-check. That
entails the recording of a date and time. If the date did not roll
over
then the PLC can seize and this can result in "functional overflow".
Another problem with annual maintenance scheduling is that the date that
the
function is activated is the same date every year, but that date is by
no
means the same in everything that has annual mainenance scheduling.
*********
|