John Koskinen's Responses to
Questions
from Paula Gordon
Concerning National and Global
Aspects of Y2K
Introduction
In March, 2000, approximately two weeks before his tenure as
head
of the President's Council on Year 2000 Conversion came to an end, John
Koskinen agreed to respond to a written set of questions concerning Y2K.
I submitted the following list of questions to him on March 20. I
received
his responses on March 22. I later asked for and received his
permission
to quote his responses.
My thanks to John Koskinen for so graciously agreeing to respond to
this set of questions and for allowing his responses to be quoted.
I share
this material in the hope that these questions and responses will help
focus
light on Y2K, on what happened prior to the rollover and what has been
occurring to date.
The Questions
The list includes 25 different topics and approximately 60 questions.
The questions were sent by e-mail. The following is a sample of the
form in
which the questions were sent:
1)
Incident Reports
I understand that there were 6000 incident reports received by ICC in the
first five days of the year.
~ First, is this true? Were there 6000 incident reports received
by ICC in the
first five days of the year?
~ How many more reports have been received to date?
~ Has this data been made available?
~ If it has not been made available, will it be made available?
~ If it cannot be made available because of the names of companies and
business that are mentioned, could the incident reports be made public
if
these names were deleted?
The
Responses
Mr. Koskinen's responses to the questions are interspersed
throughout each section. I mention this in order to clarify that
what follows
is not a transcript of a "live dialogue", but rather responses to one list
of
pre-submitted e-mailed questions.
What is reprinted here includes every single one of his e-mailed
responses. All of these responses appear in their entirety with the
exception of a few edits indicated by "........." These few
deletions do not
alter the meaning of his responses to the questions that were asked.
I did not correct any typos in the responses that he e-mailed me.
Other
Modifications
I have modified several of my own questions in the list that appear
below. I replaced two specific references with more general descriptors
owing to the potentially sensitive nature of the information included in
the
original question. I have also added a few specific references and
made
some references more precise. All modifications appear in brackets.
I have added "PG NOTES" after some of the questions and responses.
Some of these notes contain background material concerning the basis for
the
questions. Some of the notes include comments in response to Mr.
Koskinen's
answers. In some instances, I have also added information gathered after
Mr.
Koskinen sent me his responses.
Some Recommended
References
The reader may find it helpful to review or be aware of the following
references. These references should help put the questions and answers
in
a broader context. I have also attached appendices that include materials
that are not readily accessible.
Reference A ~ The February 29, 2000 Senate Report (see
http://www.senate.gov/~y2k/documents/final.pdf)
For a list of reported
incidents involving Y2K, see the appendix in this pdf document or
see
http://hv.greenspun.com/bboard/q-and-a-fetch-msg.tcl?msg_id=002iY6
Reference B ~ The March 29, 2000 Final Report of the President's
Council
on Year 2000 Conversion (http://www.y2k.gov/docs/LASTREP3.htm
)
Reference
C ~ January 17, 2000 Comments and Impact Ratings (Paula
Gordon) (http://users.rcn.com/pgordon/y2k/.
Click on "Comments,
Essays, and Op-Ed Pieces")
Reference D ~ Presentations by Olivia Bosch, Rosanne Hynes, and others
at the January 24 -25, 2000 Conference in Livermore,
California sponsored by Lawrence Livermore
National Laboratory (http://cgsr.llnl.gov
Click on January 24 - 25, 2000
Conference on Y2K, then click on Agenda)
Reference E ~ Grassroots Information Coordination Center Web site: This
Web site focuses on Y2K and infrastructure concerns. It serves as
a
repository of media coverage, wire service reports, government reports
and
the like. The Grassroots Information Coordination Center Web site
at
http://www.greenspun.com/bboard/q-and-a.tcl?topic=Grassroots%20Informa
tion%20Coordination%20Center%20%28GICC%29
Reference F ~ Glitch Central at http://www.ciaosystems.com/glitchcentral.htm
is a Web site
that has been tracking reports of Y2K-related problems.
Explanatory Note Concerning the Use of Some Unnamed Sources
There are many people who are in a position to provide needed information
or expertise concerning Y2K who feel little, if any inclination to do so.
In
their minds, the disincentives to being forthcoming concerning problems
or
issues relating to Y2K may far outweigh any possible incentives.
The
following is a list of some major reasons that individuals may refrain
from
speaking out or may require that they remain anonymous when they do:
~ fear of losing a job, jeopardizing a contract, or otherwise adversely
affecting one's career or business,
~ liability concerns or fear of litigation,
~ fear of other possible consequences or reprisals
~ Non Disclosure Agreements preventing the sharing of proprietary
information,
~ organizational or peer pressure,
~ a desire to avoid controversy,
~ a climate that can prove hostile to people who are candid about sensitive
or controversial Y2K-related concerns.
For these and other reasons, it has been necessary for me to promise
anonymity to a number of sources who have provided me information.
*******
John Koskinen's Responses to Questions from Paula Gordon
Concerning National and Global Aspects of Y2K
March 22, 2000
(With Notes Added 4/4/2000)
1) Incident Reports
Paula Gordon: I understand that there were 6000 incident reports
received
by the Information Coordination Center (ICC) in the first five days of
the
year.
First, is this true? Were there 6000 incident reports received by
ICC in
the first five days of the year?
John Koskinen: I love rumors. While I don't know the final
number of
events, there were nothing like 6,000. If there were, we would have
advised
people of that fact to counter the media claims that Y2K really had not
been
much of a problem.
PG NOTE (4/4/2000): Since receiving Mr. Koskinen's response, I checked
again
with my primary source regarding the extraordinarily large number of
incident reports that were received in the first days following the
rollover. This source assures me that there were many thousands of
incident
reports within the first four to five days of the year. (The previous
mention of "6000" had been a ballpark figure.) It may be possible
that Mr.
Koskinen has discounted any reports of incidents that he felt were of "no
significance". It may also be that only the incidents that
were thought by
ICC staff to be significant were brought to his attention.
(PG): How many more reports have been received to date?
Has this data
been made available?
JK: Every glitch of any significance was made public since we needed
to
establish that the world really had met and conquered a significant
challenge.
(PG): If it has not been made available, will it be made available?
If it
cannot be made available because of the names of companies and
business that are mentioned, could the incident reports be made public
if
these names were deleted?
JK: The information will be available in the Presidential records.
PG NOTE 4/4/2000 It is good to know that these reports will
be made
available in "the Presidential records".
2)
The "Powering Down" of the National and Global Infrastructure
PG: I have heard a wide range of figures concerning the extent to
which
national and global infrastructure were "powered down".
Do you have any
data on this?
JK: No
(PG): Did the President's Council urge the "powering down"
of the
infrastructure?
JK: Absolutely not. We said, in the face of rumors that some
pipelines were
going to shut, that each company needed to make that decision on its own.
All we asked was that they coordinate their plans with their local
utilities and emergency managers.
Several transit authorities noted that they would stop for a few minutes
over the rollover, but, with our encouragement, they all made those
announcements to the public.
(PG): If so, did you have reason to believe that there
would be such a
widespread effort to "comply"?
JK: There is no information that there was a widespread effort to
comply.....
(PG): Or did "powering down" efforts come as a surprise to the President's
Council? I recall at the October 7 [1999]
briefing on the chemical industry
that there did not seem to be a call to "power down" chemical plants.
As I
recall, there was concern that shutting down production could have
its own
problems.
JK: We were not surprised because there's no evidence that a significant
powering down occurred.
PG NOTE 4/4/2000: There have been numerous reports by reputable
individuals in and out of government concerning the "powering down" of
specific sectors and regions. It is something of a mystery
that others close
to what was happening should have reported "powering down" efforts, while
Mr. Koskinen knows of "no evidence that a significant powering down
occurred."
3) The ICC
PG: Did the ICC fulfill its stated purpose?
JK: Yes
(PG): Did the purpose of the ICC change after the first week or so
of
January?
JK: No
(PG): To what extent did the ICC take a proactive stance?
For instance,
we had exchanged e-mail prior to the rollover concerning how the ICC
(or
IY2KCC?) would be alerting others of problems that might be triggered a
few
hours later in another part of the world. The case in point was a
Chinese
nuclear power plant that was nearly identical to ones in France.
The idea
was that if the facility in China had problems, the similar facilities
in
France would be immediately notified. Was this done?
By the same
token, was such action taken concerning the nine or so problems in
the first
hours at Japanese nuclear power plants?
JK: There was no occasion to warn others of glitches that occurred
that
might affect them since there were no such incidents. We publicized
fully
every glitch, including those with the monitoring systems at the Japanese
plants, as they occurred, but it was fairly clear that those were
local events
--which turned out to be the case.
PG NOTE 4/4/2000: Perhaps, the question was not
as clear as it might
have been. Prior to the rollover, it had been my understanding
that when a
specific instrument, system, or piece of machinery, etc. failed,
that
potentially interested parties in other parts of the world would be notified
so
that they would be forewarned of the possibility of similar failures.
4) The (December 13, 1999) Fact Sheet on Baseline Sector Data
[See
attached appendices]
PG: Does additional material exist on baseline data beyond this fact
sheet
and did ICC make use of additional baseline data beyond the data in the
fact sheet?
JK: Anything of interest or significance was published. Our
goals were to
get that information out, not keep it to ourselves.
(PG): What use was made of the baseline data that were compiled prior
to
the rollover?
JK: We educated the press and the public about the fact that things
go
wrong everyday and they should not assume that a failure on January 1 was
necessarily a Y2K failure.
Since there were not a significant number of Y2K failures, the benchmarks
were not needed or used to make judgments about whether failures
reported were Y2K or normal occurrences.
(PG): Why in some instances on the fact sheet is the focus
solely on a few
days period of time right around the rollover? Many IT consequences
as
well as embedded systems failures can take weeks, if not months to
manifest as problems.
JK: The focus generally was on the what happens every day -- not
just
December 31. If a failure occurred later, the benchmarks were still
relevant.
As you know, the fear that glitches would occur later and would be serious
turned out to be unfounded.
(PG): Comparative data that I have seen in all high hazard sectors
show an
incidence of problems in all of these sectors at record highs when compared
with the same time frame (January and/or February) in prior years.
Has
the ICC tracked the uncommonly high number of problems involving the oil
and gas sector, the chemical sector, nuclear power plants, planes,
and
trains?
JK: No.
(PG): Has the ICC been aware of the comparisons with prior year incidents
that others have worked on? I thought that this had been the reason
for ICC
collecting baseline data, so that such comparisons could be done.
For instance, I do not recall the ICC reporting anything about problems
with refinery or pipeline problems. According to some sources, the
problems have been at an all time high since the first of the year.
Indeed I
know of a report that shows that during the first part of January,
the number
of explosions involving natural gas, methane, and propane was over 1000%
higher than the same period in prior years. (This particular report
was
based on OSHA data, Product Safety Lists, and the UN's OSHA-like data
base.)
JK: The problems you cite have not been identified by anyone
with
knowledge of the events as having anything to do with Y2K.
(PG): Did the ICC gather information concerning refinery explosions
or
unscheduled maintenance and pipeline ruptures or explosions?
JK: We collected information that anyone responsibly could establish
was a
Y2K failure. While you've been focused on these explosions, they
have not
been identified as being Y2K related, nor have they created any significant
problems for the public or the economy.
(PG): If such information was not gathered by ICC, was it gathered
by any
government agency?
JK: Not for Y2K purposes. I don't know of any other collections
or analyses,
but all the agencies reported to us -- and we to the public -- Y2K
glitches. If we could have found a Y2K problem in a pipeline or refinery,
we would have been delighted to tell the press and the public about it
to
remind them of the significance of all the work that had been done in
safely moving the world into the 21st century.
(PG): If it was not gathered, why wasn't it gathered?
PG NOTE 4/4/2000: The responses to Topic 4 are most interesting.
Mr.
Koskinen is not denying that there were an excessive number of problems
of the kinds that I have noted. He is saying that he was not personally
aware of anyone "with knowledge of the events" who was making the
connection between Y2K and the excessive number of problems that were
occurring.
5)
Concerns Regarding Possible Electric Power Problems and Actual
Problems
PG: Joe Weiss [of the Electric Power Research Institute] stated at
the
Lawrence Livermore Conference in January that the industry did not
know
how things would turn out. One person I shared this with was
quite upset
when I shared this with him. He felt that had the public been told
about this
uncertainty that there would have been more effort to prepare just in case
something did go wrong. He feels that instead the public was subjected
to a
round of Russian Roulette that we luckily won. He does not feel that
this is
the way that public policy should be conducted. Do you have any
comments?
JK: Calling this Russian Roulette is silly. I'm sorry your
friend is upset,
but if you look back at the industry and government pronouncements you'll
see that the public was, in fact, warned to prepare for much more than
actually happened anywhere.
(PG): What is the reason that the ICC has made no mention of
the reports
around the time of the rollover of the thirty some problems with electric
utilities in the U.S.?
JK: I don't know where you're getting your information. No
one has
established that any problems with utilities were Y2K. The press
was all
over the place and they did not raise them this issue either. Your
sources
seem to be operating in the dark, so to speak.
(PG): What is the reason that the ICC has made no mention of the
reports
around the time of the rollover of the scores of problems with electric
utilities that occurred around the world?
JK: Because there were no such reports other than those we reported.
You
keep assuming that we were trying not to report problems. If you
reread my
briefings, you will see that we reported issues that were so minor people
normally would never mention them, because we were concerned that the
public and the press would conclude that Y2K was nothing but hype.
(PG): It appears to be the case that a problem or a failure was not
reported
as a "reportable failure" if the problem or failure was quickly worked
around
of if there was no disruption of service owing to "workarounds" or contingency
plans. It this correct?
JK: This was not our policy. I am sure that organizations here
and around
the world did fix some problems quickly and not report them and I so advised
the press during my briefings during the rollover.
(PG): It would be helpful if all the "non reported failures" could
now be
reported so that the public would understand how extensive the problems
had been and how great the possibility of disruptions had been. Also the
public would be apt to have a far better appreciation of the extraordinary
efforts that went into contingency planning and crisis management here
and
around the world. Can that story be told now?
JK: ..... I can assure you that we've been putting out information
-- as Bruce
McConnell did at the IYCC and the Senate report did -- on any reports we
have had about Y2K failures.
PG NOTE (4/4/2000): Several comments:
~ There is an untold story concerning the extent to which contingency
planning and crisis management efforts helped ensure that there were a
minimum of problems around the time of the rollover.
~ Failures or malfunctions were not always reported as "problems" if they
were "worked around" or other steps were taken that prevented larger
problems.
~ It is interesting to note the different problems that were reported by
different sources.
~ It should also be noted that there has been inadequate attention given
to
date to the disincentives to report problems in both the public and the
private sectors.
~ More will become known concerning private sector failures that were not
publicly reported during the first three months of the year with the filing
of quarterly reports, SEC reports, insurance claims, and law suits.
6)
Nuclear Power Plants Massive and Last Minute Turning Back of
Clocks
PG: It is my understanding that considerable effort was made in December
of 1999 to convince the nuclear power industry to roll back clocks to 1972
and that this was indeed done. Is this the case?
JK: Not to my knowledge. I have heard nothing about this.
(PG) To what extent did such a roll back take place?
JK: I have no information on this.
PG NOTE (4/4/2000): It is interesting that Mr. Koskinen did
not know about
these efforts that took place in December of 1999.
(PG) If it did occur, what are the implications for future remediation
efforts
that will now be needed to turn short term fixes into long term or permanent
fixes?
7) The Chemical Safety Board's CIRC Reports
PG: Were the CIRC reports a part of the ICC's collected data?
JK: The ICC received reports on Y2K failures from all agencies of
the
government. CIRC reports are not reports of Y2K failures.
(PG): It is my understanding that the CIRC reports are not
intended to track
all problems, that they represent only a portion of problems that occur
involving the chemical sector. If that is so, was ICC relying primarily
on
industry sources for reports?
JK: We relied upon industries, state and local governments and regulators,
and all Federal agencies for reports -- along with a monitoring of all
press
reports here and around the world.
PG NOTE (4/4/2000): The CIRC reports are not intended to capture
all
problems. Owing to litigation and liability issues, insurance
claims, and
bottomline concerns, it would be more surprising than not if the reports
of
incidents that have occurred since the first of the year included any
specific or detailed mention of suspected or proven Y2K-related IT systems
problems or failures or embedded systems, PLC-related, or SCADA system
problems or failures. It appears from what Mr. Koskinen was saying that
the
ICC was relying on industries and regulators among others to suggest any
possible connections with the problems that occurred and Y2K. If
no
connection to Y2K was identified or no possible connection was suspected
or
acknowledged, then it sounds as if no report of the problem would
have been
forwarded to the ICC in the first place.
8) EPA Reports
PG: Was the ICC getting reports from EPA?
JK: Yes.
(PG) Were those reports made public?
JK: Yes, to the extent that they provided any information on Y2K
glitches.
Otherwise, they were incorporated in our generaly conclusions of no
reported problems.
PG NOTE (4/4/2000) Again, owing to litigation and liability issues,
insurance claims, and bottomline concerns, it would be more surprising
than
not if the reports of incidents that have occurred since the first of the
year included any specific or detailed mention of suspected or proven
Y2K-related IT systems problems or failures or embedded systems,
PLC-related, or SCADA system problems or failures. It appears from what
Mr.
Koskinen was saying that the ICC was relying on industries and regulators
among others to suggest any possible connections with the problems that
occurred and Y2K. If no connection to Y2K was identified or no possible
connection was suspected or acknowledged, then it sounds as if no
report of
the problem would have been forwarded to the ICC in the first place.
9) DOT's Office of Pipeline Safety
PG: Was ICC getting reports from the Office of Pipeline Safety?
JK: Yes, through the Department of Transportation which monitored
that
information.
(PG): Were all such reports being tracked?
JK: Yes, the deparments compiled all Y2K information and forwarded
it to
experts in the ICC
PG NOTE (4/4/2000) As noted before, owing to litigation and liability
issues, insurance claims, and bottomline concerns, it would be more
surprising than not if the reports of incidents that have occurred since
the
first of the year included any specific or detailed mention of suspected
or
proven Y2K-related IT systems problems or failures or embedded systems,
PLC-related, or SCADA system problems or failures. It appears from what
Mr.
Koskinen was saying that the ICC was relying on industries and regulators
among others to suggest any possible connections with the problems that
occurred and Y2K. If no connection to Y2K was identified or no possible
connection was suspected or acknowledged, then it sounds as if no
report of
the problem would have been forwarded to the ICC in the first place.
10) Sectors in the US and Abroad That Were Not Fully Remediated
PG: Little has been said concerning those sectors that did not fully
remediate. If Iraq did in fact take a fix on failure approach to
its oil
and gas sector, then it should be in dire straits and in desperate
need of
replacement parts. Indeed, some are saying that Iraq is in desperate
need
of replacement parts.
Some in the oil and gas sector in the U.S. had stated in 1999 in SEC
filings and elsewhere that they were planning to fix on failure.
Could the
increase in problems in the U.S. be related to Y2K and embedded
systems-related problems?
JK: No one in authority has established this -- or even maintained
it.
PG NOTE (4/4/2000): For coverage concerning the UN and Iraq's
requests
for equipment, see the attached appendices. The International
Energy Agency
(IEA) in its report of May 1999 (also in the attached appendices)
anticipated that non-remediated embedded systems could result in refinery
problems.
11)
Force Majeures in the Oil and Gas Sector Since the Rollover
PG: Has the ICC made any mention of the four oil and gas sector-related
"Force Majeures" that were declared in various parts of the world
since the
rollover? One of these has been in the U.S. Was no connection
seen
between these and Y2K?
JK: No one with responsibility has maintained that these had anything
to do
with Y2K.
PG NOTE (4/4/2000) Force Majeures are extremely rare. To have
four
declared within such a short period of time would seem to be worthy of
investigation.
If no one at the Department of Energy, the Federal Energy Regulatory
Commission (FERC) or the IEA was actively looking into the possible
connection between Y2K and these Force Majeures, then there would have
been
no report forwarded to the ICC in the first place. It seems quite
unlikely
that ICC would have made such a determination independently. Again,
owing
to liability and litigation issues, insurance claims, and bottomline
concerns, it would be more surprising than not if the reports of the four
Force Majeures included any specific or detailed mention of suspected or
proven embedded systems, PLC-related, or SCADA system failures. It
appears
from what Mr. Koskinen was saying was that the ICC was relying on industries
and regulators among others to suggest any possible connections with the
problems that occurred and Y2K.
12).... Problems Being Reported [in One of the Largest Departments of
the Federal Government]
There have been reports concerning large numbers of problems.... [in one
of the largest departments of the government]. Some of these problems
appear
to have fallen into the category of "non-reportable failures".
Unless
there is a security concern, it would seem helpful to bring such problems
to
the attention of policymakers, since [the problems] appear to [have
significant] implications. The government's intelligence agencies
should
surely be aware as well.
JK: We received no such reports and are not aware of any such failures.
PG NOTE (4/4/2000): In the original version of this question
that I sent to
Mr. Koskinen 3/20/2000, I had identified the department in question.
This
may well be an instance of the failure of "bad news" to travel upward and
reach those in key roles of responsibility. I am puzzled that in
this
instance he expressed no interest in checking out the information I provided
him.
13)
Ramping Down and Apparent Phasing Out of Y2K-Related Federal Efforts
PG: At the present time, it is my understanding that all Federal
agency Y2K
efforts, including intelligence agencies and oversight efforts of
Congressional Committees, have been or are in the process of being phased
out. I don't know the extent to which the General Accounting
Office will
keep a focus on Y2K. The Manufacturing Extension Partnership Program
of
NIST is scheduled to be phased out in June. The only exception that
I know
of at this time is the "informal" effort that is to continue under the
aegis
of a Deputy or Assistant Director of OMB. Is this correct?
JK: I don't know that any of this, other than the ICC and the President's
Council are closing down, is correct.
PG NOTE (4/4/2000): I find this response puzzling as well.
(PG): If it is correct, who will be making sure that short
term fixes will be
replaced by permanent fixes? Who will be taking a long range view
and
considering the lessons that are being learned from Y2K and embedded systems
challenges.
JK: Those running organizations, in the public and private sectors,
including
CEO's and CIOs, are responsible for maintaining the integrity of their
operations.
PG NOTE (4/4/2000): To the best of my knowledge, the resources and
the
assemblage of talent focused on Y2K have been or are being dismantled
throughout the Federal Government, including the Congress. There
seems
to be little possibility that there will be support for any significant
ongoing efforts. Unless the General Accounting Office continues to
focus
on Y2K, there seems to be little or no chance that there will be any ongoing
assessment of Y2K efforts and impacts or any monitoring and assessment
of
current or future Y2K-related problems. This throws into question
the
ongoing efforts that are needed to identify and address current and future
problems. It also throws into question the completion of ongoing
efforts
that are needed to complete the remediation of partially or temporarily
remediated systems.
14) Challenges
in Addressing Continuing Y2K and Embedded Systems
Concerns within Agencies and Departments
PG: It has come to my attention that in one of the largest department
of the
Federal government, it has become politically incorrect to talk about Y2K.
Yet in this same Department, many non mission critical systems have yet
to
be remediated. Problems are talked about as "computer problems".
How is
it going to be possible to continue to dedicate expertise and resources
to
ongoing remediation problems in such a climate? Also, if those who
have
the expertise are not long with the agency or have been scattered
to
different parts of the agency, how will follow up work get done and how
will on going challenges be met?
JK: I am not aware of any agency with such a problem.
PG NOTE (4/4/2000): This comment is quite perplexing.
15) The Ongoing Role of OMB in Tracking Y2K and Embedded
Systems Problems
PG: Who will be in charge? Will this be a part time responsibility?
What will be the nature of these efforts? Will ongoing efforts
include the
tracking of what is currently an abnormally high incidence of problems,
explosions, and accidents, etc. involving the following sectors: the oil
and
gas sector, the chemical sector, nuclear power plants, planes, and trains?
Will problems involving water and sewage systems also be tracked?
Will
the progress be tracked of those sectors that did not remediate prior
to the
rollover?
JK: No one has found any indication that embedded chip problems resulted
in any significant problems thus far. Going forward, organizations
remain
responsible for their operations.
PG NOTE (4/4/2000): To the best of my knowledge, two persons from
the
Federal government with embedded systems expertise participated in the
November 9, 1999 meeting on embedded systems convened the
President's Council. A half dozen or so individuals from the
private sector
who had significant hands-on embedded systems expertise were also at that
meeting. When Mr. Koskinen says that "no one has found any
indication
that embedded chip problems resulted in any significant problems thus far",
several questions come to mind: Does "no one" mean no one in
government? Does "no one" means "no person with expertise in
embedded
systems"? If it means "no person with expertise in embedded systems",
I
know persons with embedded systems expertise who would disagree.
The
meaning of his statements may also turn on what he means by "any
significant problems". There have been numerous problems involving
loss
of life, property damage, and major ecological damage, all of which
have
either been connected to Y2K or have been strongly suspected of being
Y2K-related.
16)
Expertise in Government Regarding Embedded Systems
PG: Now that Gary Fisher of the National Institute for Standards
and
Technology is no longer working on embedded systems concerns at NIST,
who at NIST or elsewhere is going to be providing expertise on this matter
to
OMB?
Who else in the Federal government can OMB call on for expertise in
following these problems and conferring regarding policy?
I have not as yet been able to identify anyone as yet at the Department
of
Energy who is familiar with the prediction that embedded systems failures
could lead to an increase in refinery outages. The IEA report
of May 1999
for a discussion of the fact that the failure of embedded systems could
indeed result in refinery problems.
PG NOTE (4/4/2000) The IEA report is quoted in the attached appendices.
JK: As noted, you and a few others are the only remaining people
who still
are waiting for the fabled embedded chip disasters to occur. The
consensus
is that, fortunately, the problem was overstated and has not resulted in
any
major problems.
PG NOTE (4/4/2000): Embedded systems failures have occurred.
They
have not been widely acknowledged as yet as being problems relating to
Y2K. This was true before the rollover and it continues to be true
now. I
am not "waiting for embedded chip disasters to occur."
From my vantage
point, embedded systems failures are continuing to occur. The
number of
"predicted coincidences" grows daily. The increasing
number of incidents
of reported problems should not come as a surprise since many embedded
systems in a wide range of sectors were either not remediated or not
adequately remediated.
Mr. Koskinen formally conferred with a group of embedded systems experts
was on November 9, 1999. The intent of the meeting was to see
if
consensus could be reached regarding a number of issues involving
embedded systems, issues that had been vigorously debated for well over
a
year. The subsequent statement released by Mr. Koskinen, along with
a
related press release are included in the attached appendices. Additional
material on embedded systems is also included in the appendices.
If embedded systems experts helped guide post-rollover assessments, it
would be helpful to find out about their perspectives and their areas of
expertise.
17) Plane-Related Issues
PG: Is there anyone at National Transportation Safety Board (NTSB)
who
is looking into the possible connection between problems with the
automated systems of the MD 80 and 90 series of planes and the problems
that this series of planes has experienced? I spoke with the Chairman
of
the NTSB in early March and he indicated that this was not an angle that
was being explored by the NTSB at that time. I do not know if that
has
subsequently changed.
JK: Y2K has not been implicated in any of those issues -- no one
with any
real knowledge ever could find a Y2K problem in airplanes that threatened
their ability to fly.
PG NOTE (4/4/2000): A very long list of known and suspect
problems
could be cited. Some relevant new reports are noted here.
These are
from main stream media sources that are posted on the Grassroots
Information Coordination Center Web site at
http://www.greenspun.com/bboard/q-and-a.tcl?topic=Grassroots%20Informa
tion%20Coordination%20Center%20%28GICC%29
~ In the early hours of the rollover two planes in Europe developed
the
same problem within a half hour of each other. Both were grounded
as a
result until the problem was corrected.
~ Problems in the MD 80 and 90 series have occurred in abnormally
high
numbers during the first months of the year. Problems
with this series of
planes continue to occur to this day. The number of MD 80 and 90
series
problems during this time period far exceeds the number of problems
occurring in comparable time periods in previous years.
~ A back up computer system problem was involved in the serious problem
that another aircraft experienced in early March.
~ Another series of planes had electrical problems as of 4/4/2000.
It would not surprise me if no one at the NTSB or the FAA has as yet begun
to look seriously into a possible Y2K connection and the exceedingly high
number of problems involving planes that have occurred since the beginning
of the year. If no one has been looking into such a connection,
it may only
mean that no software engineers with the necessary expertise have as yet
come forward or none have as yet been called on to confer concerning the
possible connection between Y2K, electrical and automated systems
problems or failures, and the problems that have been occurring.
As with
other matters that involve complex technical issues, it can be hard
for
officials who do not have that particular technical expertise to identify
those
who do, seek their counsel, benefit from their assessments and advice,
and
incorporate what they learn into decisionmaking, policymaking, and problem
solving processes.
18)
Short Term Fixes in General
PG: Is anyone in the Federal government looking at the issue
of what
needs to be done to ensure that all the various kinds of short term fixes
that
were implemented will be replaced by permanent fixes?
JK: CIOs. [Chief Information Officers]
(PG): With the disbanding of offices and teams that addressed
Y2K issues,
who will be overseeing and driving the remediation on the non mission
critical systems that were not remediated prior to the rollover.
JK: CIOs.
19)
Your Comments Concerning the Valued Role Played by Those
Who Raised the Concern of the Public Regarding Y2K
PG: In late January in a [State Department press release
dated January
27] you were quoted as saying that you were of the opinion two years ago
that it would be "the end of the world as we knew it" if the necessary
remediation was not done in time. You gave credit to those who raised
the
alarm for helping to focus attention on the problem.
Is this an accurate
restatement of what you said? If not, would you clarify
what you meant?
PG NOTE 4/4/2000 His words in that interview were as
follows:
"It was clear two years ago to me after talking with a lot of experts,
if
nobody did anything else beyond what they had already done up until two
years ago, that the world as we knew it would end."
The January 27, 2000
transcript of the interview is in the attached appendices.)
JK: .....I said that many times and continue to believe it.
However, I
think it is now time to declare victory and move on. Claiming
that the
world is still about to end casts doubt on the good work done before.
PG
NOTE (4/4/2000) During the months prior to the rollover,
my impact
ratings regarding the possible long term impact of Y2K were provisionally
between a 5.5 and 9.5. This estimate was contingent upon the extent
to
which the public sector, as well as the private sector succeeded in
efforts to
minimize impacts. In my January 17 Comments piece,
I offered my
impact rating for the first quarter of the year only. That rating
was
provisionally between a 2.5 and 5.5, this time depending on several
factors,
including the severity of fuel shortfalls. (http://users.rcn.com/pgordon/y2k/,
click on "Comments, Essays, and Op-Ed Pieces"). (The 10 point
impact
survey scale referred to here is described in Part 1 of my White Paper
on
Y2K at the same URL just cited.)
I have come to understand that much remains to be publicly disclosed
concerning all of the efforts that went into minimizing Y2K impacts.
This
includes unanticipated as well as unpublicized actions that were taken
prior
to the rollover. In some instances such actions were taken as late
as
December. It also includes the extensive contingency planning
and crisis
management efforts that helped minimize problems here and abroad.
I
hope that far more become known concerning the efforts that were made.
My intention is certainly not to cast doubt on the extraordinary
accomplishments of the thousands of people here and around the world to
avert major problems.
I have come to understand that, for a variety of reasons, the problems
that
have occurred and are still occurring are not being acknowledged as being
Y2K-related or possibly Y2K-related. Since official government monitoring
efforts of the sort needed are not in place at this time, it will not be
an easy
task to assess what has happened and what is happening. In addition,
there is no assurance that ongoing problems are being adequately addressed
now. There is no assurance that they will be adequately addressed in the
future. I would also like to see the continuation of efforts
that are
still needed.
I think that there is a chance that the evidence concerning the nature
and
scope of the problems that have occurred and of problems that are ongoing
will become more widely acknowledged as time goes by. There
appears to
be a whole host of legal, business, social/psychological, organizational,
economic, and even political reasons that to date have been serving as
barriers to such acknowledgment.
(PG): If this is an accurate rendering of what you said in January
of this
year, I wonder why you didn't say it earlier? [The reference
here goes back
to Mr. Koskinen's statements quoted in the January 27 State Department
press release that were noted at the beginning of this section.]
JK: ....... I made comments like that from the start. I continually
noted
that my disagreement with those thinking the world would come to an end
was
not a disagreement about the magnitude of the problem but with their view
that the problem could not be solved -- or could not be solved without
a
national or world-wide declaration of emergency with the public advised
to
be prepared for weeks or months of problems. I thought then -- and
was
proved correct -- that we could and would solve this problem if we could
organize the government, the US economy, and countries around the
world to deal effectively with the issue. That was done.
PG NOTE (4/4/2000): Worst case Y2K scenarios involving a simultaneous
convergence of infrastructure disruptions and technological disasters,
now
or in the foreseeable future, have been averted. This
is certainly a great
and most welcome blessing. The fact that worst case scenarios have
been
averted is not however tantamount to "solving the problem".
Problems of
a lesser magnitude and less daunting character remain. Not
all of the wide
range of problems that remain are being openly acknowledged and
addressed. Some in fact are not being acknowledged or addressed at
all.
It seems to me that a scenario is unfolding in which neither the public
nor
the private sector is doing all that needs to be done to complete the work
that was begun. Neither sector appears to be taking steps to forestall
problems that could yet emerge over the next year or more.
Neither appear
to be focusing adequate attention on what is happening now.
Problems
continue to occur. They are not receiving adequate attention.
They are not
being adequately addressed. A scenario is continuing to unfold
that is
costly in terms of loss of life, public health and safety consequences,
environmental impacts, and socioeconomic impacts, certainly not as costly
as the worst case scenarios, but nonetheless costly. The public
and private
sectors appear to have declared victory prematurely. In doing
so, support
has been withdrawn from ongoing efforts that are needed.
Challenges
remain even though the dimensions of the problem have been greatly
reduced. If these Y2K-related challenges that remain are ignored
or fail to
be adequately addressed, needless losses will continue to occur and
needless harm will result. The apparent reluctance to deal
with ongoing
problems augurs poorly for our ability to deal with future challenges and
threats that may prove even more daunting than Y2K.
20)
Ongoing and Future Problems Involving Technology
PG: As corroborated by the notes of some closed door meetings involving
the President's Council in December 1998 and January 1999, it has been
concluded by some that the Council was consciously trying to keep the full
potential seriousness of Y2K from the public and to do so through a
public
relations campaign that was designed with that purpose in mind.
If that
is an accurate description of what actually happened, then these questions
follow:
JK: ....... you know that is not an accurate description of
those meetings or
our policies......
PG NOTE (4/4/2000): My perception of the direction that the
Council's
public information efforts were taking was corroborated by my reading of
the
official notes of the December 16, 1998 meeting of the Council. These
notes can be requested under the Freedom of Information Act. The
document is entitled President's Council on Year 2000 Conversion
Meeting
Minutes. The meeting was convened by the Chair of the President's
Council at 2:05 p.m. on December 16, 1998 in Meeting Room E of the
Federal Reserve Building, 20th and C Streets, NW, Washington, DC.
Perhaps these minutes will also be made available in "Presidential papers".
This meeting included a discussion concerning "ways to work with the
media". The meeting minutes were widely distributed in the
early part of
1999. In this way the substance of that discussion became
known to many
people outside of government. An attendee of the meeting on
realizing that
I had seen the minutes confirmed the substance of this discussion.
My perceptions were also confirmed when I heard some plenary panel
presentations at the Second Global Y2K National Coordinators Meeting at
the United Nations in New York on June 22, 1999. These presentations
were by representatives from two extremely prestigious media
organizations. The presenters openly noted their adoption of
the approach
to Y2K and public information that the President's Council had been urging.
Additional perspectives concerning the nature of the Council's approach
to
public information efforts can be found in a paper that focuses on the
health
sector and the pharmaceutical industry issued by the Center for Y2K &
Society in March, 2000. (Http://www.y2kcenter.org).
(PG): If and when the public comes to realize the nature of
the public
relations campaign that was carried out, will they loose faith in their
government?
Will the public believe their government in the future if they come to
believe that in the recent past their government chose to only partially
inform them concerning the possible threats that we were facing?
What will the implications of such "managing of perceptions" be for
addressing future problems that constitute a threat to the public?
If you had it to do all over again, what would you change, if anything,
concerning the Council's apparent efforts to shape public perceptions
concerning the nature and scope of the problem?
JK: Absolutely not, since your assumption that we were somehow
nefariously shaping perceptions is incorrect. We spent significant
resources
trying to bring facts -- not assertions -- to the public, comfortable that
they
would respond appropriately to those facts. We did and they did.
In the
end, the polls all showed that they believed the reports from federal,
state
and local governments and individual critical infrastructure companies
that
they were ready for Y2K. And, in the end, to the dismay of some diehards,
those reports and facts turned out to be absolutely correct. I think
that's what the public will remember about Y2K.
21) Small Size of the Staff of the Council (11 as of December 1999?)
PG: If you had it to do all over again, what would you change?
JK: Not one body.
(PG): If you had it to do all over again, would you include
technical experts
on your staff?
JK: No.
22) General Overall Approach
PG: If you had it to do all over again, what would you change,
if anything?
JK: Nothing, including all the time I've spent discussing the issue
with those
who disagreed with our approach and criticized us vigorously, even though,
after the fact, we were right and they were wrong. It was an important
dialogue and I'd do it again, as shown by the time I'm taking to respond
to
these questions.
23) Insurance Claims and Law Suits
PG: Has the Council or the ICC been tracking insurance claims and
law suits
relating to Y2K? I heard in January that a reinsurer for oil
refineries
had an unprecedented number of claims in the first weeks of the year.
Have
you heard similar information?
JK: No.
PG NOTE (4/42000) I based my question in part on a communication
that I
received on January 23 from the head of a software engineering company.
He told of a long time associate of his who is a reinsurer who had over
100
incidents reported in the first 20 days of the year that the reinsurer
expected
would likely "end up being claims". The reinsurer noted that
this was
"atypical in the extreme." Normally this company had only one or
two claims
for the month of January with annual totals running near 40 only "in a
bad
year". According to my contact, the firm reinsures industrial policies,
primarily "manufacturing, fuel, distillates, and transportation".
It seems
important to track publicly available information concerning insurance
claims
along with related litigation, as a means of increasing understanding
concerning the nature and extent of Y2K-related impacts. Such understanding
is needed in order to assess past efforts to address Y2K and
inform current and future efforts.
24) Specific Report of the Potential for a Chemical Plant Explosion with
Major Environmental Consequences in [Another Country]
Months ago [August 3, 1999], a copy of an e-mail was sent to me in error
by
someone [in the Federal Government]. I received an e-mail from you
the
same day asking me not to pass that e-mail on to others owing to the
sensitivity of the contents. I understand that millions of
dollars went into
doing the necessary remediation in [several plants] and averting
a major
problem. I wonder if it is now possible to talk about this problem
publicly. If the name of the country should not be mentioned, can
the part
of the world
be mentioned and some specifics about the seriousness of problem that
had been averted? I think that information concerning such problems
can
be very convincing to those who doubt that the work that was done was a
worthwhile expenditure of time and money. I think that information
about
such problems is also important for the public, the media, and public officials
to know, so that they will more fully appreciate Y2K related threats and
challenges.
JK: It was not clear then and is not now whether there was a major
Y2K
threat in that plant.
PG NOTE (4/4/2000): The problem involved several plants located
in a
heavily populated area. The problem was acknowledged prior
to the
rollover as being Y2K-related. This statement concerning
the situation is
not in keeping with my reading of the "sensitive" e-mail that was
inadvertently sent to me or with other information that came to my attention
several months ago.
25)
The Role that Multinational Corporations Played in Minimizing
Infrastructure Problems in Other Parts of the World
PG: There seems to be an untold story here. I hope that the
role that
corporations played can be made public. They certainly deserve a
great
deal of credit. Will there be some focus on this in your final
report?
JK: You are very correct here. Not only do they deserve credit,
but their
work around the world with their own facilities, with host governments
and
through information sharing with competitors was a significant part of
the
reason why the rest of the world did so well. We'll try to bring
that
point to people's attention in our report and I appreciate your focus on
it.
PG NOTE: On page 22 of the Council's March 29, 2000 Final Report,
the
following brief statement is found: "And in many industries,
large
multi-national companies actually worked directly with their local
counterparts and host countries to fix basic systems."
ENDNOTE
Our viewpoints concerning Y2K and the past two years obviously differ
greatly in many ways. Because of these differences, I appreciate
even more
the time that John Koskinen took to respond to the questions I sent him.
This exchange has been helpful to me and I hope it will be of help to others
who are continuing to try to make sense of what has happened and what is
happening now. I hope as a consequence that more positive energy
might be
directed toward ongoing efforts that are needed. I hope that people
will
become acquainted with sources of information and other materials that
will
help advance their understanding. Increasing understanding on all
fronts
seems key to increasing our capacity to address current as well as future
challenges.
**************************************************************************
*******
APPENDICES
1) Transcript of State Department Interview with John Koskinen
27 January 2000
Transcript: What Happened to Y2K? Koskinen Speaks Out
http://www.usia.gov/cgi-bin/washfile/display.pl?p=/products/washfile/latest&
f=00012704.glt&t=/products/washfile/newsitem.shtml
(Administration Y2K coordinator assesses global remediation) (4,650)
The costly effort undertaken in the past two years to deal with the
Year 2000 computer problem prevented massive disruptions in systems
and services during the date rollover into the new millennium,
according to White House Y2K coordinator John Koskinen.
Koskinen, Chair of the President's Council on Year 2000 Conversion,
said in a January 18 interview in Washington that the relatively
problem-free date change that occurred is an indication not that the
Y2K problem was not serious, but that the work devoted to fixing
thousands of computer systems worldwide was successful.
Koskinen said the absence of serious Y2K disruptions in developing
countries, where remediation efforts had lagged behind those in
industrial countries, is explained by the less intense reliance in
those countries on digital technology, and by the fact that they were
able to apply the lessons learned from dealing with the problem
elsewhere.
Koskinen spoke with the Office of International Information Program's
Paul Malamud about the smooth transition into the year 2000, and the
work that made it possible.
Following is a transcript of the interview. In the transcript,
"billion" equals 1,000 million.
(begin transcript)
Q: January 1 has come and gone, and reports show that there were fewer
disruptions of computer operations and infrastructure, on a global
basis, than some had feared. In retrospect, do you feel the advance
publicity and the large amount of money that went into fixing computer
systems worldwide was overblown? Could this have been handled by
smaller "fixes" performed on an ad-hoc basis after January 1?
A: I think a lot of people did do it in an ad hoc way, at the end, and
seem to have gotten through it well. However, for organizations using
large information technology structures there was no way they could do
it at the last minute.
The major banks around the world worked on this for several years
together, because you are talking about organizations that have
millions of lines of software in code that had to be fixed. In fact,
one of the reasons that people thought the world, as a whole, was
going to have difficulty was that it takes so long to work through
those big systems.
You have to distinguish governmental organizations and private-sector
companies that had major software problems from organizations that had
more straightforward information technology challenges. I think what
happened was that some smaller organizations and governments have less
reliance on complicated systems, and therefore, a lot of their systems
either were not significantly affected by Y2K or they could take care
of those in a relatively short period of time for relatively little
money.
When people started working on Y2K no one knew exactly the full impact
of potential failures involving large networks of computers. In
addition, no one knew where in power plants, telephones systems,
chemical plants, date-sensitive "embedded processors" might have a Y2K
problem or not. My favorite example is elevators. Two or three years
ago, the assumption was that elevators were at risk. There was concern
that some elevators -- if they were dependent on date-sensitive
computer chips -- might malfunction. But after about a year of
testing, it turned out elevators did not have a problem. This meant
|