[extropy-chat] FWD (UFO UpDate) Re: UFO Research Software?

[Terry W. Colvin]

From: Isaac Koi <isaackoi2 at yahoo.co.uk>
To: <ufoupdates at virtuallystrange.net>
Date: Sat, 12 Feb 2005 15:45:46 -0000
Subject: Re: UFO Research Software?
 

>From: Nikolay Subbotin <paragloss at mail.ru>
>To: ufoupdates at virtuallystrange.net
>Date: Tue, 08 Feb 2005 15:24:41 +0500
>Subject: UFO Research Software?

>Who uses special UFO research software? I cannot find any
>software for professional UFO research.

A lot depends upon what "UFO research software" you want.

When the topic of UFO databases and software arose a few weeks 
ago, I posted a list of references to discussions of some of the 
major existing databases (including UFOCAT and Larry Hatch's *U* 
database). Since the topic has recurred already, I'm giving in 
to the temptation to send a more comprehensive email.

However, as always, time is a bit limited right now so I've had 
to leave quite a bit of material until I return to this topic 
upon finishing my damn Chronology.

As a starting point, I find it useful to split up the various 
types of software/databases that ufologists use. In this email, 
I'll divide my comments into the following categories:

A. Databases of UFO reports;

B. Databases of other information;

C. Expert systems to assist in identifying possible stimuli for 
a report;

D. Other software to assist in investigations and research.

At the very minimum, I consider this exercise to be worthwhile 
because it may assist some of the various individuals that 
appear to be putting considerable time and effort into 
developing their own databases/software. Also, the usefulness of 
databases and other software merely as bibliographical tools 
should not be underestimated given the sheer mass of literature 
and documentation relating to UFO reports. However, before 
launching into these topics, I'll just note a few cautionary 
remarks in relation to the use of computers within ufology:

(1) "Computers are a powerful tool which properly used will give 
enormous assistance to ufologists the world over... but it 
should be recognised from the outset that they alone will not 
answer the questions. [T]he UFO enigma will not be answered by 
computers but by the talented and intuitive thinking of human 
minds" per Spencer, John and Vallee, Jacques and Verga, Maurizio 
in "UFO: 1947-1987" (1987) (edited by Hilary Evans with John 
Spencer) at page 245 of the Fortean Tomes softcover edition (in 
Chapter 3.6, entitled "Computers in Ufology").

(2) "Poor data will merely produce the wrong answer more quickly 
on a computer. No technology or technique will compensate for 
deficient data." per Peter Hill, quoted in Phenomenon (1988) 
(edited by John Spencer and Hilary Evans) at page 224.

(3) "The well-known phrase "garbage in, garbage out' applies 
equally well to ufology." per Gamble, Stephen and Wootten, 
Michael and Danby, J and Smith, Willy and Kuhlemann, Bertil in 
"Phenomenon" (1988) (edited by John Spencer and Hilary Evans) at 
pages 224-237 of the MacDonald hardback edition (in Part 3, in 
the unnumbered chapter entitled "Harnessing the Computer").

(4) See also the remarks by Brad Sparks on UpDates at the 
following link:

http://www.virtuallystrange.net/ufo/updates/2005/jan/m09-010.shtml

With these caveats firmly in mind, I turn to the categories I 
outlined above.

A. Databases of UFO reports

Given that many ufologists (and non-ufologists) in modern 
society appear to be almost compulsive list-makers, it is not 
surprising that there are already a wide variety of dabases of 
UFO Reports.

Heck, there are already a considerable number of lists of 
databases (in effect, databases of databases - or "databases 
squared").

What is more surprising is that these databases rarely seem to 
be referred to by other individuals that are considering 
developing their own databases.

If I were to attempt to prepare a comprehensive list of 
databases from scratch, I would attempt to divide existing 
databases into various categories (e.g. according to the method 
of storage or access (such as online, computerised and paper 
based), or according to the type of data stored (e.g. worldwide 
reports, regional reports, or specialised (e.g. pilot sightings, 
EM reports) and gradually build up a comprehensive list of 
databases within each catagory.

However, given the existence of some attempts to list databases 
already, I think a more efficient and systematic approach would 
be to begin by starting by producing a list of references to 
lists of databases (in effect, a database of databases of 
databases, or a list of databases squared, or a "database 
cubed").

I'll begin with a list of online lists of databases (most of 
which are useful as guides to databases available online, but 
are rather weak in relation to databases supplied on CD or on 
paper):

(a) Mark Cashman's list of catalogues at the following link, 
which is clearly presented and useful (but rather limited):

http://www.temporaldoorway.com/ufo/catalog/index.htm

(b) Potentially more comprehensive, but a bit hit and miss in 
its coverage, is the following page on Francis Ridge's "NICAP" 
website. That page refers to various categories or "groups" of 
sightings. Clicking on a "group" displays a page relating to 
that category of sighting that generally begins with a list of 
databases or analyses relevant to that category.

http://www.nicap.dabsol.co.uk/special.htm

(c) Project 1947 provides a list of catalogues (which appears to 
be generally limited to those by contributors to the Project) 
at:

http://www.project1947.com/47cats.htm

(d) A slightly bare list of databases (which includes several 
regional databases rarely mentioned elsewhere) is provided by 
SUFOI at the following link:

http://www.sufoi.dk/artik-sn/new12-08.htm

(e) Few of the many computer software projects currently in 
development give any indication that existing databases/software 
were reviewed before launching into the new project. One of the 
few exceptions is the RR0 project being run by Jerome Beau, 
which not only includes a limited list of "alternatives and 
competition" but also (extremely briefly and not entirely 
clearly (possibly because of the somewhat stilted 
English/jargon)) attempts to define what is different about the 
proposed project. See the "alternatives and competition" table 
and the remarks below it at the following link:


https://sourceforge.net/docman/display_doc.php?docid=17408&group_id=70060#AlternativesAndCompetition

(f) A very brief list of databases is given by Terry Groff on his
UFO Tools website at the following link:

http://www.terrygroff.com/ufotools/statistics.html

The most striking thing about these lists, to me at least, is 
that there is very limited overlap in the lists of databases. It 
seems to me that merely combining these lists would generate a 
more comprehensive list of UFO databases than is currently 
available on the Internet.

Even more striking is the fact that the lists of databases and 
catalogues that appear in print also have extremely limited 
overlap with the above lists. For example, UFO databases are 
listed and/or discussed in the following:

Evans, Hilary in "UFO: 1947-1987" (1987) (edited by Hilary Evans 
with John Spencer) at page 46 of the Fortean Tomes softcover 
edition (Chapter 2.3.1, entitled "UFOs as Global Phenomenon").

Hall, Richard in "The UFO Evidence: Volume 2 - A Thirty Year 
Report" (2001) (edited by Richard Hall) at pages 646-647 (in 
Section 16) of the Scarecrow Press hardback edition.

Hynek, J Allen and Vallee, Jacques in their "The Edge of 
Reality" (1975) at pages 76, 78-82 (in Chapter 3) of the Henry 
Regnery hardback edition.

Gamble, Stephen and Wootten, Michael and Danby, J and Smith, 
Willy and Kuhlemann, Bertil in "Phenomenon" (1988) (edited by 
John Spencer and Hilary Evans) at pages 224-237 of the MacDonald 
hardback edition (Part 3, in the unnumbered chapter entitled 
"Harnessing the Computer").

Randles, Jenny and Warrington, Peter in their "UFOs : A British 
Viewpoint" (1979) at pages 180-181 (in Chapter 11) of the Book 
Club Associates hardback edition.

Randles, Jenny and Warrington, Peter in their "Science and the 
UFOs" (1985) at page 60 (in Chapter 4) of the Blackwell hardback 
edition.

Spencer, John and Vallee, Jacques and Verga, Maurizio in "UFO: 
1947-1987" (1987) (edited by Hilary Evans with John Spencer) at 
pages 238-245 of the Fortean Tomes softcover edition (in Chapter 
3.6, entitled "Computers in Ufology").

Sturrock, Peter in his "The UFO Enigma" (1999) at pages 166-167
(in Chapter 24) of the Warner Aspect hardback edition.

Westrum, Ronald M in "UFO Phenomena and the Behavioral 
Scientist" (1979) (edited by Richard F Haines) at pages 104-106 
(in Chapter 5) of the Scarebrow Press hardback edition.

The above lists are an attempt at a database cubed. I've started 
to generate my own database squared (i.e. a list of computer 
databases), by listing the databases listed in the 
webpages/discussions and other databases I've read about 
elsewhere (or have obtained).

Before I spend much more time on this project, I'd invite anyone 
that knows of other lists of databases to add to the above 
database cubed to do so.

Of the above list of existing lists of databases, I would 
heavily highlight in particular the article by Spencer, John and 
Vallee, Jacques and Verga, Maurizio in "UFO: 1947-1987" (1987) 
(edited by Hilary Evans with John Spencer) at pages 238-245 of 
the Fortean Tomes softcover edition (in Chapter 3.6, entitled 
"Computers in Ufology").

That article discusses a considerable number of existing 
databases. Interestingly, I don't think I've read about most of 
those databases since that article was printed in 1987. It would 
be interesting to follow up on the status and availability of 
those databases. A few hours of effort in following up the 
availability of programs or databases that took week or months 
to produce could be very rewarding.

I note in particular the following from that article (at the top 
of page 242): " This is the only publication in the world 
exclusively devoted to the use and application of computers in 
ufology. A lot of international researchers contribute to the 
[Computer UFO Newsletter] edited by Maurizio Verga. with 
articles on research projects, ready programs, proposals of 
common works and new software. There is a column, 'Offers of 
software', where there is an offer at cost price of all UFO 
programs available at the moment (about 30) for different kinds 
of computers.".

Presumably, if the authors of the relevant programs were 
prepared to make the programs available at cost price, some or 
all of them would be prepared to make them available on a 
website (such as Terry's "UFO Tools" website).

I note that the Newsletter is referred to on Maurizio Verga's 
website at the following link, but I don't know whether the 
newsletters that were produced (or the relevant programs) are 
already available online or how useful they would be.

http://www.ufo.it/verga.htm

Another previous effort that I would be interested in knowing 
more about (and may be worth noting by those that are working 
on, or thinking about, generating their own database) is the 
International Committee for UFO Research ("ICUR"). That 
organisation made an effort to consider how more comprehensive 
international databases of UFO reports could be generated and 
how (if at all) the data in various databases could be 
standardised. See the links below:

http://members.rogers.com/vlourenco/mufon/hais02.htm

http://dspace.dial.pipex.com/town/square/el82/icur.htm

That Committee boasted an impressive list of members, including 
BUFORA, CUFOS, Project UNICAT, Project URD, SUFOI and others. 
I'm aware of some background on the Committee (see the short 
list of references below), but am quite out of date. I'd like to 
know far more about this interesting endeavour. Can anyone point 
me to more up to date information? Are any of the members of the 
executive of that Committee on this List? How active was/is the 
Committee? Did it issue any reports or substantial minutes of 
its deliberations?

Some references for the International Committee for UFO Research 
(in addition to the 2 hyperlinks given above):

Blevins, Dave in his "UFO Directory International" (2003) at 
pages 89-90 (in Part 2) of the McF softcover edition.

Gamble, Stephen and Wootten, Michael and Danby, J. and Smith,
Willy and Kuhlemann, Bertil in "Phenomenon" (1988) (edited by
John Spencer and Hilary Evans) at page 224 of the MacDonald
hardback edition (Part 3, in the unnumbered chapter entitled
"Harnessing the Computer").

Randles, Jenny in her "UFO Reality" (1983) at page 52 (in 
Chapter 3) of the Hale hardback edition.

West, Arnold in "Phenomenon" (1988) (edited by John Spencer and 
Hilary Evans) at page 12 of the MacDonald hardback edition (in 
the unnumbered chapter entitled "About BUFORA and ICUR").


In this part of this email I'm merely seeking to outline how a 
comprehensive list of existing computer databases could be 
produced, not to give a list of them. (A draft list I'm working 
on is probably too long to include in this email). However, it 
would be remiss of me to fail to give a couple of comments on 
the two offline giants of the UFO database world: UFOCAT and 
Larry Hatch's *U* database.

UFOCAT: I don't think that there's any real doubt that UFOCAT is
the most famous and largest offline UFO database.

I gave a list of references to discussion of UFOCAT cut and 
paste from an incomplete draft of my Chronology in my email at 
the following link:

http://www.virtuallystrange.net/ufo/updates/2005/jan/m10-017.shtml

See also the CUFOS and UFOCAT webpages:

http://www.cufos.org/UFOCAT.html
http://www.ufocat.com/

As I remarked in that email, I think it would be in the 
interests of ufology and CUFOS for the manual for UFOCAT to be 
made available on the internet. The manual hints at the wealth 
of data and bibliographical references on various topics that 
can be extracted from UFOCAT. Also, the UFOCAT database (which 
runs on Microsoft's Access) comes with various pre-prepared 
lists relating to particularly types of sightings etc. I would 
have thought it would be good advertising for UFOCAT for one or 
more of those lists to be made freely available on the CUFOS 
website.

I'd also note the following comment from page 5 of the UFOCAT 
2002 Manual: "We would first caution potential users not to 
expect to be able to begin and end their research using only 
UFOCAT 2002-there are too many gaps in the data and, just like 
the Internet, not every source of information is as reliable and 
accurate as the next. The results obtained from UFOCAT 2002 are 
best thought of as a reference guide to the original sources for 
the crucial details. Otherwise, the distinction between poorly 
investigated reports and exhaustively studied sightings will be 
lost. However, you will substantially improve your search for 
information by accessing UFOCAT 2002. What was true when Allan 
Hendry wrote his critique of UFOCAT in 1979 is even truer today: 
UFOCAT 2002 is without peer as a reference source. Thousands of 
hours went into creating it, and months have gone into revising 
it to improve its ease of use. It exists today as the most 
comprehensive reference tool and bibliographic source on UFO 
reports in existence."


Larry Hatch's *U* Database - Given Larry's frequent posts to 
Updates, his database probably does not require any introduction 
or any reference to his website at the following link: 
http://www.larryhatch.net/

As far as I've seen, this database has not discussed in many 
books so far. However, Larry's objective appears to be very 
similar to that of Dr Willy Smith's Project UNICAT (i.e. a 
filtered catalogue of higher quality UFO reports). Project 
UNICAT's database has been discussed in several of the 
references given above, and elsewhere (e.g. in the entry 
entitled "UNICAT Project" at pages 943-944 of Jerome Clark's 
"UFO Encylopedia 2nd Edition : Volume 2 L-Z:" (1998).

Larry's database is currently only available as a Microsoft DOS 
program and its appearance is a rather basic. The sound effects 
may have been cutting edge for DOS software but are now simply a 
bit irritating. However, these rather superficial issues should 
not cause the database itself to be underestimated. The database 
is a useful tool and I look forward to seeing Larry release a 
new version of his database once its been given a new, glossy, 
Windows user interface. (Again, I find it interesting that the 
references given by Larry's database rarely seem to overlap with 
the references given for the same sightings by UFOCAT).


B. Databases of other information

Ah, well, this is a rather wide category of a mass of 
(generally) smaller databases.....

For example, there are lists/databases of different types of IFOs
(e.g. Menzel's list, which is now online at:

http://www.cufon.org/cufon/ifo_list.htm

or lists relating to a particular type of IFO (e.g. the list of 
clouds (with photos) for which a link is given on Terry's UFO 
Tools website).

More significant are the various bibliographies (by Catoe etc).
I won't attempt to list the existing bibliographies in this
email, but will simply note that several of the existing
bibliographies contain sections which are devoted to listing
bibliographies - see, in particular, the following:

(1) Codes LB and LBA in the excellent online database produced 
by the AFU, at the following link:

http://www.afu.info/booksbycodeL.htm

(2) US Library of Congress, Tracer Bullet 91-1 "Unidentified 
Flying Objects (UFOs)" containing section entitled 
"Bibliographies", available online at:

http://www.loc.gov/rr/scitech/tracer-bullets/ufostb.html

Surprising limited, unless I'm missing something, are existing 
indexes/databases of government documents. Of course, there are 
some lists (e.g. Brad Sparks' list of Project Blue Book 
"Unknowns"), but I've seen far fewer such indexes than would be 
useful. If someone has already compiled a list of such 
indexes/databases, I'd be grateful if they could let me know as 
it would produce a short cut for the database squared I'm 
producing.

The range of other existing lists/databases that might be of 
interest to ufologists is almost unlimited, for example:

(1) lists of SETI projects (such as those presented by Darling, 
David in his "The Extraterrestrial Encyclopedia" (2000) at pages 
378-383 (in the table entitled "SETI Observing Programs: 1960 to 
the Present") of the Three Rivers softcover edition and Jill 
Tarter's list in "Extraterrestrials: Science and alien 
intelligence" (1985) (edited by Edward Regis) as her tabular 
Appendix entitled "Archive of SETI observing programs 1959-84" 
at page 192 of the Cambridge University Press softcover edition.

(2) various lists of movies involving UFOS/aliens, including:

a. The list entitled "A Checklist of ETs in the Cinema" 
presented by Chris Boyce in in his "Extraterrestrial Encounter" 
(1979) at page 164 (in Appendix 1) of the David & Charles 
hardback edition, at page 152 of the 1980 revised NEL paperback 
edition.

b. The list of science fiction films with themes of either
visitors from space, or travelling to space or both presented by
Armando Simon in "UFO Phenomena and the Behavioral Scientist"
(1979) (edited by Richard F Haines) at page 53 (in Chapter 3) of
the Scarebrow Press hardback edition.

c. The list entitled "Alien Inspired Movies" presented by 
Kurland, Michael in his "The Complete Idiot's Guide to 
Extraterrestrial Intelligence" (1999) at page 290 (in Chapter 
28) of the Alpha Books softcover edition, and included in 
Appendix E at pages 315-316.

(3) List of names of supposed extraterrestrials relating to UFO 
sightings/contactees presented by Paul Christopher in his "Alien 
Intervention" (1998) at pages 81-82 (in Chapter 5) of the 
Huntington House softcover edition.

etc., etc., etc.


C. Expert systems to assist in identifying possible stimuli for 
a report

Jacques Vallee has written about an expert system called 
OVNIBASE that he developed using NEXPERT SYSTEM (developed by 
Neuron Data, Inc) to implement a screening system which could be 
operated by clerical personnel with the objective of eliminating 
most misidentifications and to enable a skilled scientific 
analyst to spend his or her time on those few cases genuinely 
worthy of full investigation.

This system was discussed by Jacques Vallee in his 
"Confrontations" (1990) at pages 212-213 (in the Appendix) of 
the Ballantine Books paperback edition. It is also discussed in 
the article by Spencer, Vallee and Verga highlighted above.

I've heard very little about this system in recent years. I 
understand that it was being developed further by a French 
group, but am not sure of its current status or availability.

D. Other software to assist in investigations and research.

This appears to be the primary focus of Terry Groff's "UFO 
Tools" website at:

http://www.terrygroff.com/ufotools/

Again, I won't attempt to list specific examples in this email 
(given its already considerable length), but will merely note 
some categories for which lists could be developed:

1. Software for checking specific IFOs, the most obvious example 
being astronomical sources;

2. Software relevant to particular types of evidence, e.g. 
Photographic evidence : image analysis software; Witness 
evidence: software/databases to assist in locating witnesses and 
calculation tools to assist in evaluating witness evidence. (In 
relation to calculation tools, in addition to noting the tools 
on Terry Groff's UFO website referred to above, I note that the 
article by Spencer, Vallee, and Verga highlighted above appears 
to briefly refer to other such calculation tools, including an 
Italian program called "Elaborazione Dati Avvistamento" 
("Sighting Data Processing"), which, at least according to that 
article, "allows the processing of many different parameters 
coming from the witness' tale. Probable sizes, altitude, 
distance and speed are some of the parameters you can 
obtain...".

3. Software for digitising information, e.g. Documents: Scanning 
software, OCR (Optical Character Recognition) software; Sound 
(e.g. lectures, radio interviews) : software such as Magix's 
Audio Cleaning Lab.

Also, it is important not to forget the full range of activities 
that may be encompassed by the term ufology, including political 
lobbying FOIA requests. There are various interactive tools 
online (and other software) that can be useful in relation to 
these areas. For example, there are websites that allow the user 
to send a fax elected representatives in a particular country, 
or to help generate the text of a FOIA request letter.

Furthermore, there are of course the fundamental software 
program (word processors, spreadsheets, databases, desktop 
publishing software, virus software, zipping software etc etc).

Perhaps the most obvious observations from reviewing the 
discussions referred to above are that many, many 
catalogues/databases have (a) been planned but not finished, or 
(b) finished but are not readily available. I dread to think how 
much time and effort has been wasted on such projects. I urge 
the various individuals on Updates that are involved in the 
development of further databases to:

(1) consider what, if anything, their project adds to existing
databases;

(2) adopt realistic goals; and

(3) consider how their project can be designed in stages or
modules,

so that others can build upon your work if you decide to
abandon it.

To help me (or anyone else) track down databases that have been 
developed but almost forgotten about, I repeat the invitation 
given above to let me (or Updates generally) know of references 
to lists of databases (other than those given in Section A of 
this email) so that a comprehensive list of databases can be 
generated and then followed up.


Kind Regards,

Isaac Koi


-- 
"Only a zit on the wart on the heinie of progress." Copyright 1992, Frank Rice


Terry W. Colvin, Sierra Vista, Arizona (USA) < fortean1 at mindspring.com >
     Alternate: < fortean1 at msn.com >
Home Page: < http://www.geocities.com/Area51/Stargate/8958/index.html >
Sites: * Fortean Times * Mystic's Haven * TLCB *
      U.S. Message Text Formatting (USMTF) Program
------------
Member: Thailand-Laos-Cambodia Brotherhood (TLCB) Mailing List
   TLCB Web Site: < http://www.tlc-brotherhood.org > [Southeast Asia
veterans, Allies, CIA/NSA, and "steenkeen" contractors are welcome.]