SIG TAPE HISTORY This document is intended to give my personal understanding of the history of the Sig Tapecopy idea. I began work on the SIG tapes in 1978, after it had already begun, but was told of what had happened in 1977 by Phil Cannon, Jim Neeland, and others. There are likely some inaccuracies in the following, but I believe they are not great. Back before Fall 1977, it was common at symposia for people to bring software on tapes (or DECtapes or paper tape) to trade with friends. This happened on the exhibit floor a lot, serving as a good continuing demo of DEC magtape products, so the DEC folks cooperated and provided machines. In Fall 1977, Phil Cannon had the idea of consolidating this and brought with him a collection the CARTS (Chicago Area Real Time Society) LUG had put together to act as a seed. Phil, I think Tony Scandora, Jim Neeland, and I think George Hamma, and maybe one or two others were involved in mastering. Phil put his collection into a box in the RSX campground asking for contributions to a master tape. The master was made by copying everything onto one tape, on Tuesday night (at that time DECUS symposia ended Thursday and Tuesday was the only free night). Then standalone PRESERV was used to make tape to tape copies of the master for everyone who brought a blank. Originally they expected the Fall '77 tape would be the last really long one, as others would just have updates and corrections. Boy, were they wrong. During the symposium, volunteers ran the tapes basically 24 hours a day for long periods to copy the material. The idea was that LUG librarians at the symposium would carry the tapes back and distribute them further. All formatting and documentation was done on the fly as the master was entered; that was why that many volunteers were there. As time went on, more contributors surfaced. I got involved in 1978 by volunteering to write index files, describing what was on the tapes. The idea of a README.1ST file in each directory was not at first universal, but was informally encouraged. By 1978, it was evident the SIG tapes (at this time, only the RSX SIG tape, though RT and RSTS quickly followed suit; I don't know too much in detail about their history) were worrying the Library Committee. At that time Phil was the RSX representative to the committee, and RSX was THE big SIG. The library at that time was very slow to turn orders around, and catalog delays were longer than now, and general members might not even get the PDP11 catalog. (There was a PDP10/20 catalog, a PDP8 catalog, and a PDP11 catalog. I forget where PDP9/15 stuff went.) One of the attractions of the sigtape mechanism was that it gave instant distribution and it worked so much better than the older informal exchanges that it was an instant success. The Spring '78 tape was full and may even have been two reels (in FLX/DOS-11 format then at 800 BPI). I don't recall exact sizes, though I have the tapes, having gotten them from friends. The USPLC viewed this as a threat to the library; Phil explained that "the SIG tapes have become the library; the rest is small by comparison". The Library Committee tried on at least two occasions to kill the project. Phil's response was that if they did, he'd take it underground and it would continue with or without Library acquiescence. There was no doubt of his ability to do so, or of the support he'd get from general membership. The early tapes were done by "handshake" methods of operation, and there was a problem with someone submitting an old version of some of Versaplot onto one of the tapes in the Fall '77 to Spring '79 period. It had slipped by; this was more likely then in that all composition was done Tuesday night, frequently into the dawn Wednesday. Also at this time it was becoming clear that people were giving up too much session time and/or sleep to make all the copies on the floor, and that demand was outstripping supply. People left blank tapes in boxes in the campgrounds which were turned into sigtapes on the fly. So Phil began getting lists together of LUG librarians to act as distribution points later. All this centered on the campground. At that time there wasn't a Library booth. By Spring 1979, the new VAX SIG put together a tape using basically the same method as the RSX SIG did. VAXen were on the floor from Fall 1978 (the RSX tape then was made on the 780; I was there) and could be used. It was clear that the Library had to have a way to get the sigtape material added. Finally in Fall 1979, Phil had a distribution network going (and there was another in the VAX SIG; Dave Schmidt was one of the early honchos of that effort) to distribute the tapes. It was announced that people would have to sign Library release forms for their submissions to the tapes (the versaplot hullaballoo having been done a year before and having led to increasing questions to submitters). Also, it was announced that tapes would no longer be made in bulk at the show. The load on volunteer copiers had grown too great. Also the Library had specials for a couple or three symposia: you paid $20 or so and were able to get a copy of the sig tape for your very own, produced by and sent out by the library. I ordered these as well as getting my own copies from the tree. The early Library produced tapes caused lots of trouble. The first one had read errors at the beginning of tape in EVERY tape when read on DEC drives. Later ones had invalid FLX format (18 byte instead of 14 byte headers) which made the tape unusable for some systems. This was for the RSX tapes and the Structured Language tapes. The Structured Languages SIG made several tapes, the last I think in 1981. They hit a serious problem in that the only universal tape format they could use was DOS format. This was inefficient (512 byte blocks) and FLX tended to write files as ASCII unless it knew they were binary types. It recognized this by extension. We had to be VERY careful reading tapes in and writing them out when any "funny extension" binary files were there and still messed some up. SL Sig's tape wound up getting more material than they could handle and made some unpopular editing choices. Up to and including 1980, the RSX tapes were distributed at 800 BPI. The VAX tapes went out at 1600 due to DEC's universal support of that density, though the early VAX tapes were made with RMS Backup, the less said about which, the better. (It was buggy from day 1 and STILL has many of the same bugs.) The VAX SIG went through a couple tapecopy coordinators before Joe Bingham. John Thompson was it in Spring 1979. Alan Goodhue had the job in Fall 1979. Then Roger Lipsett took over in Spring 1980. Joe Bingham did the job from Fall 1981 until I took over (around 1987 was it? How quickly the time flies.) I had been doing the bulk of the assembly since Spring 1985 acting as staff to Joe, and tried to cross migrate some ideas between the RSX and the VAX tapes for format. The RSX SIG had Phil Cannon as tapecopy coordinator from 1977 to 1980. Jim Neeland took over when Phil went to work for a company making non-DEC computer based products, and was coordinator until around 1984 after which I took over. I haven't kept track of the other SIGs well. Nick Bourgeois was the RT11 tapecopy coordinator for a long time, though. He also had the library committee tapecopy coordinator slot a long time, but the RT Sig never got into building their tape on the floor. The VAX SIG for a long time did all tapecopy work on the floor and Joe Bingham made one copy and massaged it. He also ran the tree for the VAX SIG, which he had well automated. I recall the printouts left in the campground trying to validate the tree. The RSX tree was more varied and Phil and Jim tried to phone each node every time to validate it. This was the greatest single work item for the tapecopy coordinator. The usual drill for tapecopy was that we'd take the tapes and generally go off-site (since by 1979 it was getting hard to find PDP11's on the floor that had tapes and disks we could use) to copy them. The tapecopy coordinator arranged this. We'd read the tapes in, in whatever format they came in. This meant we had to be able to read BRU, FLX/DOS, RMS Backup, ANSI, RT11 flavor ANSI, and some oddball others. We brought tools tapes with us each time to ensure we had FFL (Fast FLX reader) along. As each tape was read in, the directory number was changed to fit into the historical RSX scheme. This meant that the UICs of the material were consistent across time (the SUBMIT.RNO file in [300,1] of any RSX tape has the index) but were made so at initial read-in. We HAD to do this to avoid collisions, and to give us the cross reference back to submitter. During the reading process, we eventually ensured a README.something existed in the directories and made a consolidated README file to be left in the campground after all was read in. Whoever could contribute descriptions did so. Jim Neeland started writing the abstract in the form RSX81BTPE.DOC (for example) afterwards. When I started working on the VAX tape, I carried that over; it seemed a good idea, as the consolidated README files were bulky. Joe Bingham had automated the consolidated readme generation; the RSX group did it with a directory, then a TECO macro, on the fly each time, with human edits. Joe Bingham had the idea of an enforced AAAREADME.TXT file in each directory which his consolidated readme keyed off of. The RSX requested standard had been "README.1ST". I've started lately using the AAAREADME.TXT method and Joe's command procedure. The reason for the README.1ST name was that early on we used FLX for copying, which only allows selection on the first 6 characters of filename (a DOS-11 file has 6.3 characters, not 9.3 like RSX and VMS prior to V4.0). Therefore there was some reluctance to use the extra 3, even after DEC modified FLX to store them in a previously unused header word on the tape. About 1981, the RSX tape got too big to fit on two reels in DOS format at 800 BPI. We took a survey and found that most sites had BRU (which used 2048 byte blocks) and 1600 BPI, so we started producing the tapes in that format. Some tree nodes were identified to convert to 800 BPI for those sites that couldn't read 1600 (these were generally leaf nodes) and the main distribution grew eventually to fill the whole reel at 1600. We kept it there by fairly ruthless pruning, and were at times much more ruthless with .TSK files (RSX equivalent of .EXE) than the VAX tapes have been. There were some complaints, but people generally understood the reasons for the change and supported it. The tapecopy committee decided this and just presented it. I believe this happened at the Miami meeting. We were at Angel Li's site then for the copy and he had the first 6250 BPI tape on a PDP11 I had ever seen. We copied almost all of the RSX tape onto a 600' reel at 6250 and then when Phil got up and announced "here's the RSX tape; to save postage we're mailing it out at 6250 on these small reels" the 1600 BPI announcement was a welcome relief. 6250 was REALLY rare then. It had made a good joke, though. The trees continued to be the only distribution mechanism for some time. Periodically the Library would have an offering to distribute the tapes, but never could come up with a viable plan and after their format screwups of the early tapes we were chary of this. Also, we had lots of trouble getting LUG LIBRARIANS rather than LUG chairs on our lists and keeping them there. Many complaints came in to Jim, Joe, and me over time about people who were on the tree who were LUG chairs but didn't have VAX or PDP11 systems to do their copying and would say they'd farm that out. Trouble was, these operations introduced weeks of delay. We gradually found out who was good and who wasn't, but on a national scale parts of the tree broke constantly. They still do. Some parts, however, ran smoothly, and we patched up problems as they occurred. The trees were always of roughly the current size. When NLC came out to volunteer to help, Joe Bingham and I told them of the difficulties we had known. I believe Jim Neeland had been at that meeting too. The feeling I had going out was that there was an offer to provide another path to people to get the tapes. This I was most happy with. Somehow it got transmuted into a replacement for the tree as seen by the VAX and RSX steering committees. As a result, Joe stopped officially using the VAX tree, which had been a very smooth operation. I argued that problems with a relatively untried operation would put our consumers at risk and convinced the RSX steering committee that the tree should be continued as a parallel operation. I maintained the list and mailed out letters to each person on it with a complete tree list when the tapes became available, but gave the NLO contacts also. Eventually we started that back up with the VAX Sig also, since Joe got a lot of complaints when he broke the VAX tree. It appears that much of the distribution at that time went totally underground. Since then I've spawned off Dar Schumann as the keeper of the tree. His job is to keep the list validated by getting people to verify their existence by various means. It was getting to be too much for me. (Considering the time I've put into the S90 tape, giving the validation job away looks like my best single move!) He has been maintaining the list and usually getting it out in timely fashion. (His performance has been as good as anyone's, better than some predecessors.) This distribution mechanism serves a considerable number of sites, but their number is unknown. The L&T Sig, when it started, decided in 1986 or so to put out a SIG tape too. At this time, VMS Backup provides a reasonable tape format for DECUS audiences. That may change. Originally, the L&T Sig tape was supposed to be a constant list of collections that would be updated, so that every L&T tape would replace the previous one instead of accreting as the RSX, RT, RSTS, VMS, and Structured Languages tapes had. This broke down almost instantly. Owing to lots of duplication in the VAX and L&T tapes (caused by my feeling that a tapecopy coordinator should beat the bushes for material and put out the most comprehensive freely distributable collection possible and by significant overlap in charter), we decided in 1989 to combine the two into a single larger collection, which might again be broken out for library distribution, but which would let us do a better job. An additional advantage of this is that it gives two tapecopy coordinators who are familiar with the drill, in case one cannot get the job done a particular semester. It was the RSX practice, which I introduced into the VAX tape process, to have each worker take a copy of the material back. This provides redundancy which has on a few occasions been important; we have had times when the tapecopy coordinator had to call one of the others to get a readable tape. It also gives a chance for some pre-exposure to tape materials, so that software with problems (copyright problems, or misbehavior of some sort) can be identified before the general release. By and large, the preliminary tapes have not been released in any general forum. They may be released, but should be limited, and those who get them are to be told of risks and that their tapes are not final. The major danger to the process has been that we don't want multiple versions of any SIG tape to get into circulation; it causes confusion and problems with answering questions about it. Recently, issues of covert behavior have arisen. Neither Joe Bingham nor I have ever seen any examples of this other than one Reminder program that had a time-out some years back. This was verified by Joe to have been deliberate. The response was to instantly pull the affected program wherever we could, warn everyone we could, and I reset the time-out to 999 years later and put the reset version on the next VAX SIG tape with a full explanation of what had happened. This is our precedent, though I expect that the metanetwork (usenet, internet, bitnet, etc.) would aid in passing word on today. Our stated policy is to follow this kind of procedure whenever reports exist of covert behavior once we find it really exists. The intended policy for reports we can't confirm is to put notice of the problems in where they cannot be ignored and do our best to ensure that anyone wanting to ignore the reports will be fully aware what (s)he is doing. This kind of thing is a case-by-case issue and depends somewhat on the source and nature of such reports. Since these have not yet appeared, we'll settle it if it occurs. There was also a meeting between Joe Bingham and me around 1984 to settle our attitude toward shareware on the sig tapes. I was initially willing to accept it, where it was useful to people who got the tapes, as part of making a more useful tape. Joe was uncomfortable with it. His argument which convinced me he was right was that a shareware product tends not to be accompanied by source code. This means that by allowing shareware, we discourage source code on the tapes and make them ultimately less useful to those who get them. (The security issues were not so strong then, but they militate in the same direction.) We agreed that it should be disallowed, at least for VAX or PDP11 software, on the RSX or VAX tapes, with possible exceptions if sources were there where the tapecopy coordinators felt the material was of interest. Through the history of the tapes, the bias has been very strongly NOT to censor anything except for legal reasons (can't distribute someone else's code if it's copyrighted, licensed, etc.). The space limit issue has been hit a few times in the RSX SIG tape, and is beginning to hit in the VAX tapes. My innovation of compressing material has given a partial out for the VAX tapes, and the move to 6250 BPI distribution which is de facto going on also has given some relief. When the entire Kermit distribution was submitted to the VAX, RSX, RSTS, and RT tapes, Joe and I made a separate tape of it as one distribution down the trees. There has been one example (around 1980) when material in the RSX tape was delayed 6 months by Phil Cannon for size reasons also. Given our current trends, this doesn't appear likely to work well again any time soon. The method of splitting off single giant collections may offer some relief in the face of growing amounts of material. Since aggregate size has tended to be dominated by when some large packages are submitted, we may not have to use this method. It may also happen that VMS will wind down as RSX has, and some other system will come to dominate. My belief is that should VMS become less important (and I think it will), there will still be L&T submissions and the aggregate volume will probably continue to increase. The VAX/L&T tapecopy people need to be thinking about other tape formats now (I am) so that in the possible near term it might be feasible to offer the collection in, say, TAR format for those interested. I also believe that in the next few years, 4mm or 8mm tape will become widespread enough that it (or possibly those) will be able to take over from reel to reel tape as the distribution backbone medium rather than 6250 BPI tape. This will provide a very significant space increment, and allow us to abandon compression for some time to come. Over the longer term, I believe the sig tapes are the most enduringly valuable item DECUS has to offer. The fact that the material is indexed and in a somewhat standardized form makes it far simpler to locate desired items in it than it is using FTP type file copies from an Internet archive. The volume of the material is such that it would be inconvenient to assemble from FTP archives with all but the fastest net hookups, and the bulk of the distribution effort is volunteer and distributed, minimizing costs. Some of the costs (calling your colleagues) even serve to promote collegiality and information sharing in themselves, and thus provide benefits to offset them. The availability of those names and phone numbers on the tree list is one of the main reasons I fought to keep the trees, and to put the lists on each tape. I also believe that had the tapecopy idea not been started, the DECUS library would be much smaller than it is, and of almost no consequence. People will often contribute to a software sharing channel whose primary distribution is a free one, where they would not willingly do so for a library that charges what the DECUS library must. Therefore, though there has been a tension between the library and the tapecopy process due to cost differences, the library has gained in mass, and remained above critical mass for such a thing, largely because of the input as a result of the tapecopy efforts. One can track the size of the DECUS library alongside that of the SIG tapes; they match reasonably well. The availability of a documented place to obtain older tapes has been in turn valuable to the tapecopy project, since most volunteers don't have room to keep older SIG tapes around online for new LUGs or sites to get. Together they accomplish a task of making our information heritage available which neither alone would be able to do. A personal note: a tapecopy coordinator must really believe in sharing software and truly believe that knowledge and mankind as a whole are best served by sharing it. Perhaps a scientific background aids this (mine is High Energy Physics). He must be willing and able to put significant time into indexing each tape. This last time has so far been a month and a half of fairly constant work. Not all of them are so time consuming; it can take a week if things are smooth, and a really hasty job can be put together in maybe 3 days' work if everything fits and people have supplied good documentation. But the willingness to work hard on the tapes must be there, and the time to do it. Also historically the willingness to fight for the tape copy has been called upon a number of times. Finally, ability to access the networks is becoming more important as a way to gain submissions, as well as to notify the world when the distribution begins. Tapecopy is organizationally an oddball activity; formally it can be viewed as working groups within SIGs. In fact, it is driven by a group of volunteers who believe in it and whose activities interface very minimally with other DECUS groups. By this I mean that there is nothing like an "inter-SIG tapecopy committee", the activity consumes and generates little or no direct DECUS money (though the tapes do furnish a large part of the Library's orders, and the Library serves as a repository of the tapes). The budgetary independence is almost total. Master tapes are created by volunteers (it's a skilled job requiring one to know a bit about RMS attributes that is not that commonly known), distributed by volunteers exchanging tapes and paying postage themselves, and alerting one another by various means. The only DECUS money that goes into the process is the cost of mailing the trees out (150-200 copies or so, twice a year) and paying symposium registration for a library tapecopy coordinator who stays in the library booth to collect submissions and forms. This latter is required for getting the software into the library, so is properly a library expense also; some SIGs just have submissions mailed to their SIG tapecopy coordinator. (The VAX, L&T, and RSX Sigs accept these also but they should arrive by the time of the symposium to be included. Late arrivals might make it but are discouraged.) Tapecopy is in many ways even more free-standing than the Library itself (most of the rest of DECUS' activities being tied in one way or another to the symposia). Its' closest analogue elsewhere in DECUS is the symposium committee, in the sense that both groups are composed of workers and neither is very concerned about management, DECUS politics, funding, and the like. (I had the pleasure of meeting Emily Kitchen and discovering a kindred soul not too long back.) It is probably fair to say that tapecopy is a relic of an earlier DECUS organization and to ask whether such a thing could be started today. I suspect it would be much harder to do, personally. Therefore I expect it to be unique and find it so, and would very much regret its passing. I believe that continuity of SIG tape copy is therefore quite important, and that this underscores the importance of making VERY sure that the tapecopy coordinator job never passes to someone who can't do the work. This is probably enough history and philosophy for one go. THE MECHANICS OF MAKING A VAX SIG TAPE This is a VERY thumbnail sketch of how the VAX tapes are put together. The RSX tapes are harder due to need to ensure totally unambiguous UICs. At the symposium: 1. Copy all tapes into a master directory. Generally this means using Backup to put each submission in its' own directory tree below the top. Manual adjustment if a tree gets too deep for VMS. 2. Make an index of submission form material giving contributor, submission number, and description. Use for Tapecopy Forum handout. 3. Make copies of the preliminary material and carry home. At home: 1. Pull everything onto disk. If problems arise call someone else with the tape and get a good copy. 2. Rename directories so that empty subdirectories are removed and top level names make sense. Build a cross reference of submission numbers working from the Tapecopy Forum handout so that the new directory names can be tracked back. This gives "final" directory naming. 3. Ensure each directory tree has exactly one AAAREADME.TXT file that is descriptive of the submission and short enough to include in the grand summary. Create if not there, edit if too long. (Save original in another name.) Any non-distributable software gets pulled if found; do so here or anywhere else it is noted. 4. Look over size of collection. If too large, compress what you can. Anything goes here, but it MUST be clear how to regenerate the original material. 5. Compress all *.*LB files with LIBR (libr/compress/data=reduce). 6. Remove extra .LIS, .MAP, etc. files after checking that they are indeed extra. EACH must be checked. 7. Run BUILDREAD to make an AAAREADME.YYA (or .YYB) file from the AAAREADME.TXT files. 8. Print out the AAAREADME file. 9. Using the information from the AAAREADME file, and the tapecopy forum handout file, write an abstract for the tape. This is done by subdirectory so that each directory tree has some description (as brief as possible) of what is there. 10. Look at overall sizes and split the tape up into directory trees containing not over 60,000 blocks (limit of one tape at 1600 BPI) or whatever limit is currently chosen. The limit for TK50 is about 160,000 blocks, and a magtape at 6250 BPI will hold not over about 290,000 blocks in VMS Backup format. 11. Compress more material if the whole is too large; back to step 10. Iterate back further if late submissions arrive that are thought appropriate to include in the current tape. 12. Update text files in index subdirectory for the current time. 13. Send the tape to the DECUS library, the head of the NLC tree, and the heads of the VAX tree plus other contacts as arranged. The Tree manager (currently Dar Schumann) and the Other Media Working Group head (currently Bob Uleski) get tapes. OMWG is a center for getting the tapes transcribed onto oddball media and has a subordinate tree. 14. Publicize the release of the tape by transmitting the abstract wherever possible. DCS and Info-VAX are a good start. 15. Sigh a sigh of relief, and start collecting software for next time. Glenn Everhart 13 June 1990 VAX SIG Tapecopy Coordinator RSX SIG Tapecopy Coordinator