From: Steven M. Christey [coley@linus.mitre.org] Sent: Thursday, February 21, 2002 7:34 PM To: saag@mit.edu Subject: [saag] Re: Internet-Draft for "Responsible Disclosure Process" released The following messages were sent to me by Theo de Raadt of OpenBSD. I am forwarding them to SAAG with his permission. He is not on this list. The messages are separated by "==========" characters. My answers are indented and prefaced by [SMC]. ==================================================================== From: Theo de Raadt To: "Steven M. Christey" Subject: Re: Internet-Draft for "Responsible Disclosure Process" released In-reply-to: Your message of "Wed, 20 Feb 2002 00:55:01 EST." <200202200555.AAA13785@linus.mitre.org> Date: Wed, 20 Feb 2002 17:36:53 -0700 Good job. Actually, that is extreme sarcasm. You have created a document which will be used in self-defense by vendors who make bad products and do not plan or budget for repairs or maintainance, and will scare product quality research hobbiests into just not even bothering to disclose what they already know (well, publically anyways...) That is all this will change. It is lawyer bait. Therefore, this document will become part of the problem. [SMC] I hope that, by outlining a repeatable process that reporters can follow, *and* getting it accepted by the larger community, that reporters are *less* likely to be criticized because they find a bug and try to report it. They will have a community-accepted document that shows it. As you may know, some reporters are threatened with legal action when they report an issue. I speak as project leader for an operating system group that has 6 years experience at fixing security products on a wide variety of architectures in oh, normally about an hour, sometimes as much as a day. And we do this for fun and for free. [SMC] While I agree that some patches, OSes, and applictions take much longer to fix than one would like, there are issues such as regression testing, addressing multiple versions, etc. that are not necessarily feasible in such a rapid fashion. Many open source vendors can take longer than a day to fix a security issue. I can't speak for vendors (despite the first sentence of this paragraph), but it would be nice if vendors could explain *why* it can take a long time to fix issues. (It would also be nice to know how OpenBSD can do it while guaranteeing reliability, but that's off topic.) [SMC] Section 4.1, "Vendor Policy," encourages vendors to be forthcoming about how long it takes them to fix problems. Customers can then decide "how long is too long." If we can do it without such a document to help us out, why do you need to give the big corporations such help? [SMC] There are many types of vendors who do not properly handle vulnerability information. Almost 50% of all items in the CVE list (3000+ vulnerabilities) are not acknowledged by the vendor. I wish I could break it down further, but I don't have that data. This I-D addresses more than just the "many vendors don't fix bugs quickly" problem. I sense that there is a great denial against the idea that these vulnerabilities are in fact just quality assurance issues which noone wishes to take responsibility for in a fast, or should I say expensive, way. [SMC] The first bullet of the "Latent Flaws" section tries to suggest this, but some people want to remove it. ==================================================================== From: Theo de Raadt To: "Steven M. Christey" cc: deraadt@cvs.openbsd.org Subject: Re: Internet-Draft for "Responsible Disclosure Process" released Date: Wed, 20 Feb 2002 18:16:51 -0700 [EDITED OUT REQUEST TO FORWARD EMAIL TO SAAG] Reading closely, it seems that some people involved in writing this document wish to control the tide of alerts... so that discoveries are are manageable. That should be done at home, before things get started, but no vendor really wants to invest the time in doing that. (Don't act as if they do; they are not. Period.) Simply put, this document will rally the community against and VILIFY vulnerability disclosers who do not follow this exact process. On the other side, the process is massively heavy-weight. [SMC] Others have suggested the same thing. As a result, vulnerability discoverers will simply not alert. They will simply just continue to tell their friends. They will not even post anonymously. [SMC] Reporters who want to do "the right thing," *and* know how to do it, will tell the vendor regardless of whether there's a document or not. Reporters who want to do "the right thing" but *don't* know how to do it, will have a document that suggests what to do. But equally importantly, it tells *vendors* that not only should they *fix* reported vulnerabilities, they should actually make themselves *accessible* to people who want to report vulnerabilities, and they should make an effort to work with the reporter and resolve the problem as quickly as possible. The approach taken in this document is clever: to destroy the ability for a publisher to easily get fame and credit within his community, because it is outweighed by attacks from the greater community who use this document as a measuring stick. Trust me; very few will go through a month long process for fame and credit. [SMC] There are reporters who can't wait for a month because of motivations such as "getting fame" and being the first to discover something. But if someone reports a vulnerability without at least trying to work with the vendor, then they are putting all of the vendor's customers, and that is not responsible. See, in my view, fame and credit is a perfectly valid carrot to hold out to a person who has information that I can use to fix a bug. [SMC] Agreed. I think that they should get even *more* credit for trying to work with the vendor first. I want these things to be public as fast as possible. [SMC] The question of timing is a difficult one. There are many mixed opinions on this. OpenBSD's customers/users may want a vulnerability reported as soon as it is found, but there is at least one case in which a reporter reported an OpenBSD problem "as fast as possible" - and erroneously: BUGTRAQ:20001005 OpenBSD xlock exploit http://marc.theaimsgroup.com/?l=bugtraq&m=97076307710763&w=2 [SMC] You quickly pointed out that this vulnerability had already been fixed for months. But had the reporter tried the responsible route, I suspect that OpenBSD would have shown that this was a rediscovery, and this issue would not have been re-publicized, which may have caused some users to have to re-research an issue that was already fixed. If the reporter had found a new issue, OpenBSD users would have been at risk between the time the Bugtraq post was made and the time a fix was available and publicly announced. (Of course the risk would extend until the admin actually applied the patch, but that's not OpenBSD's responsibility once the issue is public.) [SMC] Obviously this is one example and a year old at that, but if guidance is available, then maybe more people will follow it. It looks like these might be other examples: BUGTRAQ:20010614 OpenBSD 2.9,2.8 local root compromise URL:http://marc.theaimsgroup.com/?l=bugtraq&m=99253282825806&w=2 BUGTRAQ:20010602 Locally exploitable races in OpenBSD VFS URL:http://marc.theaimsgroup.com/?l=bugtraq&m=99167357024081&w=2 [SMC] Actually, this latter post is an interesting case. I can't tell from the OpenBSD 2.8, 2.9, or current change logs whether this problem was real. I searched for "security" and "VFS". I fully recognize that I may be in error. [SMC] I sincerely do not mean to criticize OpenBSD's laudable goal of achieving security right out of the box. I was bringing up these examples to show the current state of affairs. This document will therefore, I believe, effectively cause vulerability information to go back underground, much more effectively than any previous attempts at no-disclosure have tried. [SMC] This concern has been raised in other places. If a reporter doesn't want to follow the suggested procedures, there is nothing preventing them from doing so, just like there is nothing preventing someone from ignoring the "common sense" disclosure practices today. [SMC] However, I agree that the document should be written in a way that does not allow vendors to from using it to threaten or prevent reporters from releasing vulnerability information... provided they do it responsibly. And back underground is not were we want this information to go, is it. [SMC] Absolutely not. So I feel this document is rather flawed, in that it outlines the rules which a reporter MUST follow (you use SHOULD, but the public will always read MUST, because they search to blame) [SMC] Some people have commented about the use of requirements language. I agree that people will read a "SHOULD" as a "must" instead. and the public will punish the reporter instead of the vendor. [SMC] That is more likely to happen today, since there is no community-accepted document that basically says, "reporters are doing a service by finding and reporting vulnerabilities." To me... it is quite apparent why such a mistake was made in this document. There are vendors involved. [SMC] They were involved, but only 4 of the 13 people who were consulted for this Internet-Draft were vendors. (That's excluding the 2 authors). See the Acknowledgements of the Internet-Draft. One of the vendors was an open source vendor. The acknowledgements include major mailing list moderators and well-known reporters, "full disclosure" and "anti-disclosure" advocates alike. [SMC] It may surprise you to know that Chris Wysopal and I had planned a second round of consultation by a wider group before publishing an Internet-Draft. This group included you. However, the IETF said that the document should be moved to public review as soon as possible, so we canceled this second consultation in support of the open IETF process. (We were not aware that private consultation for writing an Internet-Draft draft required talking to area directors first. I am grateful for the contributors that we had, otherwise this document would have been more controversial than it already is.) At a minimum, please re-read the document and tone it down, so that at no time a question arises of the reporter doing anything wrong. [SMC] Agreed, if you mean "the reporter is not doing anything wrong by letting the public know that an issue exists, after at least *trying* to work with the vendor to fix the issue." I truly appreciate your comments, Theo, despite our disagreements. Thanks, - Steve