From: IanPercival [IanPercival@email.msn.com] Sent: Friday, July 14, 2000 7:40 PM To: Info-VAX@Mvb.Saic.Com Subject: Re: I/O caching and UNIX evaluations (was: Re: got to remember ...) > But given how badly VMS file caching performance sucks compared to the rest > of the world, your statement above seems inexcusably arrogant - especially > in its implication (reinforced by your statements below) that other designs > necessarily entail losses in reliability. > > It seems reasonable to infer that either VMS engineering doesn't know in > real detail how to cache better (and maintain reliability), or knows but > doesn't consider it important (which in my experience suggests that their > knowledge may be at least somewhat superficial, since it's not perceived as > being an area worthy of real study), or knows but doesn't have the resources > to do anything (which is at least something one can sympathize with). And > even you seem to be aware that VMS engineering doesn't know how people want > to *use* caching to achieve differing trade-offs between up-to-the-second > persistence and performance, which likely implies that certain possibilities > have not received the analysis they may merit. The above is a highly misleading statement. As Steve Hoff says, it's hardly a revelation to many of the VMS engineers that variants of UNIX (and even may I say NT) have a different file data caching scheme. Writeback caching is always going to outperform writethrough caching as is the current VMS cache implementation FOR WRITES on low to medium volume. There is also a VERY strong presence in VMS Engineering of real world talent who have been at the front end of performance problems for many years (though of course there are some die hards who need to be educated still - as in any large organisation). We are working with some very large customers on some very interesting issues. The statement that we in VMS Engineering don't know how customers want to use caching is quite frankly,grossly incorrect (though of course thats open to interpretation - I'm sure that there are SOME individuals in Engineering, who have nothing to do with the file system, cache, or performance who don't know). It is true that VMS was remiss in ignoring the implementation of a more powerful data caching architecture in the past. This was for a variety of reasons and I don't want to open old wounds as that is fruitless. This has been gone over time and time again in the newsgroup and elsewhere. Some time ago in V5.x days I personally made a lot of money by co-writing, writing and selling a more powerful set of caching products (I was not working for Digital/Compaq - and yes - I was as frustrated as many that Digital would not enhance, merge or otherwise modify its existing product - find some old copies of Digital Review to see how semi-real independent testing showed true performance differences between caches). Some of the more vociferous members of this conference have NOT written any caching product anywhere. Nonetheless they are self styled experts. YES VMS currently has performance problems and YES we've been slow to react. YES we are now doing something and SORRY it cannot be INSTANTANEOUS!!!! We DO believe in testing stuff and making it as robust as we can before releasing it. Of necessity this means things happen a bit slower... Compaq has now committed to move forward on its data caching architecture. There will be a LOT of work done over the next few years - with ever increasing performance yields for different situations. XFC V1.0 is VERY competitive for read I/o caching - and that is BEFORE the performance pass that will be done on the product before final ship in December. Writeback caching is coming along very shortly afterwards, and it too will be highly competitive methinks - but of course I won't say until the code is frozen. Other things will be coming along too - which DO NOT EXIST IN UNIX/LINUX. Gosh - what will some of the naysayers do then? Lots I'm sure!!!! :-) The author of the top paragraphs is very aware of the fact that there is a new cache. Nonetheless it is easy to trash what is there currently. Repeatedly. I want to say that up to the present point, I have not heard ONE idea from any of the members of this conference on a new way to cache. The statement seems to be made over and over again that we should look at the Unix models as examples of how things should be done. Maybe some of you should chat to a maintainer or designer of a Unix cache. I have. Things are certainly not as golden as many would have us believe in terms of data reliability. Its incredibly arrogant of some people here to think that VMS engineers cannot figure out such concepts as writeback caching, temp files never hitting oxide, etc. Still, I understand the requirement to be visible too!!! We anticipate applying for several patents in the V2.0 XFC product, that in itself should indicate to some that VMS does not intend to follow, but to lead. Sorry for a slight flame on - but VMS really is actually moving on this now ( YES I know this isn't the total solution but its an indicator that we are,and have been doing things for quite some time, yes we've been slow to react - but we really will deliver something in the next few releases). Please lets move on - or at leat wait to trash XFC when it's finally released! These discussions will become far more relevant then! Ian Percival XFC Project Leader ian.percival@compaq.com Bill Todd wrote in message news:8knsbt$i4f$1@pyrite.mv.net... > > Hoff Hoffman wrote in message > news:8knpm4$4vp$1@mailint03.im.hou.compaq.com... > > > > In article <8knoba$ckh$1@pyrite.mv.net>, "Bill Todd" > writes: > > : > > :...Your previous statement that "One of the OpenVMS engineers has brought > up a > > :Linux Alpha box to see just what I/O caching scheme features are used > there" > > :suggests that you're looking for ideas, which is a good start.... > > > > Please see the earlier discussions around I/O performance comparisons. > > > > As for "looking for ideas", caching and cache implementations and cache > > designs are well-known among folks here in OpenVMS. > > Y'know, I'm inclined to give the folks in VMS engineering a great deal of > benefit of the doubt, considering how thoroughly they've been bludgeoned by > corporate politics and plain mis-management over the past decade-plus. > > But given how badly VMS file caching performance sucks compared to the rest > of the world, your statement above seems inexcusably arrogant - especially > in its implication (reinforced by your statements below) that other designs > necessarily entail losses in reliability. > > It seems reasonable to infer that either VMS engineering doesn't know in > real detail how to cache better (and maintain reliability), or knows but > doesn't consider it important (which in my experience suggests that their > knowledge may be at least somewhat superficial, since it's not perceived as > being an area worthy of real study), or knows but doesn't have the resources > to do anything (which is at least something one can sympathize with). And > even you seem to be aware that VMS engineering doesn't know how people want > to *use* caching to achieve differing trade-offs between up-to-the-second > persistence and performance, which likely implies that certain possibilities > have not received the analysis they may merit. > > The bottom line is that if you're not looking for ideas (including technical > ones), you should be. > > - bill > > Some of the key > > factors in the research involve the trade-offs between the performance > > gains and the corresponding risks to the data integrity, the relative > > differences in the I/O performance seen, complexity, resource > consumption, > > and various implementation-related factors. Very standard caching > stuff, > > in other words. > > > > What makes this whole task particularly interesting (to me and to some > > of the other engineers) is not so much the technical aspects, but rather > > involves the differing comfort levels that I expect individual customer > > sites will have with various current and various potential caching > > enhancements. Some customers undoubtedly want "extreme" caching and > > its (hopefully :-) associated performance improvements, and some will > > undoubtly prefer data integrity over performance. > > > > And as mentioned earlier, please see the earlier discussions around > > the various I/O performance comparisons folks have been pointing to. > > > > --------------------------- pure personal > opinion --------------------------- > > Hoff (Stephen) Hoffman OpenVMS Engineering > hoffman#xdelta.zko.dec.com > > > >