Everhart, Glenn From: Tony Mason [Mason@osr.com] Sent: Wednesday, February 24, 1999 9:19 AM To: 'Jim Wilson'; 'ntfsd'; 'ntdev' Subject: RE: [ntdev] RE: [ntfsd] unexpected Modified Page Writer writes Jim, The MDL needs to indicate what "type" of operation is being performed precisely because the Memory Manager needs to know if the dirty bit in the PFN should be updated (or not) when the MDL is torn down (so long as the MDL pages are locked, it doesn't matter, because the pages won't be written back.) Thus, the MDL contains a bit to indicate the direction of the data transfer (MDL_WRITE_OPERATION indicating that the data is being modified.) The confusing part is that there is also a dirty bit in the PTE - and this will be set by the CPU (on x86, although I'd also note that not all systems where NT ran supported hardware related dirty bits in the PTE) when data is copied into that buffer (for example, via a PIO driver, but not for a DMA driver!) In the case of a read from the backing store that dirty bit needs to be discarded. In the case of a write TO the backing store that dirty bit needs to be flushed through to the page frame database (which is really all that matters in NT.) Thus, you must be building an asynchronous FSD request and specifying IRP_MJ_WRITE. One alternative would be to specify IRP_MJ_READ and then switching it (after creation) to IRP_MJ_WRITE (since they are remarkably similar operations) as this would change the direction of the probe in the MDL (try it yourself - call MmProbeAndLockPages and specify IoReadAccess versus IoWriteAccess (or IoModifyAccess) and then look at the MDL flags. For the read case it won't indicate the MDL_WRITE_OPERATION and for the others it will. Of course, that might cause some other side-effect. The best thing to do may be to just simply build the IRP (it isn't really very much code, probably 15 lines or so) and then you can build the MDL the way you want to do it. Regards, Tony Tony Mason Consulting Partner OSR Open Systems Resources, Inc. http://www.osr.com Specializing in consulting, development, and training for Windows NT systems software, notably file systems and device drivers. > -----Original Message----- > From: Jim Wilson [mailto:jmw@retrieveinc.com] > Sent: Tuesday, February 23, 1999 4:50 PM > To: 'ntfsd'; 'ntdev' > Subject: RE: [ntdev] RE: [ntfsd] unexpected Modified Page > Writer writes > > > Oops, I posted this to ntdev instead of ntfsd by mistake. > Lucky I did - I got 2 > useful responses. > > First of all I wanted to make sure the others following this > thread in ntfsd see this information. > > Second, I wanted to get some feedback about a possible solution. > > I noticed that the MdlFlags for the original MDL passed to my FSD were > 0x43 (MDL_IO_PAGE_READ | MDL_MAPPED_TO_SYSTEM_VA | MDL_PAGES_LOCKED). > The MdlFlags for the new Mdl that was created with the new IRP by > IoBuildAsynchronusFsdRequest() were 0x8a (MDL_WRITE_OPERATION | > MDL_ALLOCATED_FIXED_SIZE | MDL_PAGES_LOCKED). > > After trying a couple of things -- I love trial and error > engineering :^( --I determined > that if I simply cleared the MDL_WRITE_OPERATION flag in the new Mdl > before doing IoCallDriver() the problem of unnecessary > PagingIo writes went away. > > I don't have access to the original IRP or MDL at this point > (it's common > code that's used by filesystems on both NT and Solaris that > was written to not > have or need knowledge of the particular OS structures) and > getting access to either > would be a significant effort, and would also introduce some > ugliness to the common code. > > I know that MDLs are opaque and it's bad practice to change > anything in > them, but I'm wondering is it 'safe' to clear this flag with NT 4.0? > > Max, is the 'special way' used to create a PageRead MDL that > you mentioned > documented anywhere? It sounds like this would be the best way to fix > this problem. > > Thanks, > Jim > > -----Original Message----- > From: Maxim S. Shatskih [SMTP:maxim@storagecraft.com] > Sent: Tuesday, February 23, 1999 4:45 AM > To: Simon Graham; 'Jim Wilson'; ntdev@atria.com > Subject: Re: [ntdev] RE: [ntfsd] unexpected Modified > Page Writer writes > > MDLs for PageRead operation are built in very special way. > Not by IoAllocateMdl(), certainly. MDL with fixed-size tail > is allocated as part of the structure called "inpage support block" > which is allocated from the list. > The MDL tail is 15 ULONGs - maximum cluster size for clustered inpage > operations. > (BTW - inpage clustering is used only for mapped files, not > for pagefile. > Cluster size is depends > upon whether this is an image code section or data mapping). > > Max > > -----Original Message----- > From: Simon Graham > To: 'Jim Wilson' ; 'ntdev@atria.com' > > Date: вторник 23 февраля 1999 г. 11:11 > Subject: RE: [ntdev] RE: [ntfsd] unexpected Modified Page > Writer writes > > > >Actually, the problem is that the original MDL has flags set > indicating > that > >it is for a paging read op and therefore the memory manager > should NOT > >consider the pages dirty when they are unlocked even though > the hardware > PTE > >indicates they are. If you can't use the original MDL then use > >IoBuildPartialMdl to generate a new MDL from the old one (it > is OK to build > >a partial MDL that maps all of the original one) -- this > results in the new > >Mdl having the correct flags set. > > > >BTW: you can also simply set the Mdl field in the newIrp > equal to the MDL > >from the original Irp and leave NewIrp->UserBuffer unset. > > > >Simon Graham > >Mango > > > >-----Original Message----- > >From: Jim Wilson [mailto:jmw@retrieveinc.com] > >Sent: Monday, February 22, 1999 4:39 PM > >To: 'ntdev@atria.com' > >Subject: [ntdev] RE: [ntfsd] unexpected Modified Page Writer writes > > > > > >I am seeing a similar problem, a PagingIo write of a file's > pages after > >the second time I read it. My implementation is very much like Rob's > >sounds. My FSD also creates a completely new IRP/MDL to satisfy the > >original IRP. I tried setting my NewIrp->UserBuffer = > >MmGetMdlVirtualAddress(NewMdl) > >and the problem still occurs. It sounds like Tony's fix is > for the case of > >using a new MDL > >with an existing IRP, not for the case of using a completely > new IRP/MDL. > > > >Here are the steps I do for a typical PagingIo read: > >1) The thread that receives the PagingIo IRP_MJ_READ > typically queues it > for > >a > >worker thread. > >2) The worker thread gets the system address for the > supplied IRP's MDL > >using > >MmGetSystemAddressForMdl(). > >3) The worker thread then determines what it needs to do to > actually get > the > >read data. > >4) The worker thread then creates a new IRP using > >IoBuildAsynchronousFsdRequest() > >giving it the system address, or some offset beyond the > system address, > >determined in step 2. > >5) The worker thread then sets the completion routine using > >IoSetCompletionRoutine(). > >6) The worker thread then calls IoCallDriver() with the new IRP. > > > >I could change my code to use the original MDL, as Rob > suggested works for > >him. > >However, it would be much easier given the design of the > code to use a new > >IRP/MDL like I am doing currently. Is there any way to avoid these > >unnecessary > >PagingIo writes when using a completely new IRP/MDL in this manner? > > > >Thanks, > > > >Jim > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > [ To unsubscribe, send email to ntfsd-request@atria.com with body > UNSUBSCRIBE (the subject is ignored). ] > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - [ To unsubscribe, send email to ntdev-request@atria.com with body UNSUBSCRIBE (the subject is ignored). ]