Everhart, Glenn From: Simon Graham [graham@mango.com] Sent: Tuesday, February 23, 1999 7:16 PM To: 'Jim Wilson'; 'ntfsd'; 'ntdev' Subject: RE: [ntdev] RE: [ntfsd] unexpected Modified Page Writer writes One word of warning with the fix you found; it is possible that, although you have disabled the paging writes, the pages are STILL marked as dirty (actually modified-nowrite) and will never be either cleaned or dropped; obviously a "bad thing" -- I had this problem with our file system for a while until I switched to using the original MDL (which I pass through the platform independent code as opaque data). To find out if you have this, there are a couple of things you can do (assuming you are using windbg); use "!pte
" to find the page table entry for one of the pages read, then take the physical page number for the page (shift the pte contents right 12 bits) and pass that to "!pfn pgnr" -- one of the pieces of info output is the specific list the page is on - if it says modnowrt you are toast! Simon Graham Mango -----Original Message----- From: Jim Wilson [mailto:jmw@retrieveinc.com] Sent: Tuesday, February 23, 1999 4:50 PM To: 'ntfsd'; 'ntdev' Subject: RE: [ntdev] RE: [ntfsd] unexpected Modified Page Writer writes Oops, I posted this to ntdev instead of ntfsd by mistake. Lucky I did - I got 2 useful responses. First of all I wanted to make sure the others following this thread in ntfsd see this information. Second, I wanted to get some feedback about a possible solution. I noticed that the MdlFlags for the original MDL passed to my FSD were 0x43 (MDL_IO_PAGE_READ | MDL_MAPPED_TO_SYSTEM_VA | MDL_PAGES_LOCKED). The MdlFlags for the new Mdl that was created with the new IRP by IoBuildAsynchronusFsdRequest() were 0x8a (MDL_WRITE_OPERATION | MDL_ALLOCATED_FIXED_SIZE | MDL_PAGES_LOCKED). After trying a couple of things -- I love trial and error engineering :^( --I determined that if I simply cleared the MDL_WRITE_OPERATION flag in the new Mdl before doing IoCallDriver() the problem of unnecessary PagingIo writes went away. I don't have access to the original IRP or MDL at this point (it's common code that's used by filesystems on both NT and Solaris that was written to not have or need knowledge of the particular OS structures) and getting access to either would be a significant effort, and would also introduce some ugliness to the common code. I know that MDLs are opaque and it's bad practice to change anything in them, but I'm wondering is it 'safe' to clear this flag with NT 4.0? Max, is the 'special way' used to create a PageRead MDL that you mentioned documented anywhere? It sounds like this would be the best way to fix this problem. Thanks, Jim -----Original Message----- From: Maxim S. Shatskih [SMTP:maxim@storagecraft.com] Sent: Tuesday, February 23, 1999 4:45 AM To: Simon Graham; 'Jim Wilson'; ntdev@atria.com Subject: Re: [ntdev] RE: [ntfsd] unexpected Modified Page Writer writes MDLs for PageRead operation are built in very special way. Not by IoAllocateMdl(), certainly. MDL with fixed-size tail is allocated as part of the structure called "inpage support block" which is allocated from the list. The MDL tail is 15 ULONGs - maximum cluster size for clustered inpage operations. (BTW - inpage clustering is used only for mapped files, not for pagefile. Cluster size is depends upon whether this is an image code section or data mapping). Max -----Original Message----- From: Simon Graham To: 'Jim Wilson' ; 'ntdev@atria.com' Date: вторник 23 февраля 1999 г. 11:11 Subject: RE: [ntdev] RE: [ntfsd] unexpected Modified Page Writer writes >Actually, the problem is that the original MDL has flags set indicating that >it is for a paging read op and therefore the memory manager should NOT >consider the pages dirty when they are unlocked even though the hardware PTE >indicates they are. If you can't use the original MDL then use >IoBuildPartialMdl to generate a new MDL from the old one (it is OK to build >a partial MDL that maps all of the original one) -- this results in the new >Mdl having the correct flags set. > >BTW: you can also simply set the Mdl field in the newIrp equal to the MDL >from the original Irp and leave NewIrp->UserBuffer unset. > >Simon Graham >Mango > >-----Original Message----- >From: Jim Wilson [mailto:jmw@retrieveinc.com] >Sent: Monday, February 22, 1999 4:39 PM >To: 'ntdev@atria.com' >Subject: [ntdev] RE: [ntfsd] unexpected Modified Page Writer writes > > >I am seeing a similar problem, a PagingIo write of a file's pages after >the second time I read it. My implementation is very much like Rob's >sounds. My FSD also creates a completely new IRP/MDL to satisfy the >original IRP. I tried setting my NewIrp->UserBuffer = >MmGetMdlVirtualAddress(NewMdl) >and the problem still occurs. It sounds like Tony's fix is for the case of >using a new MDL >with an existing IRP, not for the case of using a completely new IRP/MDL. > >Here are the steps I do for a typical PagingIo read: >1) The thread that receives the PagingIo IRP_MJ_READ typically queues it for >a >worker thread. >2) The worker thread gets the system address for the supplied IRP's MDL >using >MmGetSystemAddressForMdl(). >3) The worker thread then determines what it needs to do to actually get the >read data. >4) The worker thread then creates a new IRP using >IoBuildAsynchronousFsdRequest() >giving it the system address, or some offset beyond the system address, >determined in step 2. >5) The worker thread then sets the completion routine using >IoSetCompletionRoutine(). >6) The worker thread then calls IoCallDriver() with the new IRP. > >I could change my code to use the original MDL, as Rob suggested works for >him. >However, it would be much easier given the design of the code to use a new >IRP/MDL like I am doing currently. Is there any way to avoid these >unnecessary >PagingIo writes when using a completely new IRP/MDL in this manner? > >Thanks, > >Jim - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - [ To unsubscribe, send email to ntfsd-request@atria.com with body UNSUBSCRIBE (the subject is ignored). ] - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - [ To unsubscribe, send email to ntdev-request@atria.com with body UNSUBSCRIBE (the subject is ignored). ]