Fw: Re: About my driver, HIMEM/EMM386 and interrupts

[Lucas Kiwi]

Guys,

   I was pretty sure I had got it working before, so I didn't want to just leave it as it is. What I just did is restart my computer with plain FreeDOS and wrote, compiled and executed a programme like this:

   mov ax,0b800h
   mov ds,ax
   mov word ptr [100h],1f21h    ; draw exclamation mark

   mov ax,[12345678h]  ; tempt FreeDOS to respond to an exception

   mov word ptr [102h],4e3fh   ; draw a question mark
   mov ax,4c00h
   int 21h   ; exit to DOS


   What happened was that both marks where drawn and no error occurred. At that point, only HIMEM was loaded, no EMM386. I thought that meant that FreeDOS's HIMEM did indeed have that routine.
   I rebooted the computer with EMM386 and reran the program. Of course, this time, an exception was triggered. Only the exclamation mark was drawn and the program exited abnormally. This was quite sensible to me. I tried one more time not loading any of the two drivers. The first mark was drawn again, but the system halted, unable to handle the exception.
   Then I decided I'd be more drastic, so I wrote another program like this:

   xor ax,ax
   mov ds,ax
   mov word ptr [0b8100h],1f21h   ; draw exclamation mark
   mov ax,4c00h
   int 21h  ; exit to DOS

   When loading only HIMEM, this did, in fact, cause a mark to be drawn on the screen. But then I realised it could also be the fact that I was using 4DOS as the command interpreter. So I rebooted with HIMEM and FreeCOM and the system hung when I tried to execute the code.

   Conclusion: it was 4DOS that was loading this routine and not HIMEM, but as I had read at some point about HIMEM.SYS doing it, I had thought that was it in this case. Anyway, this leads me to think it'd be very handy to have a function or a small application in DOS to enable this behaviour on programmes requests. Of course, relying on this would render code incomaptible with EMM386 and emulators such as DOSBox. Am I right?  On the other hand, it would be a lot faster to access memory.

                       Lucas

PS: Because of a problem with yahoo, you may have received a truncated copy of this e-mail. Please disregard it. I think I'll switch to GMail



--- On Tue, 27/10/09, japhethx gmail <japhethx@...> wrote:

From: japhethx gmail <japhethx@...>
Subject: Re: [Freedos-devel] About my driver, HIMEM/EMM386 and interrupts
To: freedos-devel@...
Received: Tuesday, 27 October, 2009, 4:56 PM

>> However, I understood that HIMEM did something like establishing a 

>> routine that was run whenever you tried to use an offset greater than 
>> 0FFFFh together with a "real-mode-made" segment. This routine would 
>> "trap" the event and handle the switch to protected mode and back, 
>> keeping the segment base, but extending the limit to 4G, then returning 
>> to the instruction that generated the exception. Am I right?  Does this 
>> really happen?  If so, programs should not mind about their segments 
>> being initially setup in real mode.
>
> No, I'm not aware of any Extended Memory Manager (XMM) that does such a 
> thing.

MS Himem does indeed "install" such a routine, but it is installed only during
XMS block moves inside Himem.

See the MS Himem source for details. It's Public Domain, IIRC.


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Freedos-devel mailing list
Freedos-devel@...
https://lists.sourceforge.net/lists/listinfo/freedos-devel

Reading this email at work? Make a change with Yahoo!Xtra Jobs

Email slow, clunky, unreliable? Switch to Yahoo!Xtra Mail, New Zealand's new email address.
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Freedos-devel mailing list
Freedos-devel@...
https://lists.sourceforge.net/lists/listinfo/freedos-devel

[dos386]

> I'm building a driver (for DOS)

:-) For what hardware ??

> means that the driver will have to do different things to get to the tables depending
> on whether HIMEM, or EMM386, or both, or none of them are currently there.

NO. Just use XMS, it is still available even if EMM386 is additionally
present. Make a decision whether you want to support XMS 2 hosts also
or require XMS 3.0.
If no XMS host is present, 3 options:
- Refuse to run
- Hog the memory from BIOS (see FASM source) and use "raw" PM
- Use low memory only it it fits

> Also, if HIMEM only is there, this mode is already available and I can easily access

NO. It "can" happen but you can't rely on it.

> If EMM386 is loaded, I know I can use the expanded memory functions

YES (also after NOEMS ???), but you still can and should use XMS.

> but then virtual-86 mode is set and I can no loger use 32bit real-mode pointers!

IF you need PM from V86 there is VCPI.

> Yet, there has to be a way, because HIMEM works even though EMM386 is loaded.

As said, it uses INT $15 / AH=$87 then. That's also what you can use
to access
PCI memory mapped I/O (>= 2 GiB) from real mode code, even if EMM386 is there.

BTW, RBIL is wrong here, it says "linear address" but the call in fact eats
physical addresses, at least those above 1+1/16 MiB AKA $0011'0000.




--
~~~ wow ~~~

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Freedos-devel mailing list
Freedos-devel@...
https://lists.sourceforge.net/lists/listinfo/freedos-devel

[Christian Masloch]

>> but then virtual-86 mode is set and I can no loger use 32bit real-mode  
>> pointers!
>
> IF you need PM from V86 there is VCPI.

Protected mode isn't the same as flat real mode.

>> Yet, there has to be a way, because HIMEM works even though EMM386 is  
>> loaded.
>
> As said, it uses INT $15 / AH=$87 then. That's also what you can use
> to access
> PCI memory mapped I/O (>= 2 GiB) from real mode code, even if EMM386 is  
> there.
>
> BTW, RBIL is wrong here, it says "linear address" but the call in fact  
> eats
> physical addresses, at least those above 1+1/16 MiB AKA $0011'0000.

Yes, you're right. All areas above the 1088 KiB (- 16 byte) accessible in  
real mode access the physical memory. Since the XMM works by switching to  
protected/flat real mode (without remapping memory) or using Int15.87  
provided by EMM386, the linear address returned when locking a XMS memory  
block should be usable as physical address as well.

Regards,
Christian

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Freedos-devel mailing list
Freedos-devel@...
https://lists.sourceforge.net/lists/listinfo/freedos-devel

[Lucas Kiwi]

Guys,

   Yeah, it clearly will be the best for me to use XMS functions. There is still a lot I don't know about the different ways to access PM memory (including protected mode itself) so some things you guys say are perfectly clear to me while others are full of mistery :P   Is there any site where I can read in very good detail and in a way that is easy to understand, how flat-real mode, VCPI and DPMI work and how they deal with other previously loaded systems to avoid collisions?
   I really want to learn all that and whenever I find info on that it's rather cryptic. If one or more of you are willing to give me some "lessons" over Skype or something, I am willing to be very cooperative :P ha, ha. I really want to learn this thing.
   DOS386 mentioned "if you want PM, you have VCPI", I think, meaning if I wanted to access PM memory. If that is the case, that line is indeed very significative to me!  Only I wish I knew more about VCPI. Because of the nature of my driver, the memory access does not have to be very messy and it will probably be enough to just rely on XMS functions for now.
   One doubt I have about XMS functions is this: the XMM specification states that if I lock some memory, I must unlock it as soon as possible. How soon is that?  What exactly is a lock count?  I may, for many things, not need to know where my memory block is located, but sometimes, when I do need it, it would be important to keep it locked for as long as the driver is running, because it is a driver and other programs would be relying on it. Can I keep memory locked like for the whole time one process is running (i.e.: half an hour)?  How bad is that?
   About what hardware the driver is for... well, it's not for just one piece of hardware. What I am working on is a spec to link different APIs for different pieces of hardware and software. The idea is that all the elements can be accessed together in the same way and at the times they are really needed. This would keep as much free conventional memory as possible at all times, allowing dynamic modules and also providing a connection with a call-point and parameters that could be projected on any software/hardware platform. For example, we could use an interrupt number, say, 49h and registers EAX trhough EDX... but we could also use a PM pointer to code memory to be called with the parameters pushed into stack, etc.... and the spec would still hold. The result is that any operating system and any computer would be able to use the same structure (if they were interested) and OSs who participate in it will be able to help one another instead of working separatedly. It goes further, but I would need to write a whole document to explain my exact idea, so I want some code running, before I present it neatly. What do you reckon?

                  Lucas



--- On Fri, 30/10/09, Christian Masloch <cm@...> wrote:

From: Christian Masloch <cm@...>
Subject: Re: [Freedos-devel] Fw: Re: About my driver, HIMEM/EMM386 and interrupts
To: freedos-devel@...
Received: Friday, 30 October, 2009, 1:16 PM

>> but then virtual-86 mode is set and I can no loger use 32bit real-mode 
>> pointers!
>
> IF you need PM from V86 there is VCPI.

Protected mode isn't the same as flat real mode.

>> Yet, there has to be a way, because HIMEM works even though EMM386 is 
>> loaded.
>
> As said, it uses INT $15 / AH=$87 then. That's also what you can use
> to access
> PCI memory mapped I/O (>= 2 GiB) from real mode code, even if EMM386 is 
> there.
>
> BTW, RBIL is wrong here, it says "linear address" but the call in fact 
> eats
> physical addresses, at least those above 1+1/16 MiB AKA $0011'0000.

Yes, you're right. All areas above the 1088 KiB (- 16 byte) accessible in 
real mode access the physical memory. Since the XMM works by switching to 
protected/flat real mode (without remapping memory) or using Int15.87 
provided by EMM386, the linear address returned when locking a XMS memory 
block should be usable as physical address as well.

Regards,
Christian

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Freedos-devel mailing list
Freedos-devel@...
https://lists.sourceforge.net/lists/listinfo/freedos-devel

Reading this email at work? Make a change with Yahoo!Xtra Jobs
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Freedos-devel mailing list
Freedos-devel@...
https://lists.sourceforge.net/lists/listinfo/freedos-devel

[Christian Masloch]

> Is there any site where I can read in very good detail and in a way that  
> is easy to understand, how flat-real mode, VCPI and DPMI work and how  
> they deal with other previously loaded systems to avoid collisions?

I don't know about such a site. RBIL however provides much information  
about the VCPI, DPMI and XMS interface each. If you can read Assembly, you  
can also look at the source code of HIMEMX, JEMM and HDPMI.

>    DOS386 mentioned "if you want PM, you have VCPI", I think, meaning if  
> I wanted to access PM memory. If that is the case, that line is indeed  
> very significative to me!  Only I wish I knew more about VCPI.

Read the RBIL description of the Int67.DE functions.

>    One doubt I have about XMS functions is this: the XMM specification  
> states that if I lock some memory, I must unlock it as soon as possible.  
> How soon is that?

I think it doesn't have to happen at all. You don't even have to unlock  
the block before deallocating it (when your driver is removed from  
memory). Someone else might know about that better.

> What exactly is a lock count?

The lock count is incremented each time you lock that memory block. Each  
time you call XMS to unlock the block, it's lock count is decremented.  
Only if it reaches zero, it's actually unlocked. This way the owner may  
lock the block multiple times but it really is unlocked only after  
unlocking it as often.

In case you thought about it as some kind of timer: no, it isn't.

> I may, for many things, not need to know where my memory block is  
> located, but sometimes, when I do need it, it would be important to keep  
> it locked for as long as the driver is running, because it is a driver  
> and other programs would be relying on it.

You could require these programs to register as soon as they want to use  
the driver, and unregister when they're done using it. (You may want to  
lock XMS memory blocks the whole time anyway though.)

> Can I keep memory locked like for the whole time one process is running  
> (i.e.: half an hour)?  How bad is that?

As mentioned, I think that's allowed.

Regards,
Christian

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Freedos-devel mailing list
Freedos-devel@...
https://lists.sourceforge.net/lists/listinfo/freedos-devel

[dos386]

> > > but then virtual-86 mode is set and I can no loger use 32bit real-mode pointers!
> > IF you need PM from V86 there is VCPI.
> Protected mode isn't the same as flat real mode.

NO, but there is no absolute need to use flat [un]real mode.

> flat-real mode, VCPI and DPMI work and how they deal with other
> previously loaded systems to avoid collisions?

DPMI is not very relevant for real mode drivers. The important
quetions are:

- Do we have a 32-bit CPU ?
- Real or V86 mode ?
- Do we have an XMS host, version 2 or 3 ?
- If V86, do we have VCPI at least (if not, we are most likely inside
NTVDM) ?

> mentioned "if you want PM, you have VCPI", I think, meaning if I
> wanted to access PM memory.

NO. You need VCPI if you want to run your own PM code but find yourself in V86.
The XMS host as well as BIOS or EMM386 will temporarily switch to PM or unreal
for you when memcopying for you.

> Only I wish I knew more about VCPI.

Google for "VCPI.DOC" or "VCPI.ZIP" or "VCPI spec" or so.

FYI: VCPI is the "award winning standard" from 1988 :-D

FYII: forget or postpone VCPI, get your driver working using XMS and
INT $15 first.

> it will probably be enough to just rely on XMS functions for now.

Exactly.

> One doubt I have about XMS functions is this: the XMM specification states
> that if I lock some memory, I must unlock it as soon as possible. How soon is that?

Just before unhogging it should be fine.

> would be important to keep it locked for as long as the driver is running

Do it, hog and lock your XMS at driver start, and unlock and unhog
just before driver removal.

> Can I keep memory locked like for the whole time one process is running (i.e.: half an hour)?

Even 1 year :-D

> How bad is that?

NOT at all.

> can read Assembly, you can also look at the
> source code of HIMEMX, JEMM and HDPMI.

And FASM of course (hogging physical memory from BIOS or XMS host),
beware that the unreal mode used by it is VERY special and unique. :-)






--
~~~ wow ~~~

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Freedos-devel mailing list
Freedos-devel@...
https://lists.sourceforge.net/lists/listinfo/freedos-devel