DPMI32: a DOS target platform for Virtual Pascal.

Model description

The created programs run on most current DOS versions (3.3+) or DOS emulations and require an 386 or better processor and coprocessor or emulator. Memory mode is flat 4 GB where Mem[0] is the first byte of of Memory of the machine or session and for example the text video memory starts with Mem[$b8000].
Other models exist where nil^ is not a valid memory loction, but this model was choosen because of its simplicity and easy programming.

Useable DOS-Extenders are:

• DOS/4G,DOS/4GW of Rational Systems, Tenberry
• PMode/W of Charles Scheffold and Thomas Pytel
• CauseWay of Michael Devore, 3.5 from Scitech
• WDos/X of Michael Tippach
• Zurenava of Sergey Belyakov
• E.O.S. of Eclipse
• DOS32/A of Supernar Systems

Most DOS-Extenders support following memory interfaces:

• DPMI = Dos Protected Mode Interface
  provided by most DOS emulations
• VCPI = Virtual Control Program Interface
  interface provided by most DOS memory managers like Emm386/QEMM386
• own control of memory and processor if the processor is not switched into protected mode
  memory allocation ist done mostly by BIOS or XMS
  this variant allows more privileges and is a bit faster.

Independent of underlying memory interface the program can always use DPMI-services (that gave it the name). Some extenders can configured to provide virtual memory by swap files.

Implemented Units

• VpSysLow - here are most parts of platform depeded code stored
  for DPMI32 file VpSysD32.pas is included.
  Except shared memory and event semaphores it is implemented completely.
  If OS/2 or DR DOS TaskMgr are found at runtime, SysExecute can start programs asynchronous.
• System - platform independent in most parts
  For execption handling clones of OS/2 functions from Dpmi32.pas are used.
  Still undefined is the dll entry/termination code.
  Port/PortW/PortL is implemented in BASM in System.pas for DPMI32
• Crt, Dos, WinDos, Classes, Objects, ExeHdr, SysUtils, Turbo Vision..
  nearly no DPMI32 specific lines
• VpUtils, WinCrt
  partly separate implementation for most platforms
• MThread - defauld implementation of Threadfunctions for DPMI32
  The chosen multithreading is nonpreemptive and cooperative.
  There exist no priority classes, the next thread number will selected by random function.
  Some blocking or cooperative functions like Delay/SysCrtlSleep, (Sys)KeyPressed, (Sys)ReadKey cause switching to other threads.
• Resource - give access to resources of the program
  Called mainly by VpSysLow Resource functions
  Resources are embedded directly into the program image by PE2LE to save filesystem calls and make them compressible with normal exe-packers.
  The selected format does not allow named resources, but only Integer/Integer pairs for identifier and type.
• ESysErr
  This library only forces the linker to include ESysErr.RES to have message texts for most known DOS error codes.
• Dpmi32, Dpmi32df
  Calls to DPMI-services like memory management and exception handlers
• Idle, INT_13, INT_2526, SVGA, VSVGA, Taskm, WFSE,..
  Function libraries for VESA, disk, Wdos/X extensions
• deb_link
  This unit is the DOS-part needed to debug DPMI32-programs from VP IDE

EXE-Format, Linker, EXE-Packers

I wanted to use the internal linker from VP because of its many benefits. The supported DOS-Extenders know the LE-EXE-Format. The internal linker produces the LX-Exe (OS/2) and PE-Exe (Win32) format. LX would be better because LX is LE improved for OS/2 2.0, but the linker does not generate fixup records for programs. I wrote PE2LE. PE2LE also can do other things like binding the selected DOS-Extender into the generated executable.
To produce the now in Win32 format expected resources i created a RC1 program that uses OS/2 RC and then converts the result.

If you have finished and think about distributing your DPMI32-program you can compress it with an "generic" LE-compressor. On "real" operaing systems compressing executables has the disadvantage to that it prevents to share shareable code and constant segemnts between differen processes (Exepack 1/2 (LxLite) is an exception because the decompression is integrated into the OS). But for DOS (and DPMI32-programs are DOS-programs) this is not a problem. It is even recommended to use special executable compressors for LE because they compress better than most file archivers.

• 32Lite from Oleg Prokhorov
  32Lite uses aPLib compression library
  Please use -8:0 -9:0 switch with version 0.02d
• UPX from Markus F.X.J. Oberhumer and László Molnár
  UPX uses UCL compression library
  The use of UPX together with WDos/X leads to problems.
• XPack (xe) (needs fix)
  XPack uses aPLib compression library
• most DOS-Extender have theyr own build in compression, but it is then unusable with other extenders.

Description of debug modules

To debug a DPMI32-program from IDE i wrote unit deb_link. It is embedded into your compiled program and connects with debug-dll VpDbgDll.DLL. This dll is loaded by the IDE and communicates with it by the documented ODAPI interface.
In OS/2 the communication is done via two one-way pipes: Pipe one transmits commands from VpDbgDll like "i want 4 bytes at memory location y !", on the second deb_link sends the results like "i could read them, 4 bytes". This communication is fast and does not cost much resources because waiting any any end for more data blocks the waiting process. For Windows i could not found a similiar way, it is done by using 2 files in your filesystem. This does not block waiting processes and thus the whole communication is very slow.
Normally the initialisation code of deb_link would run after the initialisation code of the System unit and it would not possible to debug the initialisation of the runtime library. To change this PE2LE sets the entrypoint to deb_link if it find it in your program. Deb_link can not make calls to the runtime library anyway because you could not set breakpoints int the RTL otherwise.

execution order under OS/2:

• VP IDE instrucs VpDbgDll to start a program
• VpDbgDll recognizes a DPMI32-program and calls PE2LE if needed to convert it to an DOS executable
• VpDebDll starts a new DOS session with modified settings:
  device driver VPDDID.SYS will load and reserve the memory area at $00401000 (around 4 MB, the start address of a Win32 executable) and also will service to transmit the pipe names.
• The adaptable Autoexec.bat replacement \vp21\bin.d32\auto_os2.bat is executed
• The DOS-Extender portion of your program will run, it installs and loads user portion
• User portion checks for VPDDID.SYS an moves the program to $00401000
• Deb_link ask VPDDID.SYS for pipe names and opens connection to VpDbgDll. All of the following code can debugged from IDE.

In Windows things run similar, but here VPDDID.SYS is replaced by VPDDID.EXE and can not allocate memory at $00401000, auto_os2.bat is not executed.
To debug DPMI32-programs you have to insert deb_link into the uses clause. But your final program does not may contain deb_link, i recommend the following:

    program test;

    uses
      {$IfDef DPMI32}
        {$IfDef debug}
          deb_link,
        {$EndIf}
      {$EndIf}
      Crt,
      ....

Then put debug into conditional symbol entry in options/compiler dialog.

Help for DPMI32

After installing D32 you will find documentation of additional utils and units in VP online help. Source for this help is included in \vp21\source\d32\hilfe .
In \vp21\examples\d32 are some samples to demonstate some DPMI32 units.
A good programming documentation that also covers DPMI is Ralf Brown`s Interrupt List.

Comparison between BP 7 Real mode/DPMI 16

• The minimal program is larger than in BP. This is mainly caused by things to support ther programmer like exception handlers and location info
• An DOS-Extender is needed. If your program constantly calls DOS functions it will slowed down a bit by switching beween DOS mode and protected mode.
• Upper limit of array types is 2/4 GB instead of 64 KB.
• The program can use more comfortable GetMem/New instead of XMS/EMS calls to allocate and use memory.
• On most PC's the DOS-Extenders for DPMI32 are more stable than Borland's DPMI16

Porting of DOS-functions from BP 7 sources to DPMI32

• In most case there perhaps exist a platfrom independent function. Best place to look for it is VpSysLow, VpUtils and SysUtils. You can also use the search function on online help contents.
• Unit DOS does not have MsDos/Intr-functions in VP. For DPMI32 there exist similar replacements in Dpmi32.pas that also care about switching from protected mode to DOS: intr_realmode and callf_realmode. The equivalent to Dos.registers is Dpmi32df.real_mode_call_structure_typ. But you have to note that you can not expect that your program is loaded in first megabyte - the address room of DOS. To allocate buffers in low memory function Dpmi32.GetDosMem can used.
• Transformation of 16-bit BASM: here same hints like for MsDos/Intr apply.
  Do not try to assign protected mode selectors into real mode segments, do not use real mode segments in protected mode. You have no write permission using the CS selector in protected mode (uses DS:).
  If called function does not expect segment register or you do not care about the result you can still use traditional BASM:

        asm
          mov ax,$0003
          int $10
        end;
  

  Most times it is recommended to use 32-bit registers instead of 16-bit because this will produce smaller and faster code.

Installation of DPMI32 into Virtual Pascal

1. Extract the archive with "arj x -y dpmi32_lfs_279.arj D:\vp21\" into the base directory of Virtual Pascal, here D:\VP21.
   Arj for DOS, OS/2 or Windows will needed.
   (alternative zip/rar packages exist)
2. run applydif.bat. This will configure ini and batch files for your directory structure and updates classes.pas+sysutils.pas.
3. change to source\rtl subdir and launch _all.bat or _all.cmd, depending on host OS.
   This will recompile the runtime library and system unit for all platforms, and may take some amount of time...

Functions to implement

On my work/wish list are:

• Network support by using WATT-32 tcp/ip library
• DLL-support in the runtime library
• LxLite for VP 2.x
• graph unit for DPMI32, OS/2

Problem Solution

If you find a problem with DPMI32 and want contribute to its solution please send me a following details:

• Can you reproduce the problem ? How ?
• Does it appear on other platforms ?
• Include small source and compiled executable that show the problem
• Send improvement proposals

Motivation for DPMI32

The biggest program i wrote and that had cost me much of my spare time is Typ:
a filetype/ compiler/ packer/ .. -recognition, archive,.. lister program.
The project started 1994 from a program that had to react better to filename extenstions in Norton Commander than the internal function (.mod: Modula-2 source or Amiga module ?).
The program grow in size and i improved id.
At the time i tarted using OS/2 i wanted to have a OS/2 port and found the beta version of Virtual Pascal...
With the development of Virtual Pascal 2.0 there was added the Win32 target.
I do not have much use of it but it made VP more portable by introducing OS dependent and independent layers of the runtime library.
At about this time i got problems with debugging Typ in BP: the program went to big !
So i created a DOS target for VP.

Screen with Ratrace excample