Annotation of uae/docs/README.PROGRAMMERS, revision 1.1.1.7

1.1       root        1: You can help to make this program better. If you fix bugs or implement new
                      2: features, I'd be grateful if you send me patches. For a list of interesting
                      3: projects, and for a brief summary on how UAE works, see below.
                      4: 
                      5: A few guidelines for anyone who wants to help:
                      6: - Please contact me first before you implement major new features. Someone 
                      7:   else might be doing the same thing already. This has already happened :-(
                      8:   Even if no one else is working on this feature, there might be alternative
                      9:   and better/easier/more elegant ways to do it.
                     10: - If you have more than one Kickstart, try your code with each one.
                     11: - Patches are welcome in any form, but diff -u or diff -c output is preferred.
                     12:   If I get whole source files, the first thing I do is to run diff on it. You 
                     13:   can save me some work here (and make my mailbox smaller).
                     14: 
                     15: Some possible projects, in order of estimated difficulty:
1.1.1.2   root       16: - Add gamma correction
1.1.1.3   root       17: - If the serial port still isn't working (I've got no idea, I don't use it),
                     18:   fix it.
1.1       root       19: - Someone with a 68020 data sheet might check whether all opcodes are
                     20:   decoded correctly and whether all instructions really do what they are 
                     21:   supposed to do (I'm pretty sure it's OK by now, but you never know...).
1.1.1.3   root       22: - Add more 2.0 packets to filesys.c
1.1.1.2   root       23: - Multi-thread support is there now, it just needs someone to test it on a SMP
                     24:   machine and to fix it so it improves speed instead of slowing the thing 
                     25:   down.
1.1       root       26: - Improve the Kickstart replacement to boot more demos.
                     27: - Snapshots as in CPE. Will need to collect all the variables containing
                     28:   important information. Fairly easy, but boring. (Use core dumps instead :-)
1.1.1.2   root       29:   _If_ someone attempts this, please be more clever than the various CPC
                     30:   emulators and dump state only at one fixed point in the frame, preferrably
1.1.1.3   root       31:   the vsync point. Also talk with Petter about this.
1.1       root       32: - Find out why uae.device has to be mounted manually with Kick 1.3.
                     33:   The problem seems to be that we don't have a handler for it. I _think_ what
                     34:   we need is the seglist of the standard filesystem handler. Problem is,
                     35:   DOS hasn't been started when the devices are initialized and so we can't get
                     36:   to the DosBase->RootNode->FileHandlerSeg pointer, and then there is the
                     37:   confusing matter of BCPL GlobVecs and other weird stuff...
1.1.1.2   root       38: - Some incompatibilities might be fixed with user-modifiable fudge variables
                     39:   the same way it's done in various C64 emulators.
                     40: - With the new display code, it would probably be easier than before to
                     41:   implement ECS resolutions - however, a lot of places rely on the OCS timing
                     42:   parameters and display sizes.
1.1       root       43: - Figure out a diskfile format that supports every possible non-standard
1.1.1.6   root       44:   format [I'll work on this soon (I hope)].
                     45: - Port bsdsocket library code to other operating systems than Win32.
1.1       root       46: - Implement 68551 MMU. I have docs now. Not among the most necessary things.
1.1.1.3   root       47:   Should be done like exception 3 handling: add code to genamode in gencpu.c.
1.1       root       48: - Implement AGA support. Some bits and pieces exist.
                     49: - Reimplement Amiga OS. (Well-behaved) Amiga programs could then be made
                     50:   to use the X Window System as a "public screen". Of course, not all the
1.1.1.3   root       51:   OS would have to be re-done, only Intuition/GFX/Layers (which is enough).
                     52:   [Started, look at gfxlib.c - not usable yet.]
1.1.1.2   root       53: - Find some extremely clever ways to optimize the smart update methods. Some
                     54:   ideas:
                     55:   a) Always use memcmpy() to check for bitplane differences. If no differences
                     56:      are found, see if BPLxDELAY got modified, if so, scroll.
1.1.1.3   root       57:      Problems:
1.1.1.2   root       58:       * You'd still have to draw a few pixels around the DIW borders. Not very
                     59:         hard.
                     60:       * Scrolling with memcpy in video memory can be terribly slow (no, I
                     61:         shouldn't have bought the cheaper video card with DRAMs)
                     62:       * At least every 15 pixels a full update has to be done since the
                     63:         bitplane pointers get updated after that. And that's with the slowest
                     64:        scrolling - if the playfield scrolls faster, the benefit converges
                     65:        against zero.
                     66:      You could also do vertical scrolling tests, but similar problems arise - 
                     67:      where should one check? One line above/below? What about faster
                     68:      scrolling? You could use the bitplane pointers as hints, but with
                     69:      double/triple buffering this gets problematic, too.
                     70:      On the whole, I don't think it would be worth the effort, even if it
                     71:      works very well for a few games.
                     72:   b) Well, there is no b). If I thought of something I forgot it while
                     73:      writing a).
1.1       root       74: - Port it to Java and Emacs Lisp
                     75: - A formal proof of correctness would be nice.
                     76: 
                     77: 
1.1.1.3   root       78: Source file layout
                     79: 
                     80: src/      contains (mostly) machine-independent C code.
                     81: include/  contains header files included by C code.
                     82: md-*/     CPU and compiler dependent files, linked to machdep by configure
                     83: od-*/     operating system dependent files, linked to osdep by configure
                     84: td-*/     thread library dependent files, linked to threaddep by configure
                     85: sd-*/     Sound code. sd-* is only for sound systems which are not OS specific
                     86:           or for which no "od-*" directory exists. Linked to sounddep
1.1.1.7 ! root       87: scg-*/    SCSI code.
        !            88:           This is the same code as in the cdrecord libscg, so this will become
        !            89:           obsolete as soon as the libscg is released seperately. In the meantime,
        !            90:           UAE will be distributed together with the code itself, because it
        !            91:           also requires patching: not every libscg implementation is thread-safe
        !            92:           in the current cdrecord version.
1.1.1.3   root       93: targets/  Contains header files which contain some information about which
                     94:           options a specific port of UAE understands.
                     95: 
                     96: 
                     97: Coding style
                     98: 
                     99: As long as your code is hidden in a file buried in md-*/ or od-*/ where I
                    100: never have a look at it, you can probably get away with not following these
                    101: guidelines. 
                    102: 
                    103: * Do not include CR characters.
                    104: * Do not use GNU C extensions if you can't hide them in a macro or in a
                    105:   system-specific file so that an alternative implementation is available
                    106:   when GNU C is not used.
                    107:   This applies to _all_ OS/CPU/compiler specific details. Basically, nothing
                    108:   of that sort should appear in src/*.c (we're a bit away from that goal at
                    109:   the moment, but it's getting better).
                    110: * Make sure your code does not make assumption about type sizes other than
                    111:   the minimum widths allowed by C. If you need specific type sizes, use the
                    112:   uae_u32 type and its friends.
                    113: * Set up your editor so that tab characters round up to the next position
                    114:   where ((cursorx-1) % 8) == 0, i.e. 8 space tabs. Do not use 4 space tabs,
                    115:   that makes the code awful to read on other machines and worse to edit.
1.1.1.5   root      116:   (I'm talking about the tab character here, not indentation!)
1.1.1.3   root      117: * Lines can be up to 132 characters wide. Use SVGATextMode for the Linux
                    118:   console, or use a windowing system in a high resolution.
                    119: * C++ comments are a no-no in C code.
                    120: * Indentation - look at some code in custom.c and try to follow it. Don't
                    121:   use GNU 2-space-in-weird-places indentation, I find it awful. But _do_
                    122:   follow the GNU rules for adding whitespace in expressions, and those for
                    123:   breaking up multiple-line if statements.
                    124:   Fixed indentation rules almost never make sense - break the rules if that
                    125:   makes your code more readable.
                    126:   Hint: Get jed from space.mit.edu, /pub/davis. It can indent your code
                    127:   automatically. Put the following into your .jedrc, and it will come out
                    128:   right:
                    129:   C_INDENT             = 4;
                    130:   C_BRACE              = 0;
                    131:   C_BRA_NEWLINE                = 0;
                    132:   C_Colon_Offset       = 1;
                    133:   C_CONTINUED_OFFSET   = 4;
                    134: 
                    135: 
1.1       root      136: How it works
                    137: 
                    138: Let's start with the memory emulation. All addressable memory is split into
                    139: banks of 64K each. Each bank can define custom routines accessing bytes, 
                    140: words, and longwords. All banks that really represent physical memory just 
                    141: define these routines to write/read the specified amount of data to a chunk 
1.1.1.3   root      142: of memory. This memory area is organized as an array of uae_u8, which means 
1.1       root      143: that those parts of the emulator that want to access memory in a linear 
1.1.1.3   root      144: fashion can get a (uae_u8 *) pointer and use it to circumvent the overhead of
1.1.1.2   root      145: the put_*() and get_*() calls. That is done, for example, in the
1.1       root      146: pfield_doline() function which handles screen refreshes.
                    147: Memory banks that represent hardware registers (such as the custom chip bank
                    148: at 0xDF0000) can trap reads/writes and take any necessary actions.
                    149: 
                    150: To provide a good emulation of graphical effects, only one thing is vital:
                    151: Copper and playfield emulation have to be kept absolutely synchronous. If the
                    152: copper writes to (say) a color register in a specific cycle, the playfield 
1.1.1.2   root      153: hardware needs to use the new information in the next word of data it
1.1       root      154: processes.
                    155: UAE 0.1 used to call routines like do_pfield() and do_copper() each time the
                    156: CPU emulator had finished an instruction. That was one of the reasons why it
                    157: was so slow. Recent versions try to draw complete scanlines in one piece. This
                    158: is possible if the copper does not write to any registers affecting the
                    159: display during that scanline. Therefore, drawing the line is deferred until
1.1.1.2   root      160: the last cycle of the line. However, sometimes a register which affects how
                    161: the screen will look is modified before the end of the line (think of copper
                    162: plasmas). That's what "struct decision thisline_decision" is for. It is
                    163: initialized at the start of each line. During the line, whenever a vital
                    164: register is changed, one of the decide_*() functions is called and may modify
                    165: thisline_decision. There are several independent decisions:
                    166:  - which DIW should be used
                    167:  - where does data fetch start/stop (or is the line in the border altogether)
                    168:  - where should sprites be drawn (note: the same sprite can appear more than
                    169:    once on one scanline, see Turrican I world 3 levels 1 and 3 for the best
                    170:    example)
                    171:  - what are the playfield pointers at the start of DDF. Related, what data do
                    172:    they point to.
                    173:  - what are the playfield modulos at the end of DDF
                    174:  - coppermagic with the colors is remembered for later use
                    175:  - so is copper magic with the bitplane delay values. I used to think there
                    176:    was no useful application for modifying BPLCON1 while data is being
                    177:    displayed, but Sanity demos can make Amiga emulator programmers look real
                    178:    old.
                    179: 
                    180: All of this is remembered while the raster line is processed by the hardware.
                    181: After the line (at hsync), all the decisions are made if they weren't made
                    182: before. At that point the line can be drawn by playfield_draw_line.
                    183: Additionally, all the decisions from the previous displayed frame are saved
                    184: and compared with the new ones, since often lines are not modified between
                    185: frames. This saves a lot of redrawing work.
1.1       root      186: 
                    187: The CPU emulator no longer has to call all sorts of functions after each
                    188: instruction. Instead, it keeps a list of events that are scheduled (timer
                    189: interrupts, hsync and vsync events) and their "arrival time". Only the time
                    190: for the next event is checked after each CPU instruction. If it's higher than
                    191: the current cycle counter, the CPU can continue to execute.
                    192: 
1.1.1.2   root      193: Things that can't be supported with the current "decision" model:
                    194:   - Changes in lores/hires mode during one line. Dunno whether that was ever
                    195:     used in reality.
                    196:   - Changes to the bitplane DMA bit during one line. Hardly useful and not
1.1.1.3   root      197:     likely to be used. [but there are at least two programs which do ugly
1.1.1.2   root      198:     things like that, and there are some hacks in UAE that make those programs
                    199:     work (Magic 12 Ray of Hope 2 is one of these demos)]
                    200:   - Changes in bitplane data during one line. If programs do this kind of
                    201:     thing, it's most likely accidental and the program is broken. Can happen
                    202:     with programs that use the blitter incorrectly, like all the Andromeda
                    203:     demos.
                    204:   - others? (fill in if you can think of anything)
                    205: 
                    206: All in all, it's unlikely that this causes compatibility problems. If it does,
                    207: fudge values could be introduced (although that sort of thing gets messy
                    208: quickly).
1.1       root      209: 
                    210: 
1.1.1.3   root      211: * Native code vs. 68k code
1.1.1.2   root      212: 
1.1.1.3   root      213: It is possible to call native code from 68k code; autoconf.c has some routines
                    214: which make setting up a call trap very easy. However, it is not as easy to
                    215: call 68k code from native C code, at least not while Amiga Exec multitasking
                    216: is running. You ask why?
                    217: 
                    218: Amiga process1 calls native function foo
                    219: Native function foo calls some 68k function and goes into 68k mode
                    220: Amiga context switch happens, process1 is put to sleep and process2 gets run.
                    221: Amiga process2 calls native function foo
                    222: Native function foo calls some 68k function and goes into 68k mode
                    223: Amiga context switch happens, process2 is put to sleep and process1 gets run.
                    224: Process 1 completes the 68k function called by foo and returns from 68k mode.
                    225: 
                    226: There. Now we are in function foo again. When it called the 68k code, process2
                    227: was active. Now process1 is active, and the function we called in process2
                    228: hasn't completed yet. What a mess.
                    229: 
                    230: To get around this, you need to do some stack magic. Code to do this exists,
                    231: but it must be adapted for each port, since setting up a different stack is
                    232: completely non-portable.
                    233: 
                    234: 
                    235: * How multithreading in filesys.c works
                    236: 
                    237: AmigaOS is nice enough to start one processes for each mounted filesystem. All
                    238: of these run in the 68k emulation code, i.e. in the main UAE thread. This is
                    239: the reason why multithreading is desirable: if the main UAE thread blocks
                    240: waiting for I/O, the CPU emulation can't continue to run. Since the Amiga OS
                    241: is capable of multi-tasking, it is possible that other code could run until
                    242: the I/O operation is complete. The most important bit of code that can run is
                    243: the code that moves the mouse pointer - it's unpleasant if the pointer does
                    244: not follow mouse movement during disk/CD accesses.
                    245: 
                    246: When a packet is received by the filesys.asm code, filesys_handler is called.
                    247: This function always runs in the main UAE thread.
                    248:  - In the single-threaded case, this function performs the action that was
                    249:    requested, then returns 0 to indicate "action completed, reply packet".
                    250:    Nothing else is performed.
                    251:  - In the multi-threaded case, filesys_handler figures out which unit the
                    252:    packet was for and sends the packet to the UAE thread responsible for
                    253:    handling this unit. filesys_handler returns 0 to indicate: queue the
                    254:    packet. Also, one (at that point unused) field in the packet is set to
                    255:    0 to indicate that the action was not completed.
                    256: 
                    257: The latter case is the interesting one. The thread that got the packet does
                    258: the following:
                    259:  - perform the action as usual
                    260:  - set the "command complete" field in the packet to -1.
                    261:  - send a message to the AmigaOS (!) filesystem process. However, it can't do
                    262:    that without some effort. We can't call 68k code from the emulator easily.
                    263:    So we have to use an Amiga interrupt. The filesystem init code sets up an
                    264:    Exec IntServer for the EXTER interrupt, and hsync_handler() checks
                    265:    periodically whether the filesystem needs an interrupt and raises one if
                    266:    necessary.
                    267:    Only one dummy message is used per filesystem unit, which is allocated at
                    268:    startup. This means that there must be some locking to prevent the unit
                    269:    thread from sending the same message twice to the same port. To determine
                    270:    whether the message is free, three counts are kept. "cmds_sent" is
                    271:    incremented by the UAE thread whenever it has completed a command.
                    272:    "cmds_acked" is set to the same value of cmds_sent at the point that the
                    273:    interrupt handler got invoked and decided it must send a message. Finally,
                    274:    cmds_complete is set to this value at the time the AmigaOS process receives
                    275:    the dummy message. Whenever cmds_acked == cmds_complete, the dummy message
                    276:    is free to be sent again.
                    277:    
                    278: The EXTER interrupt basically walks through the units, looks at the cmds_*
                    279: fields and sends the dummy message to the Amiga filesystem process when
                    280: possible and necessary.
                    281: 
                    282: When the Amiga filesystem process receives such a dummy message, it does the
                    283: following:
                    284:  - increment cmds_complete as described above.
                    285:  - walk through the queue of unprocessed commands and see which ones now have
                    286:    a status of -1, indicating that they are finished. These are removed from
                    287:    the queue and replied to.
                    288: 
                    289: 
                    290: * Calltraps at fixed locations
                    291: 
                    292: F0FF00: return from 68k mode.
                    293: F0FF10: must have gotten lost somewhere ;)
                    294: F0FF20: used by filesys.c to store away some information from the startup
                    295:         packet.
                    296: F0FF30: filesys_handler().
                    297: F0FF40: startup_handler(), handles only the startup packet for each
                    298:         filesystem.
                    299: F0FF50: used by the EXTER interrupt which we set up for the filesystem.
                    300: F0FF60: used by the uaectrl/uae-control programs (see uaelib.c)
                    301: F0FF70: used by the task that gets set up for the mouse emulation.
1.1.1.7 ! root      302: F0FF80: getchipmemsize()
        !           303: F0FF90: uaeexe
        !           304: F0FFA0: timehack
1.1.1.5   root      305: 
1.1.1.3   root      306: * How the compiler works
1.1.1.2   root      307: 
                    308: .. yet to be written. To be decided, in fact.
1.1.1.3   root      309: 
                    310: 
                    311: Portability
                    312: 
                    313: This section was out of date. I'll rewrite it.
                    314: Some day.

unix.superglobalmegacorp.com

This archive runs on limited infrastructure. Preserving old code on modern bandwidth. Automated agents are requested to crawl responsibly.