Annotation of uae/docs/README.PROGRAMMERS, revision 1.1.1.3

1.1       root        1: You can help to make this program better. If you fix bugs or implement new
                      2: features, I'd be grateful if you send me patches. For a list of interesting
                      3: projects, and for a brief summary on how UAE works, see below.
                      4: 
                      5: A few guidelines for anyone who wants to help:
                      6: - Please contact me first before you implement major new features. Someone 
                      7:   else might be doing the same thing already. This has already happened :-(
                      8:   Even if no one else is working on this feature, there might be alternative
                      9:   and better/easier/more elegant ways to do it.
                     10: - If you have more than one Kickstart, try your code with each one.
                     11: - Patches are welcome in any form, but diff -u or diff -c output is preferred.
                     12:   If I get whole source files, the first thing I do is to run diff on it. You 
                     13:   can save me some work here (and make my mailbox smaller).
                     14: 
                     15: Some possible projects, in order of estimated difficulty:
1.1.1.2   root       16: - Someone running *BSD on a x86 might want to try using X86.S on such a
                     17:   system. It's likely that only configure needs to be modified.
                     18: - Add gamma correction
1.1.1.3 ! root       19: - If the serial port still isn't working (I've got no idea, I don't use it),
        !            20:   fix it.
1.1       root       21: - Someone with a 68020 data sheet might check whether all opcodes are
                     22:   decoded correctly and whether all instructions really do what they are 
                     23:   supposed to do (I'm pretty sure it's OK by now, but you never know...).
1.1.1.3 ! root       24: - Add more 2.0 packets to filesys.c
1.1.1.2   root       25: - Multi-thread support is there now, it just needs someone to test it on a SMP
                     26:   machine and to fix it so it improves speed instead of slowing the thing 
                     27:   down.
1.1       root       28: - Improve the Kickstart replacement to boot more demos.
                     29: - Snapshots as in CPE. Will need to collect all the variables containing
                     30:   important information. Fairly easy, but boring. (Use core dumps instead :-)
1.1.1.2   root       31:   _If_ someone attempts this, please be more clever than the various CPC
                     32:   emulators and dump state only at one fixed point in the frame, preferrably
1.1.1.3 ! root       33:   the vsync point. Also talk with Petter about this.
1.1       root       34: - Find out why uae.device has to be mounted manually with Kick 1.3.
                     35:   The problem seems to be that we don't have a handler for it. I _think_ what
                     36:   we need is the seglist of the standard filesystem handler. Problem is,
                     37:   DOS hasn't been started when the devices are initialized and so we can't get
                     38:   to the DosBase->RootNode->FileHandlerSeg pointer, and then there is the
                     39:   confusing matter of BCPL GlobVecs and other weird stuff...
1.1.1.2   root       40: - Some incompatibilities might be fixed with user-modifiable fudge variables
                     41:   the same way it's done in various C64 emulators.
                     42: - With the new display code, it would probably be easier than before to
                     43:   implement ECS resolutions - however, a lot of places rely on the OCS timing
                     44:   parameters and display sizes.
1.1       root       45: - Figure out a diskfile format that supports every possible non-standard
                     46:   format.
                     47: - Implement 68551 MMU. I have docs now. Not among the most necessary things.
1.1.1.3 ! root       48:   Should be done like exception 3 handling: add code to genamode in gencpu.c.
1.1       root       49: - Implement AGA support. Some bits and pieces exist.
                     50: - Reimplement Amiga OS. (Well-behaved) Amiga programs could then be made
                     51:   to use the X Window System as a "public screen". Of course, not all the
1.1.1.3 ! root       52:   OS would have to be re-done, only Intuition/GFX/Layers (which is enough).
        !            53:   [Started, look at gfxlib.c - not usable yet.]
1.1.1.2   root       54: - Find some extremely clever ways to optimize the smart update methods. Some
                     55:   ideas:
                     56:   a) Always use memcmpy() to check for bitplane differences. If no differences
                     57:      are found, see if BPLxDELAY got modified, if so, scroll.
1.1.1.3 ! root       58:      Problems:
1.1.1.2   root       59:       * You'd still have to draw a few pixels around the DIW borders. Not very
                     60:         hard.
                     61:       * Scrolling with memcpy in video memory can be terribly slow (no, I
                     62:         shouldn't have bought the cheaper video card with DRAMs)
                     63:       * At least every 15 pixels a full update has to be done since the
                     64:         bitplane pointers get updated after that. And that's with the slowest
                     65:        scrolling - if the playfield scrolls faster, the benefit converges
                     66:        against zero.
                     67:      You could also do vertical scrolling tests, but similar problems arise - 
                     68:      where should one check? One line above/below? What about faster
                     69:      scrolling? You could use the bitplane pointers as hints, but with
                     70:      double/triple buffering this gets problematic, too.
                     71:      On the whole, I don't think it would be worth the effort, even if it
                     72:      works very well for a few games.
                     73:   b) Well, there is no b). If I thought of something I forgot it while
                     74:      writing a).
1.1       root       75: - Port it to Java and Emacs Lisp
                     76: - A formal proof of correctness would be nice.
                     77: 
                     78: 
1.1.1.3 ! root       79: Source file layout
        !            80: 
        !            81: src/      contains (mostly) machine-independent C code.
        !            82: include/  contains header files included by C code.
        !            83: md-*/     CPU and compiler dependent files, linked to machdep by configure
        !            84: od-*/     operating system dependent files, linked to osdep by configure
        !            85: td-*/     thread library dependent files, linked to threaddep by configure
        !            86: sd-*/     Sound code. sd-* is only for sound systems which are not OS specific
        !            87:           or for which no "od-*" directory exists. Linked to sounddep
        !            88: targets/  Contains header files which contain some information about which
        !            89:           options a specific port of UAE understands.
        !            90: 
        !            91: 
        !            92: Coding style
        !            93: 
        !            94: As long as your code is hidden in a file buried in md-*/ or od-*/ where I
        !            95: never have a look at it, you can probably get away with not following these
        !            96: guidelines. 
        !            97: 
        !            98: * Do not include CR characters.
        !            99: * Do not use GNU C extensions if you can't hide them in a macro or in a
        !           100:   system-specific file so that an alternative implementation is available
        !           101:   when GNU C is not used.
        !           102:   This applies to _all_ OS/CPU/compiler specific details. Basically, nothing
        !           103:   of that sort should appear in src/*.c (we're a bit away from that goal at
        !           104:   the moment, but it's getting better).
        !           105: * Make sure your code does not make assumption about type sizes other than
        !           106:   the minimum widths allowed by C. If you need specific type sizes, use the
        !           107:   uae_u32 type and its friends.
        !           108: * Set up your editor so that tab characters round up to the next position
        !           109:   where ((cursorx-1) % 8) == 0, i.e. 8 space tabs. Do not use 4 space tabs,
        !           110:   that makes the code awful to read on other machines and worse to edit.
        !           111: * Lines can be up to 132 characters wide. Use SVGATextMode for the Linux
        !           112:   console, or use a windowing system in a high resolution.
        !           113: * C++ comments are a no-no in C code.
        !           114: * Indentation - look at some code in custom.c and try to follow it. Don't
        !           115:   use GNU 2-space-in-weird-places indentation, I find it awful. But _do_
        !           116:   follow the GNU rules for adding whitespace in expressions, and those for
        !           117:   breaking up multiple-line if statements.
        !           118:   Fixed indentation rules almost never make sense - break the rules if that
        !           119:   makes your code more readable.
        !           120:   Hint: Get jed from space.mit.edu, /pub/davis. It can indent your code
        !           121:   automatically. Put the following into your .jedrc, and it will come out
        !           122:   right:
        !           123:   C_INDENT             = 4;
        !           124:   C_BRACE              = 0;
        !           125:   C_BRA_NEWLINE                = 0;
        !           126:   C_Colon_Offset       = 1;
        !           127:   C_CONTINUED_OFFSET   = 4;
        !           128: 
        !           129: 
1.1       root      130: How it works
                    131: 
                    132: Let's start with the memory emulation. All addressable memory is split into
                    133: banks of 64K each. Each bank can define custom routines accessing bytes, 
                    134: words, and longwords. All banks that really represent physical memory just 
                    135: define these routines to write/read the specified amount of data to a chunk 
1.1.1.3 ! root      136: of memory. This memory area is organized as an array of uae_u8, which means 
1.1       root      137: that those parts of the emulator that want to access memory in a linear 
1.1.1.3 ! root      138: fashion can get a (uae_u8 *) pointer and use it to circumvent the overhead of
1.1.1.2   root      139: the put_*() and get_*() calls. That is done, for example, in the
1.1       root      140: pfield_doline() function which handles screen refreshes.
                    141: Memory banks that represent hardware registers (such as the custom chip bank
                    142: at 0xDF0000) can trap reads/writes and take any necessary actions.
                    143: 
                    144: To provide a good emulation of graphical effects, only one thing is vital:
                    145: Copper and playfield emulation have to be kept absolutely synchronous. If the
                    146: copper writes to (say) a color register in a specific cycle, the playfield 
1.1.1.2   root      147: hardware needs to use the new information in the next word of data it
1.1       root      148: processes.
                    149: UAE 0.1 used to call routines like do_pfield() and do_copper() each time the
                    150: CPU emulator had finished an instruction. That was one of the reasons why it
                    151: was so slow. Recent versions try to draw complete scanlines in one piece. This
                    152: is possible if the copper does not write to any registers affecting the
                    153: display during that scanline. Therefore, drawing the line is deferred until
1.1.1.2   root      154: the last cycle of the line. However, sometimes a register which affects how
                    155: the screen will look is modified before the end of the line (think of copper
                    156: plasmas). That's what "struct decision thisline_decision" is for. It is
                    157: initialized at the start of each line. During the line, whenever a vital
                    158: register is changed, one of the decide_*() functions is called and may modify
                    159: thisline_decision. There are several independent decisions:
                    160:  - which DIW should be used
                    161:  - where does data fetch start/stop (or is the line in the border altogether)
                    162:  - where should sprites be drawn (note: the same sprite can appear more than
                    163:    once on one scanline, see Turrican I world 3 levels 1 and 3 for the best
                    164:    example)
                    165:  - what are the playfield pointers at the start of DDF. Related, what data do
                    166:    they point to.
                    167:  - what are the playfield modulos at the end of DDF
                    168:  - coppermagic with the colors is remembered for later use
                    169:  - so is copper magic with the bitplane delay values. I used to think there
                    170:    was no useful application for modifying BPLCON1 while data is being
                    171:    displayed, but Sanity demos can make Amiga emulator programmers look real
                    172:    old.
                    173: 
                    174: All of this is remembered while the raster line is processed by the hardware.
                    175: After the line (at hsync), all the decisions are made if they weren't made
                    176: before. At that point the line can be drawn by playfield_draw_line.
                    177: Additionally, all the decisions from the previous displayed frame are saved
                    178: and compared with the new ones, since often lines are not modified between
                    179: frames. This saves a lot of redrawing work.
1.1       root      180: 
                    181: The CPU emulator no longer has to call all sorts of functions after each
                    182: instruction. Instead, it keeps a list of events that are scheduled (timer
                    183: interrupts, hsync and vsync events) and their "arrival time". Only the time
                    184: for the next event is checked after each CPU instruction. If it's higher than
                    185: the current cycle counter, the CPU can continue to execute.
                    186: 
1.1.1.2   root      187: Things that can't be supported with the current "decision" model:
                    188:   - Changes in lores/hires mode during one line. Dunno whether that was ever
                    189:     used in reality.
                    190:   - Changes to the bitplane DMA bit during one line. Hardly useful and not
1.1.1.3 ! root      191:     likely to be used. [but there are at least two programs which do ugly
1.1.1.2   root      192:     things like that, and there are some hacks in UAE that make those programs
                    193:     work (Magic 12 Ray of Hope 2 is one of these demos)]
                    194:   - Changes in bitplane data during one line. If programs do this kind of
                    195:     thing, it's most likely accidental and the program is broken. Can happen
                    196:     with programs that use the blitter incorrectly, like all the Andromeda
                    197:     demos.
                    198:   - others? (fill in if you can think of anything)
                    199: 
                    200: All in all, it's unlikely that this causes compatibility problems. If it does,
                    201: fudge values could be introduced (although that sort of thing gets messy
                    202: quickly).
1.1       root      203: 
                    204: 
1.1.1.3 ! root      205: * Native code vs. 68k code
1.1.1.2   root      206: 
1.1.1.3 ! root      207: It is possible to call native code from 68k code; autoconf.c has some routines
        !           208: which make setting up a call trap very easy. However, it is not as easy to
        !           209: call 68k code from native C code, at least not while Amiga Exec multitasking
        !           210: is running. You ask why?
        !           211: 
        !           212: Amiga process1 calls native function foo
        !           213: Native function foo calls some 68k function and goes into 68k mode
        !           214: Amiga context switch happens, process1 is put to sleep and process2 gets run.
        !           215: Amiga process2 calls native function foo
        !           216: Native function foo calls some 68k function and goes into 68k mode
        !           217: Amiga context switch happens, process2 is put to sleep and process1 gets run.
        !           218: Process 1 completes the 68k function called by foo and returns from 68k mode.
        !           219: 
        !           220: There. Now we are in function foo again. When it called the 68k code, process2
        !           221: was active. Now process1 is active, and the function we called in process2
        !           222: hasn't completed yet. What a mess.
        !           223: 
        !           224: To get around this, you need to do some stack magic. Code to do this exists,
        !           225: but it must be adapted for each port, since setting up a different stack is
        !           226: completely non-portable.
        !           227: 
        !           228: 
        !           229: * How multithreading in filesys.c works
        !           230: 
        !           231: AmigaOS is nice enough to start one processes for each mounted filesystem. All
        !           232: of these run in the 68k emulation code, i.e. in the main UAE thread. This is
        !           233: the reason why multithreading is desirable: if the main UAE thread blocks
        !           234: waiting for I/O, the CPU emulation can't continue to run. Since the Amiga OS
        !           235: is capable of multi-tasking, it is possible that other code could run until
        !           236: the I/O operation is complete. The most important bit of code that can run is
        !           237: the code that moves the mouse pointer - it's unpleasant if the pointer does
        !           238: not follow mouse movement during disk/CD accesses.
        !           239: 
        !           240: When a packet is received by the filesys.asm code, filesys_handler is called.
        !           241: This function always runs in the main UAE thread.
        !           242:  - In the single-threaded case, this function performs the action that was
        !           243:    requested, then returns 0 to indicate "action completed, reply packet".
        !           244:    Nothing else is performed.
        !           245:  - In the multi-threaded case, filesys_handler figures out which unit the
        !           246:    packet was for and sends the packet to the UAE thread responsible for
        !           247:    handling this unit. filesys_handler returns 0 to indicate: queue the
        !           248:    packet. Also, one (at that point unused) field in the packet is set to
        !           249:    0 to indicate that the action was not completed.
        !           250: 
        !           251: The latter case is the interesting one. The thread that got the packet does
        !           252: the following:
        !           253:  - perform the action as usual
        !           254:  - set the "command complete" field in the packet to -1.
        !           255:  - send a message to the AmigaOS (!) filesystem process. However, it can't do
        !           256:    that without some effort. We can't call 68k code from the emulator easily.
        !           257:    So we have to use an Amiga interrupt. The filesystem init code sets up an
        !           258:    Exec IntServer for the EXTER interrupt, and hsync_handler() checks
        !           259:    periodically whether the filesystem needs an interrupt and raises one if
        !           260:    necessary.
        !           261:    Only one dummy message is used per filesystem unit, which is allocated at
        !           262:    startup. This means that there must be some locking to prevent the unit
        !           263:    thread from sending the same message twice to the same port. To determine
        !           264:    whether the message is free, three counts are kept. "cmds_sent" is
        !           265:    incremented by the UAE thread whenever it has completed a command.
        !           266:    "cmds_acked" is set to the same value of cmds_sent at the point that the
        !           267:    interrupt handler got invoked and decided it must send a message. Finally,
        !           268:    cmds_complete is set to this value at the time the AmigaOS process receives
        !           269:    the dummy message. Whenever cmds_acked == cmds_complete, the dummy message
        !           270:    is free to be sent again.
        !           271:    
        !           272: The EXTER interrupt basically walks through the units, looks at the cmds_*
        !           273: fields and sends the dummy message to the Amiga filesystem process when
        !           274: possible and necessary.
        !           275: 
        !           276: When the Amiga filesystem process receives such a dummy message, it does the
        !           277: following:
        !           278:  - increment cmds_complete as described above.
        !           279:  - walk through the queue of unprocessed commands and see which ones now have
        !           280:    a status of -1, indicating that they are finished. These are removed from
        !           281:    the queue and replied to.
        !           282: 
        !           283: 
        !           284: * Calltraps at fixed locations
        !           285: 
        !           286: F0FF00: return from 68k mode.
        !           287: F0FF10: must have gotten lost somewhere ;)
        !           288: F0FF20: used by filesys.c to store away some information from the startup
        !           289:         packet.
        !           290: F0FF30: filesys_handler().
        !           291: F0FF40: startup_handler(), handles only the startup packet for each
        !           292:         filesystem.
        !           293: F0FF50: used by the EXTER interrupt which we set up for the filesystem.
        !           294: F0FF60: used by the uaectrl/uae-control programs (see uaelib.c)
        !           295: F0FF70: used by the task that gets set up for the mouse emulation.
1.1.1.2   root      296: 
1.1.1.3 ! root      297: * How the compiler works
1.1.1.2   root      298: 
                    299: .. yet to be written. To be decided, in fact.
1.1.1.3 ! root      300: 
        !           301: 
        !           302: Portability
        !           303: 
        !           304: This section was out of date. I'll rewrite it.
        !           305: Some day.

unix.superglobalmegacorp.com

This archive runs on limited infrastructure. Preserving old code on modern bandwidth. Automated agents are requested to crawl responsibly.