[[tests:arnold_test_suite]]
 
Translations of this page:
Table of Contents

Results of ACE to Arnold test suite

These results are related to the great test suite created by Kevin Thacker for Arnold.

The tests are not enough to ensure a perfect emulation, but they give an idea of the overall quality of the emulation.

The good news is that ACE is performing perfectly with most of the tests, and the failing ones are generally related to secondary features never used in CPC softwares.

Anyway, all the tests should pass 100% at the end!

The test machines

Here is the list of the real hardwares used to run the tests.

Type Description Comments
0 Amstrad CPC6128 with CRTC HD6845SP1) French firmware (f3)
1 Amstrad CPC6128 with CRTC UM6845R2)
Amstrad CPC6128 with CRTC UM6845R3) 		French firmware (f3)4)
English firmware (v3)
2 Amstrad CPC6128 with CRTC MC6845P5)
Amstrad CPC664 with CRTC MC6845P6) 		English firmware (v3)
English firmware (v2)
3 Amstrad 6128plus revision A
Amstrad 6128plus revision D
Amstrad 6128plus revision G 		French firmware (f4)7)
French firmware (f4)
French firmware (f4)
4 Amstrad CPC6128 version O French firmware (f3)

How to read this ?

Each test was run on the real hardware and then on ACE for each CRTC type. Each line is reporting the results for one test, and the columns are giving information about these results.

In the first column

This is the global status for the test.

  • ”:-D” means that the test makes no difference between ACE and the real hardwares.
  • ”:-)” means that the test is reporting differences which are not critical for emulation quality. It should be fixed but it is not a priority unless a known software requires it.
  • ”:-|” means that the test is reporting errors that shall be fixed in ACE. Some softwares might already be impacted by this flaw.

In the column "Results with ACE"

The sub-columns are for the well-known 5 CRTC types.

For both “on screen report” and “interactive” tests:

  • “8-)” means that the test is totally successful.
  • ”:-(” means that the test partially failed.
  • “m(” means that the test totally failed.
  • an empty column means that the test was not perfomed yet.

For “visual” tests:

  • if a grab is present, it fully matches CPC display and the test is totally succesful.
  • ”:-(” means that the test partially failed.
  • “m(” means that the test totally failed.
  • an empty columns means that the test was not perfomed yet.

Please note that the test suite is not perfect yet. Some “on screen report” test are failing on real CPC, and are then marked successful on ACE when they are failing exactly in the same manner. Also, some “visual” test are not totally stable on real CPC. Refer to the additional notes on the test to understand the details.

The facts!

Test name Results with ACE 1.16 Kind of result Comments
0 1 2 3 4
:-D 8kscreen Visual (static)
:-D brunword 8-)8) 8-)9) 8-)10) 8-)11) On screen report Failing in the same manner on CPCs and ACE, which is good.
:-D clr Visual (color is cycling)
:-D clrbit5 Visual (color cycling at startup)
:-D colours Visual (static)
:-D cpcborder Visual (static)
:-D cpccol Visual (color cycling)
:-D cpcpen Visual (static)
:-) cpctest 8-)12) 8-)13) :-(14) 8-)15) On screen report Some tests are failing in the same manner on CPCs and ACE, which is good.
:-) cpu :-(16) :-(17) :-(18) :-(19) On screen report Only one test is failing.

This is a known bug on Z80 CMOS where IFF2 is copied into P/V flag before being updated when an interrupt occurs during LD A,R or LD A,I.

This Z80 bug is not emulated by ACE and won't be emulated anytime soon (totally useless and over complicated to implement in a efficient way).
:-D gaint Visual (static)
:-D crtc0_3_4_r8 Visual (static)
:-D crtc1_r6 Visual (static)
:-) crtc3_4_status 8-)20) 8-)21) 8-)22) :-(23) On screen report Some tests are failing in the same manner on CPCs and ACE, which is good.

On CRTC 3, cursor blinking and cursor start/End are not emulated at all (then, status cannot be reported on Reg3).

It might me added in the future, but since it is totally useless it is a top low priority.
:-D crtcinp Visual (static)
:-) crtctest :-(24) :-(25) :-(26) 8-)27) On screen report Some tests are failing in the same manner on CPCs and ACE.

Basically, everything is working apart of some stuff related to reg8.

Please note that test code needs to be fixed to work as expected (LD A,'3' should be removed at CRTC selection: poke &400d,asc(”?”)).

On real CPC+ this test is triggering the memory refresh issue; some parts of the (display) memory is destroyed during tests.

CRTC register 8 is not fully implemented yet in ACE, the related tests should be fixed when it is done. Also, R8 - count lines sometimes fails on real CPC with CRTC 1 too!
:-) crtcstatus 8-) :-(28) :-( On screen report
:-D crtc_r1 Visual (static)
:-D crtc_r12 8-) 8-) 8-) Interactive On CRTC 1 the changing area is &6E-&6A and not &6E-&69 as stated in the test.

On CRTC 3 the changing area is &70-&6C and not &72-&6C as stated in the test.
:-D crtc_r12_r5 8-) 8-) 8-) Interactive
:-D crtc_r12b 8-) 8-) 8-) Interactive
:-D crtc_r13 8-) 8-) 8-) Interactive
:-D crtc_r1_2 Visual (static)
:-) crtc_r4 8-) :-(29) 8-) Interactive On CRTC 3 the changing area is &B0-&AC and not &B1-&AA as stated in the test.
:-D crtc_r4b 8-) 8-) 8-) Interactive
:-D crtc_r5 Visual (static)
:-| crtc_r5b 8-) m( 8-) Interactive Sick mode of CRTC 1 is triggered at &5EA and not &5EC as stated in the comments of the test
:-| crtc_r5c 8-) m( 8-) Interactive
:-| crtc_r5d :-(30) 8-) 8-) Interactive
:-D crtc_r5e Visual (static)
:-| crtc_r5e2 m( :-(31) 8-) Interactive
:-D crtc_r5ep Visual (static)
:-| crtc_r5f m( :-(32) 8-) Interactive On CRTC 1 changed are triggered at &285-&281 and not &286-&281 as stated in the comments of the test
:-| crtc_r6 :-( Visual (static with some part of lines blinking) This test is not stable on CRTC 1.
:-D crtc_r6b 8-) 8-) 8-) Interactive
:-| crtc_r6c 8-) :-(33) 8-) Interactive
:-| crtc_r6d 8-) :-(34) 8-) Interactive
:-D crtc_r7 8-) 8-) 8-) Interactive This is not mentionned in the test comments but VSync alignement regarding display words is changing between ranges &2AE-&2AA and &2AB-…) on CRTC 1
:-| crtc_r9 :-(35) :-(36) :-(37) Interactive
:-| crtc_r9b :-(38) :-(39) 8-) Interactive On CRTC 3 the changing area is &90-&8C and not &91-&8C as stated in the test.
:-D dkram 8-) 8-) 8-) 8-) On screen report DK'Tronics bugs emulation of ACE should be activated.
:-) flood :-( Visual (static) Small issue on CRTC 2… well… not a big deal. To be fixed one day or another.
:-) flood2 :-( Visual (static) Small issue on CRTC 2…
:-) ppi :-( :-( :-( 8-) On screen report In fact the test works with no issue in PPI mode 0 (the only one usefull on CPC) but mode 1 and 2 are not emulated by ACE.

Since these modes are totally useless it is not a priority by now to add them.

Also, some tests are failing in a similar way on both the real hardware and ACE.

In addition, I noticed that on some CPC the VBL can be triggered from the PPI (at least with the bizarre Gate Array).

I guess also there is a bug in one of the tests: the Hi-Z test on PSG register write does not do what is expected (it selects register FF for reading).
:-D ppiaudio 8-) 8-) 8-) 8-) Audio and visual (border color cycling)
:-D psg 8-) 8-) 8-) 8-) On screen report
:-D psgexer 8-) 8-) 8-) 8-) Audio feedback
:-D type1ctrl 8-) 8-) 8-) 8-) On screen report Some tests are failed in a similar way on both the real hardware and ACE.
:-) rtestopcodes 8-) 8-) 8-) :-(40) On screen report IM 0 emulation on CPC+ is missing; but it is actually totally useless and not a priority.
:-D inout 8-) 8-) 8-) 8-) On screen report
:-D modetrig 8-) 8-) 8-) Interactive
:-D vblank Visual (static)
:-| dmatest N/A N/A N/A :-( N/A On screen report Two tests are failing.

The only one that might have some impact is “dma int request test (dcsr bits)” which is triggering an issue when expecting bit 1 from DCSR to be reset at some point.

The second failing test is “CRTC R0 length and dma” for one single value (when R0=1) which is very unlikely to happen in real life.
:-D dmatiming N/A N/A N/A N/A Visual (static)
:-D asic_test N/A N/A N/A 8-) N/A On screen report
:-D asic_raster N/A N/A N/A N/A Visual (static)

More tests to come…

Bonus

Here is a small AREXX script to compile, load and run a test from Arnold Test Suite into ACE.

Usage: 

RunTest.rexx <testname>

Script: 

/* RunTest.rexx - Compile and run Kevin Thacker Arnold Test Suite
 *
 * Syntax: RunTest.rexx <test name>
 *
 * $VER: RunTest.rexx 1.3 (28.02.18) © 2016-2018 by Philippe 'OffseT' Rimauro <offset@cpcscene.net>
 * Requires ACE 1.13 or later
 *          Pasmo Z80 assembler (must be in the path)
 */

OPTIONS RESULTS
SIGNAL ON BREAK_C

PARSE ARG test".asm"
IF test="" THEN DO
    PARSE ARG test
END

IF test="" THEN DO
    CALL FatalError('No test name specified!','My mistake!')
    EXIT
END

/* External libraries */
LoadLib("rexxsupport.library")

appname = "ACE Rexx"

/* Look for file test */
IF ~EXISTS(test".asm") THEN DO
    CALL FatalError('Test 'test' not found!','Gasp!')
    EXIT
END

/* Look for ACE running */
IF ~SHOW('P','ACE.1') THEN DO
    CALL FatalError('ACE CPC Emulator process not found!','Oops!')
    EXIT
END

/*
 * Main
 */
ADDRESS "ACE.1"

/* Display ACE version */
"VERSION"
SAY "ACE version" RESULT "found."

/* Reset ACE */
SAY "Cleaning up everything..."
"BREAKPOINTS ACTIVATE 0"
"START"
"RESET HARD"

/* Let the firmware boot */
DELAY(100)

/* Pause ACE while working */
"PAUSE"

/* Assemble code */
SAY "Assembling source code..."
ADDRESS "COMMAND" "pasmo --amsdos "test".asm T:"test".bin T:"test".symbol"

/* Check for assembling error... */
IF RC~=0 THEN DO
    "START"
    EXIT
END

/* Load code into ACE! */
SAY "Loading code into ACE..."
"LOAD FILE T:"test".bin "

/* Start ACE again */
"BREAKPOINTS ACTIVATE 1"
"START"

/* Auto-launch the assembled program */
SAY "Auto-launching the assembled program!"
"KEYBOARDSTROKE CALL 256"

EXIT

/*
 * External lib loader
 */
LoadLib:
    PARSE ARG lib

    IF SHOW("L", lib)=0 THEN
        IF ADDLIB(lib, 0, -30, 0)=0 THEN
            CALL FatalError(lib' not found!*NPlease install it.','Okay!')
    RETURN RC

/*
 * Fatal error
 */
FatalError:
    PARSE ARG message,button

    ADDRESS "COMMAND" 'RequestChoice >NIL: TITLE "'||appname||'" BODY "'||message||'" GADGETS "'||button||'"'
    EXIT
1) , 3) , 5) , 6) Gate Array 40010
2) bizarre Gate Array (40010 compatible)
4) Hey! That's the one that printed Quasar CPC pages!
7) Hey! That's the one on which most of Quasar CPC pages were created!
8) , 9) , 10) , 11) 8-) rom test &4000 en/dis-FAILED (got 33 expected 0D)
12) , 13) 8-) CPC int (HSYNC width)-FAILED (got 00 expected 01)
14) :-( Deadlock due to reg2=0 specific case not emulated)
15) 8-) CPC int (HSYNC pos)-FAILED (got 00 expected 01) / 8-) CPC int (HSYNC width)-FAILED (got 00 expected 01)
16) , 17) , 18) , 19) 8-) ld a,r/ld a,i p/v int test-FAILED (got 04 expected 00 - FAIL)
20) , 22) 8-) RC==0 and RC==MR-FAILED (got 01 expected 00) / 8-) VBlank-FAILED (got 40 expected 00) / 8-) Vsync end-FAILED (got 01 expected 00) / 8-) Cursor flash rate-FAILED (got 00 expected 10) / 8-) Cursor active on scanline-FAILED (got 01 expected 00)
21) 8-) RC==0 and RC==MR-FAILED (got 01 expected 00) / 8-) VBlank-FAILED (got 00 expected 40) / 8-) Vsync end-FAILED (got 01 expected 00) / 8-) Cursor flash rate-FAILED (got 00 expected 10) / 8-) Cursor active on scanline-FAILED (got 01 expected 00)
23) 8-) VBlank-FAILED (got 40 expected 00) / :-( Vsync end-SUCCEEDED instead of FAILED (got 01 expected 00) / m( Cursor flash rate-FAILED (got 00 expected 10) instead of SUCCEEDED / m( Cursor active on scanline-FAILED (got 00 - OK) instead of (got 01 expected 00)
24) 8-) vsync r8 test (r8=1)-FAILED (got 00 expected 01) / m( Deadlock: vsync r8 test (r8=3) / 8-) vsync r8 htot/2 test-FAILED (got 01 expected FF) / m( Deadlock: R8 - count lines
25) 8-) vsync r8 test (r8=1)-FAILED (got 00 expected 01) / 8-) vsync r8 test (r8=3)-FAILED (got 00 expected 01) / 8-) vsync r8 htot/2 test-FAILED (got 01 expected FF) / m( R8 - count lines-FAILED (got 9B expected 99) instead SUCCEEDED
26) 8-) vsync r8 test (r8=1)-FAILED (got 00 expected 01) / 8-) vsync r8 test (r8=3)-FAILED (got 00 expected 01) / 8-) vsync r8 htot/2 test-FAILED (got 01 expected FF) / m( Deadlock: R8 - count lines / 8-) vsync length test-FAILED (got 10 expected 01) / m( hsync pos test-FAILED (got 01 expected 00) instead of SUCCEEDED (only a few value are failing, probably related to wrong CRTC 2 emulation when R2=0) / m( hsync length measure (type 2)-FAILED (got FF expected 2F) instead of SUCCEEDED (Probably related to wrong CRTC 2 emulation when R2=0 too)
27) 8-) vsync r8 test (r8=1)-FAILED (got 00 expected 01) / 8-) vsync r8 test (r8=3)-FAILED (got 00 expected 01) / 8-) vsync r8 htot/2 test-FAILED (got 01 expected FF) / 8-) R8 - count lines-FAILED (got xx expected yy)
28) Almost ok, just a few values are shifted
29) Ok, except &AF where “sick mode” is not emulated
30) , 31) , 32) , 33) , 34) , 36) , 39) Ok, except one area triggering which is shifted by 1 microsecond
35) , 38) Ok, except one area triggering which is shifted by 1 microsecond, and the flikering value which is not flickering
37) Ok, except the extra line of the second changing area which is not displayed properly
40) IM 0 not emulated
tests/arnold_test_suite.txt · Last modified: 2018/03/06 19:19 by Philippe Rimauro
 
Except where otherwise noted, content on this wiki is licensed under the following license: Public Domain