Notes from Marco on NetBSD and the Orange Pi

Marco says (with regard to multiple cores on the Allwinner H3):

I found the relevant BSD code to be much easier to track than the Linux counterpart. Its basically in these two files:

This one has the first core entry point "start:": http://bxr.su/NetBSD/sys/arch/evbarm/sunxi/sunxi_start.S
And this one has the entry point "cortex_mpstart:" for the other cores: http://bxr.su/NetBSD/sys/arch/arm/cortex/a9_mpsubr.S

(Note that the 7.1.2 NetBSD source tree has no sunxi named files, apparently these are recent additions (since 3-15-2018).
The second link exists exactly as is though, which is nice.  And it has the entry points named below.

The second file also has "cortex_init:" and "arm_cpuinit:", which the above entry points both call in this order.
They seem to do the main job (enabling SMP bit, enabling MMU, dealing with caches, setting up stacks...).

I've integrated them into my xinu tree and they work fine for me.
The pi1541 project sets up stacks not only for the active SVC mode, but also for all other CPU modes.
Indeed I found several sources stating that this is necessary).
So I've also set up those stacks for all modes on all cores.
Additionally I've added core startup synchronization and quickly hacked in the hardware spinlock to get rid of the apparent terminal mess
caused by the cores tracing at the same time ;-)
I will try to get the code into a testing fork of that multicore_mig branch on github the next days so others can try it.

AND

I've learned -alot- the last days about ARM initialization and MMU setup.
Fascinating topic. I found that BSDs startup code is a relatively easy to understand source of information.
(google "coretex_init").

On Kyu: The data aborts I'm having in the OPI core and memory tests are simply because you've zeroed the first page table descriptor,
thus prohibiting access to everything below 1MB, including the SRAM you use for those tests ;-)
The good thing is that it shows that the page table works as intended,.
Other than that, the OPI cores always start up for me and I get a nice blinking red LED.
I failed to reproduce any stability problems so far.
Of course there is stuff left to do (like having the cores enable their MMU). I find it especially fascinating -just how slow- cores are without MMU/caches enabled.
Cycles required for the same code are multiplied by the thousands.

On Xinu: The multicore_mig branch seems to have the latest code and builds fine.
But when starting the first core it crashed.
Considering the traces left in the startup code, it seems to me like it was given up on.
However, after adding  BSDs startup code and fixing several problems it also runs all cores fine with enabled MMUs and Caches so far.
The last thing I'm currently struggling with: As soon as the additional cores enable their MMU, all reads to the CPUCFG registers (CNT64_LOW for example) return zero.
RAM access works fine. Core0 can also read everything just fine.
I doubt its a pagetable problem, because the cores can write and read the SRAM at 0x0000000 fine.
I believe either its some register I'm not aware of yet or its a caching issue.

Oh, and heres a riddle for you:

https://github.com/real-xinu/xinu-orangepi/blob/multicore_migration/system/mmu.c

Look at mmu_set_dacr() and find the bug. That silly one took me a few hours ;-)