PRDS stack usage optimization in CAM clear

[canthony]

cam_cache controls the clearing of CAM across multiple processors using CPU connects.

To signal each CPU to clear its CAM and wait for all CPUs to do so uses the following protocol:

The CPU issuing the clear:

Set cam_wait.
Retry:

For each running CPU except itself:

Set fast_cam_pending [cpu]
Send a connect to the CPU.

Wait for all fast_cam_pending flags to clear; if timeout, go to retry.
Clear cam_wait.

The connect fault handler catches the connect fault and:

If fast_cam_pending [self]

Clear fast_cam_pending [self]
Clear CAM
Wait for cam_wait to clear, with interrupts masked and interrupt inhibit turned off to avoid Lockup Fault.

If a CPU is slow to respond, the issuing CPU will resignal the CPUs. If one of the CPUs did receive the connect fault and is waiting for cam_wait to clear, the connect fault handler will create a new PRDS stack frame and wait for cam_wait to clear there. When it does, the stack frame will be released and execution will return to the earlier cam_wait loop.

With the emulator running under host resource starvation conditions, it was observed that the timeout would happen frequently and often multiple times in a single CAM clear cycle; to the point where a PRDS stack overflow would result.

A simple patch to the cam_wait code prevents allocation of new stack frames on timeout, reducing overhead and increasing the PRDS stack size safety margin.

By adding a flag indication that a cam_wait loop is already on the stack, the timeout's connect fault can avoid creating a new stack frame and resume the executing in the first stack frame.

cpa orig>wired_fim.alm wired_fim.alm

Inserted in B:
B335
B336      " check to see if this is a nested connect due to a cam_cache timeout
B337                szn       prds$in_cam_wait_loop         are we already wait
\cing?
B338                tnz       more                yes skip waiting on this one
B339
Preceding:
A335                eppsb     prds$               push a frame onto the prds


Inserted in B:
B360                stc1      prds$in_cam_wait_loop  flag that we are waiting
B361
Preceding:
A355                inhibit   off       <-><-><-><-><-><-><-><-><-><-><-><->


Inserted in B:
B373                stz       prds$in_cam_wait_loop  clear waiting flag
B374
Preceding:
A366                cams      0                   now, clear A.M.'s

Comparison finished: 3 differences, 9 lines.


cpa >ldd>sl1>s>prds.cds prds.cds

A108          2 pad_mod_8 (6) fixed bin,
Changed by B to:
B108          2 in_cam_wait_loop bit (36) aligned,
B109          2 pad_mod_8 (5) fixed bin,

Comparison finished: 1 difference, 3 lines.

A tgz file with the edited source code and a bootable MST tape is available at
​https://drive.google.com/file/d/1hqD80jVMR__M8f_qHhdBNp_f60M_PaUU/view?usp=sharing

[Gary Dixon]

There is a possible mis-design issue described in this bug scenario. The prds is designed for use by interrupt handlers and certain faults like page fault (directed fault 0) that are expected to handle their interrupt or fault WITHOUT being interrupted during handling. Therefore, most (ALL?) handlers are expected to push their stack frame onto the BEGINNING of the prds, rather then pushing a frame atop any already present on the prds stack. That is true for interrupts handled on that stack by fim.alm, and for page_fault$fault.

The scenario described above talks about additional stack frames being pushed onto the prds if a connect fault is retried. This can be problematic in several ways.

• While interrupts are being masked, is the connect fault code designed to properly handle multiple stack frames pushed onto the end of the prds stack (rather than overwriting the beginning of the stack, thereby invalidating existing stack contents)? This may be one aspect of the bug being reported here.

• The prds is limited to 9 pages in size, and those pages are wired in memory. Thus, depending on stack frame sizes, the stack might overflow if the connect interrupt is retried too many times.

[canthony]

See ticket #344 Heavily loaded host systems may crash Multics with a PRDS stack overflow.