/*$Header: /export/disk01/cvs/CVS/pacs/obsw/dmc/sources/logbook.txt,v 1.8 2009/04/23 13:51:12 amazy Exp $*/ This logbook has been started on the 22/02/01 while the development of the software has been started a few months earlier. However, during the first few months, we have mainly performed basic tests to slowly build a first prototype of the application. Here is the status of the OBS (On-Board Software) on the 22/02/01 : - DpuReceiver - receives and understand read/write/trigger commands - executes read/write commands and send response to DpuSender - forwards trigger commands to Sequencer - status : COMPLETED according to current requirements - DpuSender - is able to send any kind of packets (ACK, NACK, READ, HK, HK_DIAG) when requested by another task. - status : COMPLETED according to current requirements - Sequencer - the complete skeleton is implemented (to interprete and execute trigger commands and sequences). - only about 15 commands have been implemented until now - receives its synchronisation from a DecReceiver - status : 30 % COMPLETED according to current requirements (a lot of commands to be implemented). - HkController - the complete skeleton is implemented (to perform the HK acquisition, to write the measures in a buffer and to send it to the DpuSender). - only about 10 HK measures have been implemented until now - status : 15 % COMPLETED according to current requirements (a lot of measures to be added). - HkDiagController - since the skeleton is the same as the HkController, it is also completed. - status : 80 % COMPLETED (packet formating is not yet definitive). - DecReceiver - simply receives a dummy message from a DEC Simulator on a PC to generate the synchronisation signals needed by the Sequencer. - status : 10 % COMPLETED ******************************************************************************* WORK 23/02/01 ------------- A prototype of the ChopGrat.s IRS has been written. It generates a simple trajectory in 16 steps (between the old target and the new one). A Chopper simulator task has been written. It works at a user defined frequency (nominally 10Hz). It can be started and stopped thanks to START_CHOPPER_SIMULATOR and STOP_CHOPPER_SIMULATOR commands. The only thing it does is writing the CurrentSetPoint in CurrentPosition. New HK measures have been added to be able to monitor the IRS functionnality through Diagnostic housekeeping. For performance tests purpose, we have also added the CPU workload in these measures. TESTS 23/02/01 -------------- To validate the basic ChopGrat.s IRS, we make HkDiag acquisition at 20Hz while the pulse generator works at 10Hz. By this way, we can observe that the CurrentSetPoint is being modified slowly (in 16 steps) when one of the MOVE_CHOPPER command is sent. Performance tests have also been performed to evaluate CPU workload. Note : since the IRS is really not complete by now, performance results are representative of final application. However, they can be used to extrapolate the final results. The IRS is about 35 instructions (not optimized). It takes about 1.5 µs to execute the entire IRS. The first 4 instructions and the last 4 ones toggle a LED so, by looking at the oscilloscope, we can measure the time spent inside the IRS. We have also to ensure that the IRS is effectively called for each pulse generated. Since the TrajectoryIndex is incremented at each IRS call and since it is included in the HkDiag every second, we can roughly check (on a long period) that TrajectoryIndex has been correctly incremented. Results : Pulse Freq (Hz) CPU Workload Every IRQ is caught ----------------------------------------------------------- 1 6.8 % YES 4000 8.5 % YES 8000 11.0 % YES 15000 12.6 % YES 25000 17.8 % YES 75000 38.6 % NO (99.996% are caught) 150000 70.8 % NO (98.5% are caught) ************************************************************************************ WORK 26/02/01 ------------- Dithering has been added to chopper commands. A random number distribution has been generated with Excel and included in "pid_ctrl.c". We have also update the HK Diagnostic mechanism to send the data only when the gHkDiagPacketBuffer is full. It will be tested tomorrow. TESTS 26/02/01 -------------- The default sequence has been updated to include a MOVE_CHOPPER_RELATIVE_DITHER. The HK diagnostic allows us to monitor the evolution of dithering values. Note : We have not yet executed this sequence during a long enough period to validate that the circular indexing of the random number distribution is working fine (TODO). ************************************************************************************ WORK 27/02/01 ------------- The Verbose mechanism has been updated since we were generating too much information when a high level of detail was required. We have now added some categories to allow us to display the information related only to the selected categories (through the SetVerboseCategories command). A small program (HkViewer.exe) has also been built on the PC to allow to visualize the HK data in Excel. TESTS 27/02/01 -------------- Thanks to the new verbose mechanism and to HkViewer, we can validate that the Hk Diagnostic measures are buffered until the buffer is full or until the Hk Diagnostic is stopped by command. Note : we have not yet validated that the packet is sent when the Sequence ID is modified (DONE on 11/04/01). Note : the last entry of the HK file stored on the PC always contain a repetition of the last acquired measure. We have to check wheter this problem is coming from the SimDPU or from the OBS (DONE on 10/04/01 problems comes from HkViewer). ************************************************************************************* WORK 14/03/01 ------------- Included CRC file (coming from IFSI) in the project and adapted them to our standards. It has been tested on the PC but not yet on the Mosaic (DONE on 20/03/01). ************************************************************************************* WORK 15/03/01 ------------- Modified DpuRec according to the new ICD. Not tested yet (DONE on 20/03/01) ************************************************************************************* WORK 16/03/01 ------------- Modified SimDpu according to the new ICD. ************************************************************************************* WORK 20/03/01 ------------- LABEL 0.20 The following projects have been labeled 0.20 once the new DPU - DEC/MEC ICD has been validated : - OnBoard - SimDpu - SimDec - ComExe - ComDll TESTS 20/03/01 -------------- Test of the new packet format between DPU - DEC/MEC (including test of CRC) (see 15/03/01). 1. Test of trigger commands : Sent few commands to modify the verbosity. Effects can be seen directly on host console. 2. Test of write commands and HkDiag packet: Write the HkList, start and stop the HkDiag acquisition and look if content is valid. The Write command allow us to test the CRC (see 14/03/01). Generation of bad packet format to test the error code generation (the following cases have been tested) : 2.1 Invalid MemID (TODO) 2.2 Invalid Parameter ID (DONE) 2.3 Invalid Length (TODO (this message is not yet generated in the application)) 2.4 Invalid Checksum (DONE) (modified SimDpu so it generates an invalid checksum) 2.5 Could not write in memory (TODO) 3. Test of load/dump/check command When we write the HkList through a write command, the address of the list is displayed on the host console. By this way, we can perform the same action through a load command and validate that the effect is the same. We load the new list (Load Command) and dump it right after (Dump Command). ************************************************************************************* WORK 02/04/01 ------------- Modified the SimDec so it togglees a Readout Flag (connected on FLAG0 of the DSP) each time it sends a readout. The ChopGrat ISR has also been modified to take into account this flag. Now, modifications of Chopper SetPoints are only taken into account at Readout Flag transitions. TESTS 02/04/01 -------------- Checked on the oscilloscope that the readout flag was toggled at the correct frequency. ************************************************************************************* TESTS 03/04/01 -------------- Tested a sequence at high frequency (readout = 100 Hz, Interrupt = 8KHz, HkDiag = 200 Hz). The diagnostic housekeeping shows that the sequence counter and the readout flag are modified unexpectedly. No explanation have been found today. ************************************************************************************* WORK 04/04/01 ------------- Modified the maximum number of loops to 5. It has not been tested yet (DONE on 05/04/01). Also modified the LOOP command to allow 'LOOP 0 TIMES'. TESTS 04/04/01 -------------- Connected the jtag and the ADSP 21020 Emulator to try to find the problem presented yesterday. Procedure to debug with the jtag : 1. Reset the mosaic board (the led shall be flashing). 2. Connect the JTAG. 3. Start the WICE Emulator with the mosaic.ach 4. Resume the execution of the program on the mosaic (press F4). 5. Load symbols (from onboard.exe). Do not load the .exe with the JTAG. 6. Upload application program through the Virtuoso Host Server. 7. Start the simulators. Note : 1. It's better to configure hardware setpoints before starting the application. 2. The whole system is not very stable. Don't hesitate to try 2 or 3 times the same thing. The JTAG debugging did not help find the problem. However, by adding more housekeeping, we could notice that the unexpected changes of the variables were always happening when a diagnostic housekeeping packet was sent. In fact, when Diagnostic Housekeeping is programmed at high frequency (200 Hz in this case), it happens that the DpuSender does not have enough time to send the buffer between two hk acquisition. So, the HkDiagnosticController starts rewritting at the beginning of the buffer before it is sent by the DpuSender. The problem has been solved by the use of two buffers (one being filled by the HkDiagnosticController and one being sent by DpuSender). ************************************************************************************** TESTS 05/04/01 -------------- Modified the default sequence with additional loops (tried with 0, 5 and 6 loops) (see 04/04/01) ************************************************************************************** WORK 09/04/01 -------------- Corrected various little problems with the diagnostic housekeeping. Also modified the SimDPU and HkViewer to take into account these modifications. ************************************************************************************** WORK 10/04/01 -------------- Corrected various little problems with the diagnostic housekeeping. Also modified the SimDPU and HkViewer to take into account these modifications. ************************************************************************************** WORK 11/04/01 -------------- Corrected various little problems with the diagnostic housekeeping. Also modified the SimDPU and HkViewer to take into account these modifications. Also added the SET SEQUENCE ID command; Modified the SimDpu accordingly. TESTS 11/04/01 -------------- Various tests for validating the diagnostic housekeeping (mainly by stepping in the SimDpu to validate the format). Validated that the SET SEQUENCE ID works (by looking at the nominal housekeeping). And that the Diagnostic housekeeping are sent each time that the sequence ID is modified (by looking at the Dpu Sender Packet Counter in the nominal housekeeping and by looking at the verbose information on the host console.) (see 27/02/01). ************************************************************************************** WORK 12/04/01 ------------- Modified the SimDec so it can receive packets (without interpreting them). Added a DecController to the onboard software and coded a few commands (validate, switch detector on, ...). Nothing has been validated yet since its just a first draft. ************************************************************************************** WORK 19/04/01 ------------- Modified the organisation of the Housekeeping tables to allow 16 blank spaces for further upgrade of HK capabilities during flight without patching. All the tables are now grouped in one. (TODO : to be documented in the user manual) ************************************************************************************** WORK 20/04/01 ------------- Created the PacketEncoder task (nothing really implemented until now). Also created the DetectorSimulator task so the onboard software can work without the SimDEC on the PC. The DetectorSimulator can, simulate at adjustable time interval, the reception of a readout coming from any of the detector (one at a time). This is done by raising the event used by the SpaceWire drivers to notify that data has been received. (DONE on 25/04/01 : to be documented in the user manual) TESTS 20/04/01 -------------- Tested the application without the SimDEC. We can check (with the verbose information) that a sequence is executing correctly. *************************************************************************************** WORK 23/04/01 ->27/04/01 ------------------------ Integrated the PacketEncoder and the DetectorSimulator. Created a SimSPU able to receive packets from the PacketEncoder. TESTS 27/04/01 -------------- Tested the application with a SimDPU and a SimSPU. The detector simulator is used in the onboard application to generate readouts at 250Hz. The CPU workload is measured (22%) but it seems that the SimSPU is not fast enough to receive all the packets from the PacketEncoder and half of the packets are lost. We have to find another solution to test the performance of the onboard application (TODO). *************************************************************************************** WORK 02/05/01 -> 04/05/01 ------------------------- Continued work from last week. Started to test the new version. When we use the DetectorSimulator (no SimDEC), the application crashes one or two seconds after we have started the DetectorSimulator. The verbose information is not sufficient to identify the origin of the problem. When we don't use the DetectorSimulator (but we use the SimDEC), the application crashes when we execute the default sequence. It crashes in the 'Validate Blue Science Data' command. Going further into detail, we have identified that the application crashes when the Sequencer makes a KS_FifoPut(BLUE_DEC_FIFO, &msg). Going further into detail again, we have identified that the application crashes when the DecController task is awaken by any mean (KS_FifoGetW() or KS_TaskSleep()) once the spacewire link has been initialized (KS_RawLinkCOMITest() returns RC_OK). Finaly, since we could not find the origin of the problem, we came back to the previous version and coded the change one after each other with more testing than the first time. **************************************************************************************** WORK 07/05/01 -> 11/05/01 ------------------------- Came back to previous version and finaly found the bug in a memcpy command (source being larger than destination !). Corrected it and revalidated. Added the Label command and tested the sequence provided by Helmut in the "TC and HK List". Also validated that the housekeeping diagnostic can be synchronized with readouts. ***************************************************************************************** ONE YEAR OF WORK WITHOUT LOGBOOK !!!! ***************************************************************************************** 26/07/02 -------- Workload measures ----------------- Test 1 : 1 channel in spectroscopy ------ No Red channel. BOLC sending data. Blue DEC sending data. Dummy SPU able to receive 256 packets a second. Start-up (during 4 sec) : 42% - 47% Steady execution : 41.5% Once we stop the BOLC : 31.5% Once we stop the SPU : 26.0% Once we stop the DEC Sim : 15.5% Once we enable all PID Ctrl : 18.5% => around 60% with complete software (2 channels) Test 2 : photometry ------ No Red channel. BOLC sending data. Blue DEC sending data. Dummy SPU able to receive the photometry packets a correct speed. Steady execution : 38.0% Once we stop the SPU : 35.0% Once we stop the BOLC : 24.0% Once we stop the DEC Sim : 15.5% Conclusion : Blue DEC Receiver : 9.5% Blue Packet Encoder (spec) : 5.5% Blue Packet Encoder (phot) : 3.0% BOL Receiver :10.5% PID Controllers : 3.0% Memory size : Data : 15 KW Program : 22KW 10/10/02 ******** Start working on the crisa board again. To avoid to forget to include __asm("CALL ___lib_setup_environment;"); in the main() (in NODE1.c) each time NODE1.c is regenerated, I tried to call it from root_netload() that is the first function that is called in the main. However, for some unknown reason (maybe lib_setup_environment initializes the stack ...), the OBS does not start properly if we call it there. So, go back to previous solution and don't forget to include it !!!! Put some test code in InitializeSequencer() : to start sending HK-like packets. It works. Now, try to trigger the sending by a timer. It works 14/10/02 ******** Leave the 1355 initialization code in the Sequencer code. Moved the dummy HK packet sending inside HK code. Put HK_DIAG in sleep. Seq makes KS_TaskGroupStart(GROUP_HK) Result : 1st attempt : HK has started (seen in wice020). The HK task is waiting for the link connection (link is running but no NUL is received). 2nd attempt : start-up procedure : chip reset in wice020, wait 15 sec. start comexe start simdpu upload software start ASW (address = 8032) quit simdpu quit comexe press F4 in wice (halt) start LinkReceiver press F4 in wice020 (run) The connection is established (seen on PC) Packets are received 3rd attempt : same procedure as 2nd attemp except that I don't press F4 (halt/run) on the wice020. Failed : no connection 4th attempt : same as 3rd with increased time before SEQ initializes the link. It works ! (Is it again this master/slave switch-on order ?) 5th attemps : same as 4th. FAILED ! 6th : SUCCESS ! 7 : SUCCESS ! 8 : SUCCESS ! 9 : SUCCESS ! 10 : SUCCESS ! Testing the reception of packets : -------------------------------- Modify the SEQ such that after the initialization of the link, it waits for the simdpu to connect and wait for a packet to be received before starting the HK task. Procedure : chip reset in wice020, wait 15 sec. start comexe start simdpu (for SUSW) upload software start ASW (address = 8032) quit simdpu quit comexe start comexe start simdpu (for ASW) send any trigger command to DMC check in simdpu directory that hk.dat is created and increases its size 1st attempt : SUCCESS ! Involving DPU_SENDER task : ------------------------- Modify the SEQ such that it does not wait anymore for a command from DPU. It starts HK and GROUP_DPU. HK has been modified such that it writes a msg in DPU_SEND_FIFO every second. DPU_SEND should send a packet (with undefined data inside) 1st attempt : NO connection (all tasks are waiting for a connection). 2nd attempt : Connection OK. Packets are received. Back to nominal HK task ----------------------- Use the same HK task as OBS for AVM. HK Timer is started by SEQ in the init code. SUCCESS !!! HK is received by the DPUSim. Format is correct. Out of the HK, we can see that DPU_Receiver claims that he has received an unknown message type (although he should not have received anything). TO INVESTIGATE. The CPU workload functions seem to work Back to almost-nominal Sequencer -------------------------------- In these tests, we are going to uncomment code to go back as close as possible of the DMC OBS for AVM. Since IRQ3 seems not to work perfectly, we will not install the CHOP_GRAT ISR. Only the 5 basic tasks will be started (DPU_SEND, DPU_REC, HK, HK_DIAG, SEQ) Only the link to DPU will be started Note : from now, I will not document the tests failing because of a connection problem. SUCCESSFUL tests 17/10/02 ******** JMG made some modification to the #define in order to run the ISR on the crisa board. It works. Note : a lot of defines don't have their final values because of misunderstanding between JMG and NM. It will be corrected soon. The HK was not received at correct frequency (we could see 18200 ISR calls between two packets). This is due to the change of processor frequency (from 20Mhz to 18Mhz). The .nli file has been updated. No change is observed. Exit Virtuoso Project Manager and open the project again (and compile). It works !!!! The SwitchOnRedDEC and BlueDEC have been tested (and OFF), they work ! 18/10/02 ******** Re-enabled the Blue channel tasks and the detector simulators. Connected a LinkReceiver on the Blue SPU. Received the data from the detector simulator. Everything is fine. Did the same for Red channel (re-enabled SMCS2). The LinkReceiver receives only one packet.The smcs2_isr is not called at the end of the transmission of the first packet (put a breakpoint in wice020). Enabled the IRQ in the DMPSC registers : //SMCS2 interrupt must be level sensitive *((int*)K_DMADD_DMPSC_GENCONFSTT) = 0x00000801; //enable SMCS2 interrupt *((int*)K_DMADD_DMPSC_INTMASK) = 0x00003FF7; The ISR is entered after the first packet is sent. But it is not called anymore afterwards. Note : While stepping in the code in IRQ3 interrupt vector, the DEC sim has been powered ON then OFF !!!! 21/10/02 ******** Changed the links affectation. DPU is now using link 4 (1st link of SMCS2). It is connected to LinkReceiver. LinkReceiver receives 2 packets and then, nothing ... After stopping the execution, a look in SMCS2_ISR doesn't show any interrupt pending (so, it's logical that DMPSC does not forward it to IRQ1). Step in the code : When sending the first packet, right after the packet has been transmitted, I can see IRQ1 bit set to one in IRPTL. SMCS2_ISR contains 0x00000010 !!!! (this means EOPx has been received !!!!). New debug session, simply put a breakpoint in smcs2_isr. The first time it is called, SMCS2_ISR is 0x000000c4 (this includes 'data transmitted'). New debug session : put a breakpoint in DSMCS_TransmitBuffer. we can see SMCS2_ISR going to 0x000000d4 (this includes 'data transmitted'). But, the software never stops again on this breakpoint !!!! New debug session (the same as above) : we can see SMCS2_ISR going to 0x000000d4. In the ISR, SMCS2_ISR is copied in r0. After a few instructions, r0 is 0x00000010 !! (I don't know when its value has changed) r0 is reset right after it is set !!!!!! (does DMPSC reads it ?) should I reset the interrupt in INTRST of DMPSC ? 23/10/02 ******** Added a reset of external interrupt 0 in INTRST of DMPSC. The LinkReceiver is now receiving all the packets. I think SMCS2_ISR was reset by the emulator itself and not by the DMPSC. Updated the testSpacewire (linkReceiver). It now has a mode where it checks that all the value of the science packet are identical. If any of these value is different from the other, it stores the packet on disk (wrong_packetXXX.dat). 25/10/02 ******** ReModified smcs.s and SmcsDrv task to signal the disconnect_parity error in the task status. Since yesterday evening, there are some connections problems between DPU and DMC. The connection succeeds but not packet are transmitted/received. Put the DMC as slave and DPU in master (the same config as for the SUSW). Compare the content of SMCS registers : - when SUSW is not connected yet with DPU (DPU is not started) - when ASW is not connected yet with DPU (DPU is not started) - when SUSW is waiting for packets from DPU (DPU is started, connection is done, no packets have been sent) - when ASW is waiting for packets from DPU (DPU is started, connection is done, no packets have been sent) SUSW_NOT_CONN ASW_NOT_CONN SUSW_CONN ASW_CONN SMCS Registers : -------------- 0x00 SICR 0x00000002 0x00000002 0x00000002 0x00000002 0x01 TRS_CTRL 0x00000010 0x00000010 0x00000010 0x00000010 0x02 RT_CTRL 0x00000000 0x00000000 0x00000000 0x00000000 0x03 ------- 0x04 ISR_0 0x00000000 0x00000000 0x00000000 0x00000000 0x05 ISR_1 0x00000000 0x00000000 0x00000000 0x00000000 0x06 ISR_2 0x00000000 0x00000000 0x00000000 0x00000000 0x07 ISR_3 0x00000000 0x00000000 0x00000000 0x00000000 0x08 IMR_0 0x0000001D 0x01D0741D 0x0000001D 0x01D0741D 0x09 IMR_1 0x0000001D 0x01D0741D 0x0000001D 0x01D0741D 0x0A IMR_2 0x0000001D 0x01D0741D 0x0000001D 0x01D0741D 0x0B IMR_3 0x0000001D 0x01D0741D 0x0000001D 0x01D0741D 0x0C COMI_CS0R 0x00000020 0x00000020 0x00000020 0x00000020 0x0D --------- 0x0E COMI_ACR 0x00000008 0x00000008 0x00000008 0x00000008 0x0F PRCIR 0x00000000 0x00000000 0x00000000 0x00000000 Link 1 registers : ---------------- 0x10 CH1_DSM_MODR 0x0000000C 0x00000000 0x0000000C 0x00000000 0x11 CH1_DSM_CMDR 0x00000000 0x00000004 0x00000008 0x0000000C 0x12 CH1_DSM_STAR 0x00000000 0x00000001 0x00000019 0x00000009 0x13 CH1_DSM_TSTR 0x00000000 0x00000000 0x00000000 0x00000000 0x14 CH1_ADDR 0x00000000 0x00000000 0x00000000 0x00000000 0x15 CH1_RT_ADDR 0x00000000 0x00000000 0x00000000 0x00000000 0x16 CH1_PR_STAR 0x00000004 0x00000004 0x00000004 0x00000004 0x17 ----------- 0x18 CH1_CNTRL1 0x00000000 0x00000000 0x00000000 0x00000000 0x19 CH1_CNTRL2 0x00000000 0x00000000 0x00000000 0x00000000 0x1A CH1_HTID 0x00000000 0x00000000 0x00000000 0x00000000 0x1B CH1_HCNTRL 0x00000000 0x00000000 0x00000000 0x00000000 0x1C CH1_ESR1 0x00000000 0x00000000 0x00000000 0x00000000 0x1D CH1_ESR2 0x00000000 0x00000000 0x00000000 0x00000000 0x1E -------- 0x1F CH1_COMICFG 0x00000033 0x00000033 0x00000033 0x00000033 0x20 CH1_TX_SAR 0x00000000 0x00000000 0x00000000 0x00000000 0x21 CH1_TX_SAR 0x00000000 0x00000000 0x00000000 0x00000000 0x22 CH1_TX_EAR 0x00000000 0x00000000 0x00000000 0x00000000 0x23 CH1_TX_EAR 0x00000000 0x00000000 0x00000000 0x00000000 0x24 CH1_TX_CAR 0x00000000 0x00000000 0x00000000 0x00000000 0x25 CH1_TX_CAR 0x00000000 0x00000000 0x00000000 0x00000000 0x26 CH1_TX_FIFO 0x00000000 0x00000000 0x00000000 0x00000000 0x27 CH1_TX_EOPB 0x00000000 0x00000000 0x00000000 0x00000000 0x28 CH1_RX_SAR 0x00000000 0x00000000 0x00000000 0x00000000 0x29 CH1_RX_SAR 0x00000000 0x00000000 0x00000000 0x00000000 0x2A CH1_RX_EAR 0x00000000 0x00000000 0x00000FFF 0x00000000 0x2B CH1_RX_EAR 0x00000000 0x00000000 0x00000FFF 0x00000000 0x2C CH1_RX_CAR 0x00000000 0x00000000 0x00000000 0x00000000 0x2D CH1_RX_CAR 0x00000000 0x00000000 0x00000000 0x00000000 0x2E CH1_RX_FIFO 0x00660002 0x00660002 0x00660002 0x00660002 0x2F CH1_STAR 0x00000001 0x00000005 0x00000005 0x00000005 Differences between SUSW_NOT_CONN and ASW_NOT_CONN : -------------------------------------------------- DSM_MODR is different : There are 2 ways to configure the link speed at 10 Mbps. DSM_CMDR is different : ASW configures the link to autoreconnect after disconnect error DSM_STAR is different : ASW already detects that the link is connected although the master (SimDPU has not yet been started) START is different : ASW receive fifo is not empty !!!!! Conclusions : Does the SUSW resets the link when it exits ? The reset of the SMCS should reset all the registers and clean the fifo. How can the SMCS have received data and have a connection ? Maybe the auto-reconnect detects that there is no connection and thus starts sending NULL token to try to recconect. Differences between SUSW_CONN and SUSW_CONN : ------------------------------------------- Note : In both cases, bit 3 is set in DSM_CMDR, it is supposed to be always '0'. A check in the code shows that nobody writes in it (at least in the ASW, we should check with CRISA for the SUSW). Obviously, the ASW did not even received an FCT (difference in DSM_STAR), so he is not ready to receive the data (RX_EAR). Made the same test without configuring the ASW to reconnect after disconnect error. ASW_NOT_CONN ASW_CONN 0x10 CH1_DSM_MODR 0x00000000 0x00000000 0x11 CH1_DSM_CMDR 0x00000000 0x00000008 0x12 CH1_DSM_STAR 0x00000000 0x00000009 0x13 CH1_DSM_TSTR 0x00000000 0x00000000 0x14 CH1_ADDR 0x00000000 0x00000000 0x15 CH1_RT_ADDR 0x00000000 0x00000000 0x16 CH1_PR_STAR 0x00000004 0x00000004 ... 0x2E CH1_RX_FIFO 0x00040000 0x00040000 0x2F CH1_STAR 0x00000005 0x00000005 The ASW did not receive any FCT ! New test : pause the execution after the SMCS_ChipReset : ASW_AFTER_RESET SMCS Registers : -------------- 0x00 SICR 0x00000000 0x01 TRS_CTRL 0x0000000A 0x02 RT_CTRL 0x00000000 0x03 ------- 0x04 ISR_0 0x00000000 0x05 ISR_1 0x00000000 0x06 ISR_2 0x00000000 0x07 ISR_3 0x00000000 0x08 IMR_0 0x00000000 0x09 IMR_1 0x00000000 0x0A IMR_2 0x00000000 0x0B IMR_3 0x00000000 0x0C COMI_CS0R 0x000000FF 0x0D --------- 0x0E COMI_ACR 0x00000008 0x0F PRCIR 0x00000000 Link 1 registers : ---------------- 0x10 CH1_DSM_MODR 0x00000003 0x11 CH1_DSM_CMDR 0x00000000 0x12 CH1_DSM_STAR 0x00000000 0x13 CH1_DSM_TSTR 0x00000000 0x14 CH1_ADDR 0x00000000 0x15 CH1_RT_ADDR 0x00000000 0x16 CH1_PR_STAR 0x00000004 -> should be 0 after reset 0x17 ----------- 0x18 CH1_CNTRL1 0x00000000 0x19 CH1_CNTRL2 0x00000000 0x1A CH1_HTID 0x00000000 0x1B CH1_HCNTRL 0x00000000 0x1C CH1_ESR1 0x00000000 0x1D CH1_ESR2 0x00000000 0x1E -------- 0x1F CH1_COMICFG 0x00000033 -> should be 0 after reset 0x20 CH1_TX_SAR 0x00000000 0x21 CH1_TX_SAR 0x00000000 0x22 CH1_TX_EAR 0x00000000 0x23 CH1_TX_EAR 0x00000000 0x24 CH1_TX_CAR 0x00000000 0x25 CH1_TX_CAR 0x00000000 0x26 CH1_TX_FIFO 0x00000000 0x27 CH1_TX_EOPB 0x00000000 0x28 CH1_RX_SAR 0x00000000 0x29 CH1_RX_SAR 0x00000000 0x2A CH1_RX_EAR 0x00000000 0x2B CH1_RX_EAR 0x00000000 0x2C CH1_RX_CAR 0x00000000 0x2D CH1_RX_CAR 0x00000000 0x2E CH1_RX_FIFO 0x0000008E -> is undefined after reset 0x2F CH1_STAR 0x00000005 -> should be 0 after reset It seems that the OBS is blocked by a LOCK_RES_PARAM before being able to initiate the communication with the SimDPU (before the StartReadingOnLink). BlueDecController was locking the resource (after a modification on 23/10/02). It is very strange that is has worked !!!! OK. It has worked because, on the 23/10/02, for unknown reason, the Blue DEC was staying powered on at startup and so, the code that was locking the resource was releasing it immediately ! 28/10/02 ******** After the correction of the BlueDecController, the connection with DPU can be established (we still have DPU as master and DMC as slave). Sending the SWITCH_ON_BLUE_DEC fails ! Testing the deconnexion : Get back to nominal master/slave configuration. Connect a simDPU and a SimSPU to DMC. Once they are all connected, close the SimSPU. Looking into the HK shows that the deconnexion has been detected (an error is signaled). Modified the SmcsDrv task such that, when it sees a deconnexion with the Blue SPU, it aborts the BluePacketEncoder task and restart it right after. Also modified the initialization of the link in this task : trying to start the link every second (as a master) to make sure that master starts after the slave. Unfortunately, it does not work every time ! With the same configuration as above, the connexion with Blue SPU has failed ! (while connexion with SimDPU could be established). 31/10/02 ******** Solved the SWITCH_ON_BLUE_DEC that was not working anymore. That was due to new compilation flags added recently (MODEL_EM and MODEL_AVM). I forgot to add them to the ASM_CFLAGS (assembly compilation flags). So the chopGrat isr was writing at a wrong address. However, SWITCH_ON_RED_DEC still doesn't work ! In the current version of the MIM FPGA, RED_DEC_ON is 0x00002000. However, in the interrupt (ChopGrat), the value of the controlWord is stored in gHC, then, bit 13 is set to one (at the beginning of the ISR) and the value is copied at 0x20040045. At the end of the ISR, bit 13 is cleared in gHC and then gHC is copied at 0x20040045. It means that whatever we write at the beginning of the ISR, the bit 13 is cleared at the end and the RED_DEC is powered off. Testing connection between DMC and DEC : 1st test : ---------- when switching on the blue DEC, the connection has succeeded. But, when looking at the HK, a deconnection has been observed : value of DMC_DEC_BLUE_CTRL_STATUS : 90000 -> 1B0204 -> 110000. Note : there was a retry in the connection : do { UNLOCK_RES_PARAMS; DSMCS_StopChannel(CHANNEL_BLUE_DEC); KS_TaskSleep(100); DSMCS_StartChannelAsMaster(CHANNEL_BLUE_DEC); KS_TaskSleep(1000); VERL(VER_DEC_CTRL + VER_LINK, 5, "Blue DEC CTRL : Waiting for Link1355 to initialize"); LOCK_RES_PARAMS; } while ((Link1355Test(CHANNEL_BLUE_DEC)!= RC_OK) && IS(gpBlueDecRec->TaskStatus, SIMULATION, REAL_TIMING + REAL_READOUTS)); After 153 HK packets, the connection between DPU-DMC is lost ! Looking into the HK buffer in DMC memory with WICE, we can see that the deconnexion has been observed ! 2nd test : -------- value of DMC_DEC_BLUE_CTRL_STATUS : 90000 -> 1B0204 -> 110000. Only 24 HK packets have been received but no deconnection has been detected. Later, the DEC shows FIFO_FULL but the connection DMC-DEC is still OK. Are some FCT lost somewhere ? 3rd test : --------- value of DMC_DEC_BLUE_CTRL_STATUS : 90000 -> 110000. After 41 HK packets, DEC is FIFO_FULL; no deconnection observed. After 42 Hk packets, SimDPU does not receive packets anymore. 4th test : --------- value of DMC_DEC_BLUE_CTRL_STATUS : 90000 -> 1B0204 -> 110000. Everything was working fine (126 HK packets). Then, I switched on the LeCroy. At the same time, DPU-DMC has lost connection and DEC FIFO went full (no deconnection). !!!!!! Disconnected the probes of the LeCroy. 5th test : --------- Same !!!! 6th test : --------- After the chip reset on the CRISA board, exit WICE and disconnect JTAG. Same result as 5th and 4th !!!! 06/11/02 ******** Added another compilation flag for the assembly. BUT, This line in baseMosaic is too long : ASFLAGS = -DADI21020 -D__ADSP21020__ -DCRISA_BOARD -DCRISA_DRIVERS -DNO_SMCS2 -DMODEL_EM -DNEW_MIM_FPGA_V1 Actually, NEW_MIM_FPGA_V1 can be replaced by MODEL_EM right now. 08/11/02 ******** Upgraded the code to start the link with DMC only when it is powered on and stop the link when switched off. In a first time, I did start the link a few hundred cycles after the switch-on. With this configuration, I had 2 connections and transmissions out of 6 attempts. Then, I introduced a delay between these 2 operations (1ms + hundreds of cycles). I can't get any connection ! Actually, the high level task don't have time to realize that there is a connection. Immediately after the startLink, there is a disconnect/parity error raised. Tried to configure the link to auto-reconnect after disconnect/parity error (note : this is done after the link is started). Then, we detect arount 1000 deconnection a second !!! Also tried to increase the delay between switch-on and startLink (50ms and 500ms); it doesn't help. I have written a new utility app called CustomBuildForCrisa that adds the __asm("CALL ___lib_setup_environment;"); to the NODE1.c. It is working fine but I still have some problem to include it at the correct location in the makefile. 16/12/02 ******** Values of SMCS registers after start of ASW, HK sent to DPU. And, values after the light has been switched on/off (so loosing the connection) ASW_RUNNING ASW_CONNECTION_LOST SMCS Registers : -------------- 0x00 SICR 0x00000002 0x00000002 0x01 TRS_CTRL 0x00000010 0x00000010 0x02 RT_CTRL 0x00000000 0x00000000 0x03 ------- 0x04 ISR_0 0x00000040 0x00000080 0x05 ISR_1 0x00000000 0x00000000 0x06 ISR_2 0x00000000 0x00000000 0x07 ISR_3 0x00000000 0x00000000 0x08 IMR_0 0x01d0741d 0x01d0741d 0x09 IMR_1 0x01d0741d 0x01d0741d 0x0A IMR_2 0x01d0741d 0x01d0741d 0x0B IMR_3 0x01d0741d 0x01d0741d 0x0C COMI_CS0R 0x00000020 0x00000020 0x0D --------- 0x0E COMI_ACR 0x00000008 0x00000008 0x0F PRCIR 0x00000000 0x00000000 Link 1 registers : ---------------- 0x10 CH1_DSM_MODR 0x00000000 0x00000000 0x11 CH1_DSM_CMDR 0x00000008 0x00000008 0x12 CH1_DSM_STAR 0x00000019 0x00000016 0x13 CH1_DSM_TSTR 0x00000000 0x00000000 0x14 CH1_ADDR 0x00000000 0x00000000 0x15 CH1_RT_ADDR 0x00000000 0x00000000 0x16 CH1_PR_STAR 0x00000004 0x00000004 0x17 ----------- 0x18 CH1_CNTRL1 0x00000000 0x00000000 0x19 CH1_CNTRL2 0x00000000 0x00000000 0x1A CH1_HTID 0x00000000 0x00000000 0x1B CH1_HCNTRL 0x00000000 0x00000000 0x1C CH1_ESR1 0x00000000 0x00000000 0x1D CH1_ESR2 0x00000000 0x00000000 0x1E -------- 0x1F CH1_COMICFG 0x00000033 0x00000033 0x20 CH1_TX_SAR 0x00000000 0x00000000 0x21 CH1_TX_SAR 0x00000000 0x00000000 0x22 CH1_TX_EAR 0x000001e2 0x000001e2 0x23 CH1_TX_EAR 0x000001e2 0x000001e2 0x24 CH1_TX_CAR 0x000001e3 0x0000000b 0x25 CH1_TX_CAR 0x000001e3 0x0000000b 0x26 CH1_TX_FIFO 0x00000000 0x00000000 0x27 CH1_TX_EOPB 0x00000000 0x00000000 0x28 CH1_RX_SAR 0x00000200 0x00000200 0x29 CH1_RX_SAR 0x00000200 0x00000200 0x2A CH1_RX_EAR 0x000003ff 0x000003ff 0x2B CH1_RX_EAR 0x000003ff 0x000003ff 0x2C CH1_RX_CAR 0x00000200 0x00000200 0x2D CH1_RX_CAR 0x00000200 0x00000200 0x2E CH1_RX_FIFO 0x00040000 0x00040000 0x2F CH1_STAR 0x00000001 0x00000002 TESTS : connection between DMC and DEC : -------------------------------------- Config 1 : DMC as master, no autoreconnect. There is a connection but a disconnection right after the SWITCH_ON_BLUE_DEC LEDS : RED GREEN OFF OFF Config 2 : DMC as master, autoreconnect (done after the StartChannelAsMaster) There is a connection but a disconnection right after the SWITCH_ON_BLUE_DEC LEDS : RED GREEN OFF OFF Config 3 : DMC as master, autoreconnect (done before the StartChannelAsMaster) There is a connection but a disconnection right after the SWITCH_ON_BLUE_DEC LEDS : RED RED OFF OFF Config 4 : DMC as slave (first start channel and then switch on DEC) no autoreconnect There is no connection LEDS : RED GREEN OFF OFF Config 5 : DMC as slave (first start channel and then switch on DEC) autoreconnect (done after the StartChannelAsSlave) There is a connection but a disconnection right after the SWITCH_ON_BLUE_DEC LEDS : RED RED OFF OFF Config 6 : DMC as slave (first start channel and then switch on DEC) autoreconnect (done before the StartChannelAsSlave) There is a connection but a huge amount of disconnection right after (the CPU workload jumps to 80 % to handle the interrupts !) LEDS : RED GREEN OFF GREEN Second test on same config : same result as config 5 !!! 19/12/02 ******** Test connection DMC - DEC. Questions for THE expert : If the SMCS is not configured to autoreconnect; after the first disconnect has been observed, stoping and restarting the link does not help. We need to reset the chip to be able to have a new connection. (Needs further testing). 24/01/03 ******** The version number is now readable by low-level software (const int). CustomBuildForCrisa is now called when making a 'rebuild all'; not when you make 'build' after 'generate files'. 05/02/03 ******** Tested the connection between DEC and DMC, config is : DPUSim SUSW connected on link1 of SMCS1 Blue DEC connected on link3 of SMCS1 DPUSIM ASW connected on link1 of SMCS2 After switch-on of DEC, its SMCS is reset; connection and transmission are both OK. Modified the 1355 drivers to make them more user-friendly for further testing. Now, try with config : both DPUSim on link1 of SMCS1 Blue DEC on link3 of SMCS2 connection and transmission are both OK. Update the code for Red DEC such that it gets the same as the one for blue DEC. 06/02/03 ******** Added HK_PROTO_BOARD compilation flag (defined in mim.h (the compilation command line is too long !)). Made a cosmetic correction to SequenceWriter. Temporarily enabled SWITCH_ON/OFF of Red DEC inside a sequence to test it many times. This mean : - changed their execution mode in com_list.h - changed their return value (CommandSwitchOn....) - increased the timeOut the sequencer is waiting for the synchro (since the communication is reset, no packets are received during 'some' time (max 1 sec to get the connection again + TBD time where the connection is lost); put the timeOut to 1.5 sec After 300 on/off, the blue DEC does connect but no readout are transmitted. => timeout and the execution stops. Made a second test, same result. I should make sure that the read always restart after a re-connection. 07/02/03 ******** Situation when the sequence stops after a switch-on : - RED DEC is connected and transmiting - BLUE DEC is connected but not transmiting (its gLinkStatus is MASTER) So, it seems that the StartReadingOnLink in SmcsDrv.C fails. Does it happen at the same time as the StartReadingOnLink in Link1355Read in BDecRec.c ? What happens : - a packet arrives - BDecRec receives the event, copy the buffer, starts another read operation. - BDecRec gets out of Link1355Read and send the synchro to the sequencer - sequencer executes 'switch-on the Red DEC' - RDecCtrl switch-on the Red DEC and then, wait 50ms. - during this time, the BDecRec receives many packets and can be anywhere in its code when ... - RDecCtrl reset the SMCS chip - Interrupts are raised - SMCSDRV test connection with Blue DEC; it is ok - SMCSDRV start a new read operation on Blue DEC link. It might happen while BDecRec is doing the same (or anything else). To Investigate : - BDecRec could ask the SMCSDRV to start a new read (signal its SEMA and set a bit somewhere). - Eventually, we could make sure that we don't start a read operation if another read operation is running and if the SMCS has not been reset in the meanwhile. 14/02/03 ******** Made a very long test. After 2800 on/off, the sequence stopped. Blue DEC Link Status Red DEC Link Status 28 8 28 0 28 0 28 8 28 8 28 0 28 0 38 <- seq stopped 8 38 8 38 8 38 8 38 8 28 : reading from Link1355Read 38 : reading from SMCS DRV We should really investigate : - BDecRec could ask the SMCSDRV to start a new read (signal its SEMA and set a bit somewhere). 11/03/03 ******** Investigating a bug found by MG : When starting Diag HK (measure 244 and 245), then chopper on then enable chopper pid, the setpoint has strange values (-8000, 17000, ...) First, stored the gpChopperController->CurrentPosition in an additionnal HK values (gCustomHK) and noticed that the value was not correct during the execution of EnableChopperPID. Then, changed the content of HkDiagList (241 and 242). The problem still happens. Then, in a test run : first had the problem while DiagHk was running. Stopped it. Chopper ON, enable PID and also got the problem although DiagHk was not running. Started new test run without any DiagHk. The problem also happened ! 12/03/03 ******** The value of r0 is not changed during the execution of the interrupt routine. The interrupt routine is the only code that writes in gpChopperController->CurrentPosition. 24/03/03 ******** Upgraded SMCS Drv such that it centralizes all access to the SMCS; especially to make sure that there are never two tasks (SMCS DRV + a DEC REC) calling StartReadingOnLink in the same time (or while a packet is being received). Note : this has increased the CPU workload dramatically (we reach almost 50% when both DEC REC are active and no data are transmitted to SPU). Started a new long test (sequence that switches ON/OFF the RED DEC every 4 secs). After 2 hours, the Blue DEC did connect but no transmission occured. After a switch OFF and ON, everything went fine again. 25/03/03 ******** Made the changes for the new BOLC hk entries: There are now 224 16 bits entries (there were 196 before). The 28 new entries must be added at the end of the HK packet. 31/03/03 ******** Compiled a new AVM OBS (version 5b) (I did forgot to include the auto-reconnect of 1355 links in the version 5). On the EM OBS, made few changes such that JMG can try his new servo software : - defined a segment seg_circ to store the circular buffers in PM (from 0x07ee00 to 0x07eeff). - declared buffer1 and buffer2 in seg_circ (in chopgrat.s) - moved SW, CW, TG and TK before the writable parameters in the pid structure. - added Coef1 and Coef2 arrays in pid structure (pid_ctrl.h) - in "par_list.h", changed the size of writable parameters in the pid structure from 9 to 20 (Note that the old Kp, Ki, ... parameters can not be written anymore). >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> because the new definition is PARAM( DEC_WRITE_GRATING_CTRL_PARAMETERS , gParameters.GratingController.Coef1 , 20 ) but if it was PARAM( DEC_WRITE_GRATING_CTRL_PARAMETERS , gParameters.GratingController.Kp , 29 ) ? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jean-Marie, -Pour initialiser Coef1 et Coef2, tu peux ajouter du code dans InitializePidControllerParams (dans pid_ctrl.c) (pour initialiser tous les pid avec les mêmes paramètres) ou dans Seq.c après l'appel à InitializePidControllerParams (pour mettre des valeurs particulières à chaque pid). -Dans Seq.c, tu peux initialiser les registres i en indiquant directement l'adresse du segment (je n'ai pas réussi à faire de lien vers les variables buffer1 et buffer2). exemple : i14 = 0x07ee00; >>>>>>>>>>>>>>> j'essaierai dans -J'ai modifié le simulateur DPU pour que tu puisses écrires les paramètres en float (je n'ai pas testé et j'ai considéré que les float avaient la même représentation sur le PC et sur le DMC). -Bonne chance. Je suis contactable à Garching par e-mail ou par GSM : 0494/316727. 01/04/03 (At Garching) : ********************** Tested the last version of the OBS (version 5b) on the AVM (Mosaic board) with a SimDPU running on a PC. First, take care to the switch-on procedure which is different with Mosaic and Crisa : To connect a SimDPU to the Mosaic board : - reset mosaic. - start comexe - start SimDPU - reset mosaic (it seems that it is not mandatory but ...) - upload software with Virtuoso IDE Note : Virtuoso is connected to BOLC Link (no cable switching after switch-on) and SimDPU is of course connected to DPU Link. I had some troubles when siwtching-on/off DEC : On the AVM, DECs are not really switched-off so, when a switch-off command is received, the StopLink function is called but it seems that some NULL tokens are still transmitted from DMC to DEC (green light 'connect' still green). So, when trying to reconnect after a switch-on-dec command, the link is not always initialized correctly (connection is lost at that time !). Changes : When a DEC is switched-off, its SMCS chip is reset to make sure the connection is lost. (Note from 02/04/03 : this is not an ultimate solution) 02/04/03 (At Garching) : ********************** Tested the chip reset when observing a disconnection on the DPU link. After long investigation, we discover that DMC OBS crashes ! This is caused by hundreds of disconnect error interrupts on link 2 (connected to Red SPU) ! The software is not doing anything else but handling these interrupts and this results in a crash (which is not normal). Actually, this link is not even started but the SMCS generates disconnect interrupts anyway (note : these interrupts are generated right after (or during ?) the SMCS chip reset). Solved by enabling the interrupts only once we have received NUL tokens on a link. (CHANGE MUST BE PERFORMED ON THE CURRENT VERSION AT CSL) After a discussion with Nabil and Helmut, we agreed that having the DMC as a slave on the links to DPU and SPU is one of the best solution to have successful chip resets and switch-on philosophy. Modified the OBS to make it start the connection to DPU and SPU as a slave. Made some test with the SimDPU. The first connection is always successful but the connection after a chip reset fails too much often. On DMC side, most of the time, everything seems to be fine : link is connected, NULL are transmitted and NULL have been received but, the SimDPU just seems not to see that the link is connected. But, it seems that sometimes, only the read operation is initiated but not the write so there should be some kind of problems in the high level handling of the SMCS (no idea right now). We made a code comparison with Nabil (whose SPU OBS is handling correctly the connection/disconnection/chip reset with DPU). Nabil's solution is much simpler that mine (only two links to handle). Everything is grouped in one only task. He first waits to have a connection with DPU and then try to connect to DMC. Once he has a connection, he starts the tasks which uses this connection. Once he looses the connection, he kills these tasks. When comparing what is done at low-level (writes in SMCS registers, ....) we don't find any significant differences (some operations where not performed in the same order). I tried to perform them in the same order as he does but this did not improve the reliability of the connection. 03/04/03 (At Garching) : ********************** Tried to re-organize my drivers to have a high level organization more similar to Nabil's. First removed many parts of the OBS to keep only : the HK, SEQ, DPU and SMCS_DRV. SEQ starts first, after the 15 sec, it starts SMCS_DRV which tries to connect to DPU only. When connection occurs, start the DPU_TASK_GROUP and the HK_TASK_GROUP. I had many problems in a first time because of the AbortTasks when we loose the connection. Some of the tasks were aborted between a LOCK and UNLOCK operation such that they kept the access to the resource although they were not running anymore. To solve this problem, I installed AbortHandler functions which frees the resources locked and created by the task. I thought it was the ultimate solution but since I have added these handlers, I observe a nearly random behavior of the OBS : some KS_TaskGroupStart seems not to have any effect, the Sequencer KS_FIFOGetWT never reaches its time-out (30 ms) ! This is the last problem I had and I could really not find any work-around. It is obvious that re-organizing the high level drivers is a good solution. We might wonder wether starting/aborting tasks dynamically is a good solution in a real-time application. Anyway, I need to perform many tests on the CRISA board with a very simple application to validate everything. Note that until now, everything seemed to work fine on the Crisa board but I should really try to connect more simulators together. Helmut pointed out that the trigger command 24 (SetBothSpectroParamArray) was failing sometimes in the past. His first attempt fails directly. And, all my attempts also failed. Then, I found out that, when performing the Master Reset of DECs, the 1355 connection was definitely lost. The problem was coming from the 'auto-reconnect' option that was missing in the software (Actually, the source code here in Garching was version 4 while the executable was version 4b (auto-reconnect was the only difference between these two versions). Note that the auto-reconnect is not the ultimate solution and can not guarantee that the connection will never be lost. It seems that, the 1355 connection being so unreliable, we should not take the risk to loose the connection and rely on the auto-reconnect of the SMCS to recover it. My idea (the idea of the software guy who know nothing about HW) is that the Master Reset should only reset the 'DEC' part of the FPGA but leave the '1355' part active such that we don't loose the connection. 04/04/03 (Back to CSL) : ********************** Integrated some of the changes tried in Garching in the current version of the OBS : - enable the interrupts and configure the link to auto-reconnect only once the connection is established - modification to start any link as a slave. Configured the link to DPU as a slave. Connection is ok (did not tried yet disconnect and reconnect). Configured the red DEC as a master (temporary change). Connection but no transmission. Note that I have the same behaviour on the blue DEC link although I haven't changed the configuration. 07/04/03 : ******** Integrated some of the changes tried in Garching in the current version of the OBS : - grouped the init code in SmcsDrvState function. - start SmcsDrvState only when Seq init is completed Test disconnect/reconnect with DPU I noticed that when loosing the connection with DPU, the first SMCS chip reset occured only 5 seconds after. This means that quite often, while making some tests, I start a new SimDPU before the chip has been reset (so it reconnects but there is no transmission and everything seems to be blocked). Modified SmcsDrvIrq to have the chip reset as soon as possible. It seems to work better (10 success in a row). While DMC is not connected to DPU, SEMA_HK is still signaled so, when they reconnect again, a lot of HK packets are sent to DPU (all in a row). To avoid this behaviour, the timer driving the HK is stopped when the deconnection is detected and restarted once the link is established again. Studying the bad connection betwen DMC and DEC : Actually, the connection is good but there is no transmission. One packet has been received but the second packet is not transmitted at all (SMCS chip is programmed to transmit but the transmit index stays at the start address). I will try to look at the size of the last packet received : - 1st attempt : no packet transmitted at all ! - 2nd attempt : 0x0A words transmitted - 3rd attempt : 0x0A words transmitted - 4th attempt : 0x0A words transmitted - 5th attempt : 0x0A words transmitted - 6th attempt : 0x0A words transmitted - 7th attempt : no packet transmitted at all ! - 0x09 words tranmitted but, when looking in the gaBlueSpectroPacketBuffer, I see only zeros ! - 0x09 words tranmitted but, when looking in the gaBlueSpectroPacketBuffer, I see only zeros ! Are we sure that the EOP is received ? Since we don't see any data in gaBlueSpectroPacketBuffer, I wonder if the Blue Dec Rec has been released by the EOP. I noticed that the SMCS receive FIFO did contain almost always the same value : 20EA. I finally understood that this value was the second pixel value of a ramp (EA60 -> EA20 -> ???) with the bytes inverted (or one byte is lost somewhere). I have asked Eric for more information about it. 15-05-03 ******** JMG and MG had troubles to execute a sequence. The problem was wrong link connection (RED DEC SIM on BOLC link). However, it did not solve the problem (still the 'sync time-out' error message when executing a sequence). I have checked everything, made many tests, recompiled everything more than once, could not see anything wrong. Finally, I decided to swap BOLC link and RED DEC link (and recompiled); it worked ! Then, swaped them back to nominal affectation (and recompiled): it worked again !!! So, finally, I have exactly the same software as one hour ago when it was not working but it is now working fine !!!! Note that, when it was not working, the problem was that the SmcsDrv never detected the connection with DEC Sim (Dec Sim did detect the connection all the time). Conclusion : We can't write any conclusion. 26-05-03 (Garching): ******** Unpacking of the board and EGSE PC. Some configuration done on the EGSE PC to have all the simulators up and running. Repeated the tests between SimDPU and Crisa board : OK. First connection between HLSW and DPU : success. Test the reset : fails. Look with the JTAG, everything is fine on DMC side (NUL are transmitted) but doesn't see the connection. We finally found out that the SMCS chip reset is not handled the same way on the SPU and on DMC. The SPU does not reset the LVDS drivers anymore. Made the same change in DMC code. 27-05-03 (Garching): ******** Test the new version with the new chip reset. It sometimes work, sometimes not ! In my opinion, there are still some 'external influence' that we don't understand. Made a code comparison with Nabil to see how he handles the reconnect. We found that I was using the 'auto-reconnect' option and he doesn't. Removed it, test with JTAG : it works. Wrote it to EEPROM and test again : it works again. SUCCESS, the nominal switch-on procedure finally works (at least with DPU-SPU-DMC). Once we switch-on BOLC, it crashes DMC software although the 1355 cable is not connected (this is an electrical problem !). Tried to switch the BOLC before DMC. Can't get the connection although the gLinkStatus indicates the the link is connected ! 28-05-03 (Garching): ******** The problem with BOLC was in the sequencing of tasks start-up : - SMCS drivers were started first and, since BOLC was ready to connect, SMCS drivers immediately changed the status of the BOLC tasks (to indicate that there was a connection). - Then, BOLC tasks were started and the status of tasks was initialized at 'no_connection'. Solved by adding a sleep between the start of the tasks and the start of the smcs drivers. Then, the connection with BOLC is fine but, when I try to connect to a DEC Sim, the connection is lost (due to the chip reset). Adding the auto-reconnect bit for these 3 links solved the problem. But, the connection of both DEC Sims and BOLC does not last for long : after a few seconds, all three connections are lost. TO INVESTIGATE. BUG : when sending the Switch-on BB1, the connection with DPU is lost ! Actually, the OBS is compiled without the BB code so, the command tries to initialize a timer which has not been created. 16-06-03 ******** Received the BOLC Sim at CSL. Switched-on DMC then, BOLC. No connection established. Implemented the RESET_SMCS_CHIP2 command. Switched-on DMC then, BOLC, then send the command to reset the chip. Connection established. Sending the command again and again works everytime. Switched-off BOLC, DMC OBS is still running. Switched-on BOLC, DMC OBS is still running. Chip Reset, connection ok. OFF-ON BOLC, DPU connection lost, restarted the DPU Sim, connection ok again. Chip Reset 2, connection ok. Then, switched-on RED DEC, both BOLC and DEC are connected and transmitting; then, after a few seconds, they both stop transmitting at the same time ! When looking at the SMCS register, I see that : - no disconnection is detected - both receptions are blocked in the middle of a packet. If I reset the SMCS2, the same occurs again (the first packet is not transmitted completely). 17-06-03 ******** Disconnected the BOLC and made some tests where only the RED DEC is switched on and off. 90% of the time, I can get the transmission running. If not successfull at the first time, a new chip reset always solve it. So it seems that the loss of transmission appears only when both the BOLC and the DEC are transmitting. Connected the BOLC again. Added some debug variable to know what has been the last interrupt on the SMCS chip before the transmition stops. When transmit is stopped : Last interrupt was RXED on DEC Channel. DEC Channel RX_CAR is B20 (length received = 288 words : OK) BOLC Channel RX_CAR is 2B1 which means that 177 words have been received ! And BOLC is only transmitting HK packets whose size is 112 words !!! If I look at the DPRAM used by BOLC channel, I see that only the first 112 words have ever been used; the remaining part (including adress 2B1) contain the test pattern !!! The RX_FIFO contains 0x3878556a (looks like the first word of a HK packet). Note : last ISR value is 0x11000100 (= BOLC receive fifo not empty (it was empty in the last 5 interrupts), + DEC receive fifo not empty + DEC EOP received). Enabled the RX_FIFO_FULL interrupt on BOLC channel. Now that it is enabled, I can not even receive more than one packet ! First interrupt : 00100110 (Note the Parity disconnect error on DEC link that is not started !). Second interrupt : 00000300 (BOLC RX FIFO FULL). It seems that the first HK packet has been received correctly. The SMCS has been reprogrammed correctly, the second HK packet seems to be in DPRAM. I don't know what happened to the RX_EOP of the second packet ! 25-06-03 ******** Build a small DEC Sim to run a PC to see if we observe the same behaviour. Also enabled the RX_FIFO_FULL on red DEC link. I observe the same behaviour : the first interrupt I get is a RX_FIFO_FULL and then, I never receive any interrupt anymore. I noticed that both SMCS interrupts were not configured as 'level sensitive' this could be the reason for the deadlock I observe. Changed it but it does not seem to help. Found the reason of the deadlock after the first FIFO_FULL interrupt : since, in this case, no event had to be signaled to the HLSW, the DMPSC interrupt was not reset so, the interrupt was triggered again and again ... Now that this is solved : we observe the same behaviour as previously (when the FIFO_FULL was no enabled). Finally, we noticed that when in deadlock, the SMCS IPR did contain the EOP on both channel! so the problem is that the SMCS IRQ is not forwarded to the processor by the DMPSC (which is configured as level-sensitive). We don't understand why we have to reset the DMPSC interrupt even when it is level-sensitive ! Clarification has been requested to CRISA Solution is to reset the DMPSC interrupt before reading the SMCS IPR. This should work with the DMPSC Interrupt configured as edge sensitive (and DSP interrupt level sensitive) but it does not ! We need to configure the DMPSC Interrupt as level sensitive (and DSP interrupt level sensitive) !!! This is weird ! Requested more info to CRISA. 26-06-03 ******** Let the system run 20 hours. When coming back, Red DEC is still transmitting but not BOLC (connection is still active). Switching-on the monitor close to the BOLC restarted the transmition !!!!! (and lost the DPU connection) Transmition did stop after a little more than 1 hour for unknown reason. After having reconnected the DPU sim, I could reset the SMCS Chip 2 again with success. Made another test. This time, after one hour, the transmission with DEC has been lost (BOLC' is still running). This time, I could look at the SMCS register 0x120 words have been received but it seems that no EOP has been received (or its interrupt has not been seen ? Note : CHX_STAR contains 0x0000000D which means fifo full and not empty but does not show that the EOP has been received) 27-09-03 ******** Grating tests with JYP and BM. Modified the HK routines to have a special diagnostic mode at 1KHz and added a new measure to monitor the grating PID output. Thanks to this new HK mode, we can see that the output has a normal shape with some unexpected peak. By isolating P, I and D terms, we observed that they were produced by the D term. We first review the algorithm and reduced the number of fix->float->fix conversion. No change. It appears that Kd is always 0 or 8000 (0 or 1 multiplied by FREQ). This is due to conversion from small float to integer (before multiplying by FREQ). Solution was to multiply by FREQ and make the conversion to integer afterwards. 13-10-03 ******** Added the second SPU_TR_MODE. (change already made in AVM in Garching) 14-10-03 ******** Implemented the handling of EDAC interrupts. First, added a piece of code in smcs2_isr to test if DM EDAC Interrupt has been triggered. To generate EDAC errors, I enabled the EDAC on the area 2 of Bank 1 where the Sync FPGA currently is. This is done through command : PSC_WriteDMPSCReg(K_DMADD_DMPSC_CONFBANK1, 0x94df907f); Note it is very important to write '10' (and not '11') in B1IO2ENEDAC bit field. After some debugging, it seems that it is handled correctly : - interrupt is triggered - faulty address is identified - faulty address is stored in an array I'm not sure the interrupt is reset properly at the end. It is very hard to know since it is triggered all the time and it makes strange noise on the board (!). I should test it on another memory area where it is totally safe. Duplicated and adapted the isr into pmpsc_isr to handle PM EDAC interrupt. I have not been able to test it. 22-10-03 ******** Wanted to work again on the patching. For this purpose, I have noticed that Nabil and Stefano are both using mem21k after the compilation to add some code that initializes the variables at the beginning of the execution of their program. By this way, the whole software can be stored in PM which should simplify the patching procedure (only copy PM to PM, only 3 segments, ...) Procedure : rename onboard.exe onboard.out mem21k onboard.out -o onboard.exe -a crisa.ach Note : I had to increase the size of seg_init. When I start the execution, the DSP stops by itself (with PC = 0xFFFFFF which seems to be a stack-overflow). I have asked Nabil and Stefano if they have encountered some problems (waiting for their reply). 27-10-03 ******** As suggested by Nabil, I have added a call to __lib_setup_processor at the application startup (in NODE1.c). I can still see the same problem. Connected the JTAG to see what's going wrong when using mem21k. In the lib_setup_processor code, everything goes fine except the init_pm part in __lib_setup_memory. It seems that the description of the block is wrong and at some point, the process starts the copy of a zero size block. The first iteration of the loop brings a 0xFFFFFFFF counter and the full memory is then affected (same content copied everywhere including in the useful code). I have increased the size of the seg_init. No improvement (actually, no crash but the same behaviour). I checked if seg_init was completely copied into memory (by the SimDPU). It seems that 28392 bytes have been uploaded to memory but SimDPU says that the seg_init is 28552 bytes long!!! TO investigate. Procedure to stop early in the ASW. Connect JTAG, Chip Reset Start Upload software as usual Before telling the LLSW to give hand to HLSW, interrupt LLSW (F4) Set a hardware breakpoint when PM is accessed between 0x8000 and 0xFFFFFF Resume execution (F4) Tell the LLSW to give hand to HLSW. Step in the code ... 29-10-03 ******** Found the problem. It was in the SimDPU. The MSB of the upload address was not updated correctly when the address was going from 6ffff to 70000. Modified the code to write in EEPROM. Now, only seg_pmco and seg_init are copied into EEPROM. (I think seg_rth is filled by the startup-sw). I also modified the code writing in EEPROM to use the page mode (copy 128 words in a row then wait 12 msec). It seems that it works fine. Procedure to write in EEPROM : start DMC as usual, Start SimDPU for SUSW Upload software as usual (6-31) Start HLSW as usual (6-30-8032) Start SimDPU for ASW Send the command "COPY_OBS_TO_EEPROM" (2-88-0) Wait around 6 seconds (you can check in DMC_SW_GLOBAL_ST that the bit 18 is set to 1 during the writing). Normally, you can continue to use DMC OBS as usual. Procedure to start from EEPROM : Start DMC as usual, Start SimDPU for SUSW Copy EEPROM in seg_pmco (6-32) Copy EEPROM in seg_init (6-34) Start HLSW as usual (6-30-8032) Start SimDPU for ASW 03-04-05-11-03 ************** Upgraded the SimDPU to allow patching of the DMC OBS. The patch are generated in GeneratePatch in patch.cpp. I have not yet been able to test the SCOS output for patching but the output in the form of SimDPU packets works fine. 14-11-03 ******** I have tried to split the OBS into many segments : - created seg_seq and compiled the sequencer code inside. Since then, I can not execute the OBS anymore : - even if I don't use patching (use the old upload procedure) - even if I don't use mem21k (which seems to compress all segments to zero size ! (except seg_pmco, init and rth) Question : do we really need to reduce the size of the patches ? Answer : seems not since we stay below 2000 TM packets for a patch 12-12-03 ******** Added the InitDumpAddressTable function to store the address of the most likely dumped parameters in a fixed array (always in seg_dump) such that this array can be dumped to know where all the other variables are. Right now, only included the writable parameters. I also need to define a procedure to update the user manual as easily as possible every time these addresses change. It is possible to start the OBS in the WSim020.exe and dump the array and then paste it in the Excel sheet but it needs to be optimized. 15-12-03 ******** The chopper position offset is now a parameter. It is initialized at 5109. ChopGrat.s is now using it but I could not test it. (Finally tested on 16-12-03 : OK) 17-12-03 ******** Added an implementation for the home grating command. 1. Relative move 50° 2. When error is greater than 3°, we assume that we are on the hardstop. 3. Check that the limit switch is pressed (could not be done because MIM FPGA does not read them) 4. Store the current position in the offset 5. Reset the Pid such that you don't have a peak in current when you change the position to zero. We also worked with the redundant circuit of the grating (inductosyn reads in the opposite direction). Added a parameter dedicated to Grating (Range) that contains the number of inductosyn unit between the two hardstops. If this value is zero, it means that we use the nominal circuit if different of zero, it means that we use the redundant. We have finally created two homing procedure, one for nominal and one for redundant since the limit switches are on each side of the grating. We had to stop because both inductosyn did not had the same range (250 units more or 170 units less depending on the settings). BM and JYP need to find a way to tune this. 08-01-04 ******** JMG did not like to have 2 homing procedures and we have then decided to simplify them to have only one. A parameter is now specified to notify in which direction (towards positive or negative positions) it has to be performed. The homing stops when the absolute error is greater than 3° and we don't check the limit switch (it is DPU responsibility). Cleaned up the grating pid code. 09-01-04 ******** Implemented the grating launch lock routine (not the unlock). Made some tests without the hardware connected. I have problem writing the current command into the motor and reading the limit switches. 18-02-04 ******** Concerned about the CPU consumption of the DEC Receiver task. Measures of CPU load: - idle : 183 - with red dec on : 275 - with red dec on but no data and hk is copied : 252 - with red dec on but no RES_LOCK/UNLOCK : 267 Introduced two different types of RES_LOCK, one for WRITE and one for READ. Changed it in all the files. The LOCK_READ is actually doing nothing. No big changes to performance: idle = 182. Removed the event EVENT_BLUE_DEC_SIGNAL_WHEN_RECEIVING_SECOND_TO_LAST_READOUT and replaced by a variable. 23-02-04 ******** Received and connected a new BOLC Sim. If BOLC is switched on while DMC is already on, the connection is not established immediately but if you DMC_RESET_SMCS_CHIP_2, then you get the connection. This has to be done everytime the BOLC is switched on. Started to code the Fw controller. Some more test with BOLC and DEC BUGS: - Understand why when you switch on DMC when BOLC is already on, the connection is not established (it is done only after a DMC_RESET_SMCS_CHIP_2). - After that, if you switch-on a DEC, the connection is established but nothing is transmitted. (The bug is: if a chip reset is done before the link to DEC is established, no transmission occurs). - It seems that actually, it is no more possible to get a transmission since I have added the Fw controller to chopgrat !!! - It seems that the ISR frequency is 10KHz (so the CRE clock is wrong and that might explain why the transmission is not working). 14-05-04 ******** Configured the SMCS2 in Big endian. This means no more byte swapping for BOLC. This is to avoid swapping the bytes in the DEC FPGA (it was not working anymore after the swap). 14-06-04 (AM + ER + BM) (Delivery at Garching) *********************** in case of grounding problem: 3278 check that temp sensors are ok: hk diag with 571 - 572 (23b - 23c) Values are wrong (1680565016). Check MUXADDR on the backplane. All seems good but J2-C12 (on MIM1): no signal. Replaced FPGA by 3_1(3) (the one with MASTER RESET too short). Signal on J2-C12 is correct Temp sensors are still not working. hk diag with 433 - 434 (1b1 - 1b2) (reprogrammed CUSTOM_HK_1-2 to look at ChopperTemperatureSensorVoltage and Grating) The values do not really changes. Looked at MIM2, the enable of the last 2 MUX is always at 1. We should replace the component that have already been replaced twice (74AC138). Replaced FPGA by the latest one (3_1(4)). We still have the wrong MUXADDR so, the problem comes from the FPGA (not a dust or the socket). Test the Master Reset. The 1355 link is lost aftet the reset. Added 2 commands to restart the links 100msec after the master reset. Changed the time-out after a master reset : if (KS_SemaTestWT(SEMA_BLUE_DEC_SECOND_TO_LAST_READOUT_RECEIVED, 64) == RC_TIME) the time out is now 2000 to take into account the slow readout frequency after reset (64Hz instead of 256Hz). Tested the Heater commands. They failed. The code was obsolete. Corrected. Tested the chopper. It is unstable with the parameters that were stable at CSL. Benoit sets new parameters: Kp = 0x493e0; Ki = 0x1312d00; Kd = 0x258; Kf = 0x804; Tested the grating. While homing, It bounced. Retried many times. No more problem. 15-06-04 (AM + ER + BM + FM + EC) ********************************* Found a copy/paste problem in the DEC hk. Vdda is copied twice in the hk. The second one must be replaced by VddaCurrent Desoldered the 74HCT540 on MIM2 (not replaced yet). Rework on MIM1, a suspect wire could short-circuit 2.5V and 15V (J2-B6). Desoldered the 74AC138 on MIM2. Resoldered the 74AC540 and 74AC138 on MIM2. Added 1K resistors on the 3 address lines between the FPGA and the 74AC138 on MIM2 Started the acceptance test procedure. 16-06-04 (AM + FM + EC) *********************** Disconnected the overshoot killer on the DATA board. Continued the acceptance test. Started the OBS audit. 17-06-04 (AM + FM + EC) *********************** Continued the OBS audit. Changed the potentiometer setting to recalibrate the range of the ADC of the DATA board (ref is VSS). Continued the acceptance test. During the latest functionnal test, the FW did not stop on position. The hk analog is incorrect. It seems that some values are not at the right place (i.e : the 5V ref voltage has the behaviour of a temperature sensor voltage). Made a temporary version of the OBS to monitor the hk index. It appears the the bit3 of the index (bit19 of the register) stays at 0. We checked on the backplane if it was on the data bus. It seems ok. Put the timing FPGA we had for delivery. The hk index is incrementing fine. We found a short-circuit on MIM1 between MIMWR and GND. Removed it. And back to the latest FPGA (3_1(4)). Tested the FW: OK, grating: OK, chopper: OK, temp sensors: OK. For acceptance test problem, see Helmut's list. 20:30 Let's have a beer and eat some saussages. 18-06-04 (AM + FM + EC) *********************** Moved next to cryostat with the DMC + PC EGSE + our FPU. Tried homing the grating, it bounced many times (during 44-1-1). Looked with Hk diag. It seems that within 1ms, the error gets bigger than 3°. Could not find yet the reason. Started to test the switch-on procedure with HF. The second connection with DPU (after SPU switch on) fails 50% of the time. 19-06-04 (AM + EC + BM) *********************** Once we switched on BOLC, we lost the connection between DPU-DMC. Found that the smcs 1 was configured to autoreconnect. Removed it and it solved the DPU-DMC connection problems. Tried to switch on the DECs, after a few seconds, the DECs reach FIFO_FULL (no connection loss). Tried again with smcs 2 configured to autoreconnect. No change. Back to the first lab. Looked at the bad transmission of commands to DEC. When we send 0x40, the DEC receives 0x80. We suspect that bit 6 & 7 are swapped somewhere between Bus - DPRAM - SMCS. Ask CRISA for details. Looked at the Grating problem, the excitation sinus is not regular. It comes from the 8KHz that has jitter. With OBT connected, if the soft is running or not, it does not change anything : jitter of 2µs max. Without OBT, no jitter when the soft is not running but up to 10µs jitter when the soft is running. Checked the OBT, the signal is not clean (some transition are doubled, this can explain the 2µs jitter). Reduced the OBS, removed the access to FPGA -> still have the jitter (less often). Commented out the content of the ISR -> no more jitter. Made a stupid software with no access to Memory -> no more jitter. Upgraded the stupid software to access memory (with LSb of address the same as FPGA LSB address) -> no more jitter. 20-06-04 (AM + EC + BM) *********************** OBT probs : see Anomaly report (by EC). Tried the grating now that the excitation signal is correct. No progress (small oscillation). The grating is loose, tighten the screws. Even worse: big oscillation. 21-06-04 (AM + FM + BM + ER) **************************** Replaced a chip 90C32 by 26C32 on MIM1. The OBT is cleaner now. The DMC does not count 131072 but 131380. The OBT is still not perfect, some glitches are still seen so this could explain the frequency difference (but the difference seems to be very constant...). The cable between the OBT box and the DMC is 1000Kohms. Without this cable, we have now a perfect counting of the OBT!!! Performed a test suggested by CRISA to detect defaults on the CRISA boards. When accessing the SMCS ISR register, we see that the bits are swapped two. Returned the results to CRISA. CRISA suggested to take pictures of the boards. Done While inspecting the CRISA board, we found unsolded capacitors near the LVDS of smcs 2 . Corrected. 22-06-04 (AM + FM + BM + ER) **************************** Showed the working OBT to MPE. Martin will provide a new cable. Moved to the Cryostat Lab. Connected to our PC EGSE. Started to test the connection between CPU and DECs. The connection lasted for 3-4 hours (even while doing measures on the DECs). Repeated the acceptance test (DEC section). All voltages are measured with voltmeter and checked in the HK. GR1, 2 and 3 are perfect (both analog measurement and hk), on GR 4, analog measures are perfect. The hk shows some discrepency with other groups. Note that on group 4, the 0V reference is different from the other group and, it appears that it uses bit6-7 (that are swapped). We re-computed the values by hand. This explains partially the discrepencies. All this should be checked again when the CRISA board is corrected. Tested Bias commanding: (with the CRISA board bug !!) Set of parameters sent: dec hex 32 20 8 8 8000 0V 0 1V FFF 20 8 8000 1V FFF 1V FFF CRE integration : --------------- Checked grounding continuity between the table, the cryostat, the connectors: OK Martin connects harness on cryostat side. Francis connects harness on DMC side. Test setup: ---------- DMC connected to CRE in the cryostat. DMC commanded by DPU-SIM Science data sent to LinkReceiver (dummy SPU) on a second EGSE PC. The LinkReceiver is configured to receive packets and store them on disk (with circular file numbering). For the first test of CRE, in order to be able to see all datas on the EGSE, we will keep the slow readout frequency on DEC. Here is the set of parameters used: dec hex ClockPerReadout 128 80 ReadoutPerRamp 8 8 CreCtrlReg 8000 BiasR 0V 0 ZeroBias 0V 0 test procedure to test the Red CRE: ---------------------------------- 2-19-0 to switch red DEC ON checked HK, OK same as without CRE connected 2-21-0 to switch red detector array 2-12-0 to switch blue DEC ON 2-14-0 to switch blue detector array here is the HK we have (smallest capacitor is selected and CRE are inactive ) (Voltages in Volts and currents in µA) : DMC_DECR_VDDD_1 DMC_DECR_VSS_1 DMC_DECR_VGND_1 DMC_DECR_VCAN1_1 DMC_DECR_VCAN2_1 DMC_DECR_V0BIAS_1 DMC_DECR_VBI_R_1 DMC_DECR_V0V_1 DMC_DECR_VSCP_1 DMC_DECR_VDDR_1 DMC_DECR_VDDA_1 DMC_DECR_VWELL_1 DMC_DECR_IDDA_1 DMC_DECR_IDDD_1 DMC_DECR_ISS_1 DMC_DECR_ISUB_1 2.500686666 -2.999053926 -0.000152593 0.539414655 1.899929807 -0.000457778 -0.000762963 0 -0.099185156 -0.399029511 2.499771111 3.001037629 -4.943998535 -0.068666646 -248.0468154 -0.015259255 DMC_DECR_VDDD_2 DMC_DECR_VSS_2 DMC_DECR_VGND_2 DMC_DECR_VCAN1_2 DMC_DECR_VCAN2_2 DMC_DECR_V0BIAS_2 DMC_DECR_VBI_R_2 DMC_DECR_V0V_2 DMC_DECR_VSCP_2 DMC_DECR_VDDR_2 DMC_DECR_VDDA_2 DMC_DECR_VWELL_2 DMC_DECR_IDDA_2 DMC_DECR_IDDD_2 DMC_DECR_ISS_2 DMC_DECR_ISUB_2 2.501449629 -3.000122074 0.000305185 0.498519852 1.931516465 0 0.000152593 0.000305185 -0.088198492 -0.399334696 2.500991852 3.001495407 250.0076296 -0.015259255 -250 0.007629627 DMC_DECB_VDDD_3 DMC_DECB_VSS_3 DMC_DECB_VGND_3 DMC_DECB_VCAN1_3 DMC_DECB_VCAN2_3 DMC_DECB_V0BIAS_3 DMC_DECB_VBI_R_3 DMC_DECB_V0V_3 DMC_DECB_VSCP_3 DMC_DECB_VDDR_3 DMC_DECB_VDDA_3 DMC_DECB_VWELL_3 DMC_DECB_IDDA_3 DMC_DECB_IDDD_3 DMC_DECB_ISS_3 DMC_DECB_ISUB_3 2.697073275 -3.001190222 -0.000305185 0.501419111 1.902066103 -0.00061037 -0.000762963 -0.000152593 -0.100405896 -0.400555437 2.501297037 3.001800592 250.0076296 -0.007629627 -250 -0.007629627 DMC_DECB_VDDD_4 DMC_DECB_VSS_4 DMC_DECB_VGND_4 DMC_DECB_VCAN1_4 DMC_DECB_VCAN2_4 DMC_DECB_V0BIAS_4 DMC_DECB_VBI_R_4 DMC_DECB_V0V_4 DMC_DECB_VSCP_4 DMC_DECB_VDDR_4 DMC_DECB_VDDA_4 DMC_DECB_VWELL_4 DMC_DECB_IDDA_4 DMC_DECB_IDDD_4 DMC_DECB_ISS_4 DMC_DECB_ISUB_4 2.487868892 -3.007599109 -0.026245918 0.493331706 1.881008332 -0.025940733 -0.026093326 -0.025940733 -0.115054781 -0.404675436 2.4877163 4.262825404 250.0076296 -1.30466628 -250 -1.30466628 Start LinkReceiver on EGSE PC (menu 3, 100 files) 2-86-1-1 to start link with red SPU 6-50-24-1-0 to forward science data to red SPU No ramps are seen. It is logical since the CRE active bit has not been set. 6-50-27-3-80-8-804D blue (temp sensors enabled, select biggest capacitor, and keep CRE ON) 2-16-0 set blue param 6-50-27-3-80-8-804F blue (CRE active) 2-16-0 set blue param 6-50-26-3-80-8-804D red (temp sensors enabled, select biggest capacitor, and keep CRE ON) 2-23-0 set red param 6-50-26-3-80-8-804F red (CRE active) 2-23-0 set red param here is the HK we have now: DMC_DECR_VDDD_1 DMC_DECR_VSS_1 DMC_DECR_VGND_1 DMC_DECR_VCAN1_1 DMC_DECR_VCAN2_1 DMC_DECR_V0BIAS_1 DMC_DECR_VBI_R_1 DMC_DECR_V0V_1 DMC_DECR_VSCP_1 DMC_DECR_VDDR_1 DMC_DECR_VDDA_1 DMC_DECR_VWELL_1 DMC_DECR_IDDA_1 DMC_DECR_IDDD_1 DMC_DECR_ISS_1 DMC_DECR_ISUB_1 2.499771111 -3.000122074 -0.000457778 0.505081332 1.89947203 -0.010681478 0.029602954 0 -0.099032563 -0.399945067 2.499771111 3.001037629 58.6260567 4.318369091 -70.46723838 0.083925901 DMC_DECR_VDDD_2 DMC_DECR_VSS_2 DMC_DECR_VGND_2 DMC_DECR_VCAN1_2 DMC_DECR_VCAN2_2 DMC_DECR_V0BIAS_2 DMC_DECR_VBI_R_2 DMC_DECR_V0V_2 DMC_DECR_VSCP_2 DMC_DECR_VDDR_2 DMC_DECR_VDDA_2 DMC_DECR_VWELL_2 DMC_DECR_IDDA_2 DMC_DECR_IDDD_2 DMC_DECR_ISS_2 DMC_DECR_ISUB_2 2.501144444 -3.000732444 -0.000152593 0.488143559 1.891994995 -0.025940733 0.030213324 0.000305185 -0.08865627 -0.399792474 2.501297037 3.001190222 250.0076296 3.624073 -250 0.091555528 DMC_DECB_VDDD_3 DMC_DECB_VSS_3 DMC_DECB_VGND_3 DMC_DECB_VCAN1_3 DMC_DECB_VCAN2_3 DMC_DECB_V0BIAS_3 DMC_DECB_VBI_R_3 DMC_DECB_V0V_3 DMC_DECB_VSCP_3 DMC_DECB_VDDR_3 DMC_DECB_VDDA_3 DMC_DECB_VWELL_3 DMC_DECB_IDDA_3 DMC_DECB_IDDD_3 DMC_DECB_ISS_3 DMC_DECB_ISUB_3 2.501602222 -3.003936888 -0.000915555 0.500656148 1.901150548 -0.016632588 0.029602954 -0.000152593 -0.099948119 -0.4013184 2.502365184 3.001648 68.47590564 4.966887417 -81.65990173 0.114444411 DMC_DECB_VDDD_4 DMC_DECB_VSS_4 DMC_DECB_VGND_4 DMC_DECB_VCAN1_4 DMC_DECB_VCAN2_4 DMC_DECB_V0BIAS_4 DMC_DECB_VBI_R_4 DMC_DECB_V0V_4 DMC_DECB_VSCP_4 DMC_DECB_VDDR_4 DMC_DECB_VDDA_4 DMC_DECB_VWELL_4 DMC_DECB_IDDA_4 DMC_DECB_IDDD_4 DMC_DECB_ISS_4 DMC_DECB_ISUB_4 2.487868892 -3.009125034 -0.026551103 0.49775689 1.880397961 -0.03173925 0.023499252 -0.025788141 -0.114597003 -0.405133213 2.48832667 2.973876156 250.0076296 3.158665731 -249.7558519 -1.251258889 deselect the CREs 6-50-27-3-80-8-804D blue (temp sensors enabled, select biggest capacitor, and keep CRE ON) 2-16-0 set blue param 6-50-26-3-80-8-804D red (temp sensors enabled, select biggest capacitor, and keep CRE ON) 2-23-0 set red param here is the HK we have now: DMC_DECR_VDDD_1 DMC_DECR_VSS_1 DMC_DECR_VGND_1 DMC_DECR_VCAN1_1 DMC_DECR_VCAN2_1 DMC_DECR_V0BIAS_1 DMC_DECR_VBI_R_1 DMC_DECR_V0V_1 DMC_DECR_VSCP_1 DMC_DECR_VDDR_1 DMC_DECR_VDDA_1 DMC_DECR_VWELL_1 DMC_DECR_IDDA_1 DMC_DECR_IDDD_1 DMC_DECR_ISS_1 DMC_DECR_ISUB_1 2.506179998 -2.998748741 0 0.506302072 1.899777215 -0.009765923 -0.016632588 0 -0.099185156 -0.399334696 2.499771111 3.001037629 53.91857662 -0.030518509 -62.50190741 -0.030518509 DMC_DECR_VDDD_2 DMC_DECR_VSS_2 DMC_DECR_VGND_2 DMC_DECR_VCAN1_2 DMC_DECR_VCAN2_2 DMC_DECR_V0BIAS_2 DMC_DECR_VBI_R_2 DMC_DECR_V0V_2 DMC_DECR_VSCP_2 DMC_DECR_VDDR_2 DMC_DECR_VDDA_2 DMC_DECR_VWELL_2 DMC_DECR_IDDA_2 DMC_DECR_IDDD_2 DMC_DECR_ISS_2 DMC_DECR_ISUB_2 2.501144444 -2.999969481 0.000152593 0.498214667 1.892605365 -0.025635548 -0.025788141 0.000305185 -0.08865627 -0.399487289 2.501144444 3.001037629 246.9328898 0.022888882 -250 0.022888882 DMC_DECR_VDDD_1 DMC_DECR_VSS_1 DMC_DECR_VGND_1 DMC_DECR_VCAN1_1 DMC_DECR_VCAN2_1 DMC_DECR_V0BIAS_1 DMC_DECR_VBI_R_1 DMC_DECR_V0V_1 DMC_DECR_VSCP_1 DMC_DECR_VDDR_1 DMC_DECR_VDDA_1 DMC_DECR_VWELL_1 DMC_DECR_IDDA_1 DMC_DECR_IDDD_1 DMC_DECR_ISS_1 DMC_DECR_ISUB_1 2.506179998 -2.998748741 0 0.506302072 1.899777215 -0.009765923 -0.016632588 0 -0.099185156 -0.399334696 2.499771111 3.001037629 53.91857662 -0.030518509 -62.50190741 -0.030518509 DMC_DECB_VDDD_4 DMC_DECB_VSS_4 DMC_DECB_VGND_4 DMC_DECB_VCAN1_4 DMC_DECB_VCAN2_4 DMC_DECB_V0BIAS_4 DMC_DECB_VBI_R_4 DMC_DECB_V0V_4 DMC_DECB_VSCP_4 DMC_DECB_VDDR_4 DMC_DECB_VDDA_4 DMC_DECB_VWELL_4 DMC_DECB_IDDA_4 DMC_DECB_IDDD_4 DMC_DECB_ISS_4 DMC_DECB_ISUB_4 2.487868892 -3.008209479 -0.026398511 0.493179113 1.880855739 -0.032197027 -0.032502213 -0.019837031 -0.109866634 -0.404828028 2.487868892 2.974028748 247.4669637 -1.037629322 -250 -1.319925535 select the CREs again 6-50-27-3-80-8-804F blue (CRE active) 2-16-0 set blue param 6-50-26-3-80-8-804F red (CRE active) 2-23-0 set red param change the BiasR to 50mV (C8) 6-50-27-4-80-8-804F-C8 blue (change BiasR to 50mV) 2-16-0 set blue param 6-50-26-4-80-8-804F-C8 red (change BiasR to 50mV) 2-23-0 set red param We don't see any ramps. look at the data (last pixel of the first line (dummy) of red group I) readout counter value 3 57897 2 58173 1 57937 0 57958 7 57978 The first line contains only FFFFFFFF. All missing modules contains FFFFFFF. Some pixels contain values around E32dE403 Some pixels contain values around 758975C2 Disconnected the harness. Performed the short functional test with our SPU : OK. 23-06-04 (AM + ER + BM) *********************** Performed the short functional test again with Helmut. Tested the temperature sensors: OK Check the OBT time stamping: OK (note we forgot to synchronize on BOLC before checking OBT counter in photo packets). removed the CRISA mezzanine and brang it to workshop for repair. Continue with CRISA main board only. Filter wheels test ------------------ Connected FW SPEC nominal used nominal commanding, everyting worked fine. made a complete turn in open loop. diag hk for FW spec: 22C 556 VMOTA 234 564 VMOTB 230 560 IMOTA 237 567 IMOTB 22B 555 POS_A 23F 559 POS_B see d:\prj\pacs\test Garching\FW_SPEC_1_TURN.xls 2 things we should understand: currents are measured zero voltages stay a long time at zero during movements. Connected FW PHOTO nominal used nominal commanding, can not see a position detected. made a complete turn in open loop. diag hk for FW phot: 22C 556 VMOTA 234 564 VMOTB 230 560 IMOTA 237 567 IMOTB 233 563 POS_A 239 569 POS_B The position sensors show a value near zero. The voltages are variable (looks good). But the hall sensors keep the same value Connected our filter wheel on the PHOTO connector. We could make it move. Connected the FPU PHOTO FW on the spectro connector on MIM3. It does not move. For information : filter wheel are driven in open loop by sine wave with an amplitude of 60 mA. connected the redundant spectro wheel : OK connected the redundant photo wheel : same as with nominal circuit compared the resistor values for both FW. more or less the same values Grating tests ------------- Benoit puts MIM1 and MIM3 on extension boards to make inductosyn tuning. First we connect to redundant grating. Inductosyn excitation signal is too small (between 100mV and 200mV instead of 2V). Checked the resistor value of the grating (44 ohms) + harness (16 ohms) : 60.0 ohms, OK Tried with the nominal grating, it is even worse: 67mV. We found that the resistor value on the sine/cosine line of the harness is 1350 ohms. (much more than on our harness). Tried to add 2*680 ohms resistor on each line of our harness. The sine/cosine has now decreased to 1V. Replaced the mezzanine board of the CRISA board. Replaced a DEC by a PC and sent 0x40 and 0x80. Bits are not swapped anymore. Choppper tests -------------- compiled a temporary version of the software where the chopper is commanded in open-loop. Connected the chopper, position when free looks ok but some small peaks (not influence by the primary pump the turbo pump is still running). Offset has been set to 2055 (should have been a different value see Zeiss report) Degree 0 -0.26mA -204 -CC -1 -4.3mA -3169 -C61 -2 -7.81mA -6142 -17FE -3 -11.58mA -9107 -2393 -4.1 -15.73mA -12370 -3052 -9 -35.06mA -27571 -6BB3 +1 +3.52mA 2768 AD0 +2 +7.31mA 5749 1675 +3 +11.14 8760 2238 +4.1 +15.31 12040 2F08 +9 +34.75 27328 6AC0 parameters used. Note we also changed some parameters in chopgrat.s directly (they would need to be adapted by command in the future). Kp = 0x927c0; Ki = 0x45cf180; Kd = 0x3c0; Kf = 0x8d0; Rate = 164; AccumulatorLimit = 0x7FFFFFFF; OutputLimit = 0x7fff; Scaling = 0; ErrorLimit = 0x7FFFFFFF; PosOffset = 2055; KiCurr = 0x64; CHOPPER WORKS FINE AT FIRST TRY !!!!! CRE tests (2) ------------- This time, we change the width of the non-dest sync to 2 CRE clock. Only the red array is connected. We can finally see some ramps. We tried with and without bias, with 1pF and 3pF. See "first ramps.xls" Grating tests (continued) ------------------------- Measure of the excitation current/voltages of inductosyn both on STM and on QM. All measures are identical. Removed 1/2 of the 8Kohms resistor. It doubles the signal. Removed the other 1/2 of the 8Kohms resistors and added one 100kohms. We have now 1V but 100bits of noise which is too much. Added 4 1nF capacitor on the entrance of sine/cosine. Still the same noise. 19-07-04 (AM + BM) (In Garching again) ****************** high CPU load ------------- Made some test while CPU load was high: SWOF_RDEC, CPU load goes from 75 to 62 SWOF_BDEC, CPU load goes from 62 to 24 Again when CPU load was high: SWOF_BDEC, CPU load goes from 75 to 36 SWOF_RDEC, CPU load goes from 36 to 24 So it seems that Blue DEC (or its link) is the source of our CPU load problem. As soon as DMC is in a situation where it has high CPU load, it is not more possible to send command to BOLC (everything looks fine in the hk but, obviously, it is not received by BOLC). The problem appears only after a long period of use of DMC. And disappears after a long switch-off (30 minutes). When it happens, the DEC FPGA bug happens almost at every packet. other 1355 problem ------------------ Other problem noticed, after a chip reset on DPU-DMC connection, the last command received by DMC is received again. Since it happens both with DPU SIM and real DPU (which are totally different related to chip reset strategy), it is most likely that the problem is on DMC side. Grating ------- We tried to unlock the grating (both closed-loop and open-loop commands) without success. We have not been able to configure the excitation signal properly due to insufficient range of potentiometer. A resistor must be added tomorrow by Francis. Chopper ------- Made a test in open-loop. The response is not exactly matching the model. Made some modification to the chopper algorithm (tested on the model). The chopper is stable and moves quite good but it does not seem optimal (we didn't check if it was in specs). Analog HK --------- At the end of the day, we noticed that the diag hk was not synchronized (i.e : values are acquired correctly but are not at the right place (we can see ramps in the REF_5V when the temperature sensors are on)). This seems to explain why we are not always able to drive the temperature sensors. 20-07-04 (AM + BM + FM) *********************** The analog hk is working fine this morning. So it seams that (like the CPU load problem), it is related to the temperature of DMC. So, we can read the temperature sensors: FW_SPEC = 1626 ohms FW_PHOT = 1573 CHOPPER = 1427 GRATING = 2759 FPU_T1 = 1682 FPU_T2 = 1551 CAL_SRC = 510 Introduced the chopper open-loop mode. It works fine. (2-57-1-100 to activate it, 2-57-1-0 to deactivate it) Burned OBS 5.007 in EEPROM. Now starting version 5.008. Introduced 11 new parameters to the chopper controller param block to be able to configure completely the controller without patching the OBS. Note, it also had some influence on the grating controller since they were using common code that could not be used anymore. Be carefull next time we use grating. Also added some more diagnostic hk to look at science data from DEC. Launch Lock ----------- FM+BM made some modification on MIM3 to be able to increase the current in the launch-lock. After that, in the hk, we still don't see the bits updated correctly (Launch-Locked bit is always at 1 while Launch_Unlocked is changing at any time). FM+BM looked at the voltages on the LL sensors, it appears that they are too close to the threshold (2.5). FM changes one resistor value to have this threshold at 5V. The bits in hk are now correct. We are able to move the LL many times in any direction. When using closed loop, The Launch lock does not go to the limit (it stops too early after the LL sensor has reached 5V Started to code the grating degraded mode (copied from FW controller). It is ready but not tested yet. Made a test from SCOS2000. After a while (30min), DMC is unstable again. There are 2 new entries in hk that we can monitor (the number of SMCS2 interrupt and the number of SMCS2 interrupt that are handled (the usefull one)). Here is what we see: NB Interrupts NB Interrupts handled 56069 1064 56053 1064 56054 1064 (2x512 for the DEC + 40 for the BOLC) Note: the total number of interrupts also take into account the interrupts from DMPSC but DMC_DM_SF_IND and DMC_DM_DF_IND are not incrementing so it seems that all interrupts are really coming from the SMCS chip. 21-07-04 (Garching in operation room) ************************************* DMC_DECB_HEAT1_C and V shows the PWM voltage and the current measure in temperature sensors (I think it comes from the DEC FPGA bug and that the analog hk index is wrong so the wrong value is updated). We looked at the recording from yesterday (We saw VDDR and VSS both at -3V). It appears that they are not updated at all (this confirms the DEC FPGA bug being the source of the HK problems). Diag hk when cpu load was high (Blue DEC was obviously stucked, but no increase of the CPU load. A chip reset makes the CPU load goes high). last interrupts Number of SMCS2 interrupts, (sampled at 1KHz) 00010001 00000000 00000000 00000000 7652935.0 (link3 receive FIFO not empty, link 3 EOP received) 00010001 00000000 00000000 00000000 7652935.0 00010000 00000000 00000000 00000000 7652945.0 (link3 receive FIFO not empty) 00010000 00000000 00000000 00000000 7652988.0 (link3 receive FIFO not empty) 00010001 00000000 00000000 00000000 7653031.0 (link3 receive FIFO not empty, link 3 EOP received) 00010001 00000000 00000000 00000000 7653031.0 00010000 00000000 00000000 00000000 7653057.0 (link3 receive FIFO not empty) 00010000 00000000 00000000 00000000 7653088.0 (link3 receive FIFO not empty) 00010001 00000100 00000000 00000000 7653126.0 (link3 receive FIFO not empty, link 3 EOP received, link 2 receive FIFO is not empty) 00010001 00000100 00000000 00000000 7653126.0 00010000 00000000 00000000 00000000 7653158.0 00010000 00000000 00000000 00000000 7653189.0 00010001 00000100 00000000 00000000 7653222.0 00010001 00000100 00000000 00000000 7653222.0 00010000 00000000 00000000 00000000 7653255.0 00010000 00000000 00000000 00000000 7653293.0 00010001 00000100 00000000 00000000 7653321.0 00010001 00000100 00000000 00000000 7653321.0 00010000 00000000 00000000 00000000 7653347.0 00010000 00000000 00000000 00000000 7653388.0 Link 3 is the Red DEC (but it is the Blue DEC that seems too be stucked). Back to the lab with DMC standalone. Since we always saw the problem on blue DEC, we swapped blue and red Base boards. We let it run during lunch. Actually, after a few minutes, the red DEC (that is the Blue DEC Base board !) is stucked. Its analog hk is not updated anymore. We saw it happens exactly at the time we were sending a set_param command. Can it be that we need to insert a wait time between the 6 commands sent to DEC ? Unfortunately, after we sapped blue and red Base boards back to nominal position, we saw that the analog hk of red DEC is stucked ! I think the valid bit is never set to one again. Packets are transmitted correctly (we see science data in custom hk). It happened again at the time we were sending the param block. (but, in this version of the OBS, we had introduced a 1ms wait time before each command so it seems that it does not solve anything. So, we remove the 1ms wait time). Note, I also tried with the auto-reconnect removed for smcs2 but it did not change anything. While red DEC is stucked, we try a smcs2 chip reset. The cpu load increase till 65%. analgo hk at 1KHz (note that BOLC is not powered on, blue and red DEC are at 512Hz) The last ISR is (in hex) nb of interrupts: 00000000 E36FB9 (nothing !) 10000000 E36FB9 0 (link3 receive FIFO not empty) 00004000 E36FAC 4 (link2 EOP received) 00004000 E36FB9 13 (link2 EOP received) 00000000 E36FD7 30 (nothing !) 10000000 E37007 48 (link3 receive FIFO not empty) 00004000 E3701A 4 (link2 EOP received) 00000000 E3702D 13 (nothing !) 00000000 E3705D 48 (nothing !) Then, we switched off red dec 00004000 13737FD (link2 EOP received) 00004000 13737FD 0 (link2 EOP received) 00000000 137381B 30 (nothing !) 00000000 137384B 48 (nothing !) 00004000 137386A 31 (link2 EOP received) 00004000 137386A 0 (link2 EOP received) 00000000 137388E 36 (nothing !) 00000000 13738BE 48 (nothing !) Also checked the DMPSC register during that time. It is always = 8 (external interrupt 0). So, this is really only the SMCS2 that generates the interrupts (not the EDAC). We checked the heater currents after DEC switch-on, they are = 0 (we tried to command them and compared real values with hk values; OK). Actually, with an external temperature sensor, MPE sees an increase of the temperature near the blue spectro array everytime we switch-on blue DEC so they suspected our heater. But, we also saw this increase while DMC was totally disconnected from FPU (but, it happens when we switched on the DEC !!!!). We could not reproduce it later. Measured some DEC analog HK: P15Vcur = 17600 N15Vcur = -5680 P5VCur = 9840 ACCur = 16400 DCDCTEMP_3 = -11160 DCDCTEMP_4 = -11160 Made a new version of the OBS (5.010) with the CUSTOM_HK_ENTRY_8 looking at SMCS2 IMR. And with the DEC temperature sensors expressed in ohms. 22-07-04 ******** We made a test from SCOS2000 where DEC stayed off and only BOLC was connected to SMCS2. After a while we tried to send a command to BOLC an it was obviously not interpreted by BOLC (so not correctly transmitted ?). I looked at the total number of interrupt of SMCS2 and it was incrementing by 440 every 2 seconds (while we were receiving data+hk at 20Hz) while we should have had 240 EOP received. It seems that we have 2 interrupts for every science packets ! Made a new version of the OBS (5.011) with the DEC FPGA bug masking also applied to RRR field in DMC header. Started to implement the grating degraded mode. I had to invert the sign of the output (compared to the code of the FW controller). At least it is moving. I do not have time to investigate any further. Notes: You must enter degraded mode when off You must not enable the controller 1 unit = 1 step = 1/256 of a period I should modify the controller such that it keeps the current at least a little time after the end of the move (to stabilize) or keep it always. I showed the grating moving to Sepp. We modified the P to decrease the raising time. P=0x250, stored in software. 23-07-04 ******** Tested the grating from SCOS2000. Everything was OK. Grating range measured (homing to zero then move 50¡) = 1013610 Tried to move the FW again. None of them moved. We again saw an increase of the temperature near the BLUE spectro arrays once CREs were selected. In the hk, we see a 0.1mA current and the voltage oscillating in the DMC_BDEC_HEAT_1. Sepp computed that this current should bring the detector at 2.5K. After a while, they stabilize around 2.65K which proves that there is really a current going through the heaters. 27/7/2004 (in Garching) ********* 16:00 starting disassembly JMG + MvB 28/7/2004 (in CSL) ********* starting assembly first connection to DMC, after DEC switched ON, we lost the connection DPU-DMC. could reconnect after chip reset. After a while, once we send the param block to the DEC, the blue DEC analog hk stop updating. We connected the DEC directly to the LinkReceiver and looked at the packets. In fact, the Valid bit for the analog hk is never set to 1. All other digital hk look correct. Connected Franci's setup to be able to identify if pixels are transmitted at the right position. It is ok now but we leave it connected to see if if changes after a while. 29/07/2004 ********** Let the system run for a while (at least 1 hour) with both DEC on at 256Hz. After I reconnect to DPU after a chip reset, I noticed that the CPU load was high and both DEC were still sending valid hk. (this is the proof that these problems are not related). CPU load is 63.3% switched-off Red DEC CPU load is 55.5% total number of smcs2 interrupt is increasing by 55797 send a command to decrease the blue readout freq to 128Hz CPU load is 38.8% total number of smcs2 interrupt is increasing by 27902 send a command to decrease the blue readout freq to 64Hz CPU load is 30.7% total number of smcs2 interrupt is increasing by 13949 switched-off Blue DEC CPU load is 22.7% total number of smcs2 interrupt is increasing by 0 switched on Blue DEC (it starts at 64Hz) CPU load is 30.7% total number of smcs2 interrupt is increasing by 13949 send a command to decrease the blue readout freq to 32Hz (255 clocks per readout) CPU load is 26.7% total number of smcs2 interrupt is increasing by 6976 Get diag hk at 1KHz (238-441-442 : EE-1B9-1BA): DMC_B_SPEC_READ LAST_SMCS2_ISR SMCS2_ISR_COUNT 2093304 4000 6666bf7 2093304 4000 6666bf7 2093304 0 6666c0f 2093304 0 6666c3f 2093305 4000 6666c64 2093305 4000 6666c64 Then, nothing happens between 2 packets 2093305 4000 6666c64 2093305 4000 6666c64 2093305 0 6666c76 2093305 0 6666ca6 2093306 4000 6666cd1 2093306 4000 6666cd1 So, it seems that, for each packet, 108 unexpected interrupts are received during the 2 ms preceding the EOP ! Removed the CRISA board from the rack and checked the pins of the SMCS (and all the pins on the board). We found pins 50 and 51 very close to each other (VCC and VGND). We separated them. We have soldered a wire on the mezzanine connector (on F-20) to monitor the SMCS interrupt. PD re-soldered the capacitors on the mezzanine links. CRISA board back in the rack. Started a new test with both DEC at 256Hz. Waiting for a CPU load increase (We send commands to DEC or reset SMCS2 every 5 minutes). After 30 minutes it happens. We can see the interrupts on the scope. We can see 18 packets of interrupts (18 lines). This behaviour could look like the FIFO_NOT_EMPTY interrupt (but this interrupt is masked and the interrupt register is empty !). Note that, when it happened, we had a lost of connection loss between DPU-DMC (when switching-on/off DEC power supplies) and the connection between DEC-DMC was not stable at all (the connection remains but DEC is in FIFO_FULL (FCT lost ?)). 02/08/04 ******** Changed the links affectation such that the DECs are now on SMCS1 (Only swapped DECS and SPUs). It is running fine (need to wait for a long time ...). Note that we can not read the DEC hk because the SMCS1 is not configured as big endian (=> hk index is not at the right place) Let the system run for 24 hours. After 8 hours, the Red DEC stopped sending packets. Note that both green LEDS are still on after 24 hours. After 17 hours, the Blue DEC stopped sending packets. Note that both green LEDS are still on after 24 hours. After 24 hours, I did a chip reset and the packets were sent again at the right rate. The CPU load is still low. 03/08/04 ******** Right after the 24 hours test, I went back to the nominal configuration (DEC on SMCS2) and started the system again. Immediately, I observe the high CPU load. Made a small test program where link 2 and 3 of smcs 1 are sending data to link 2 and 3 of smcs 2. Every 10 seconds, a chip reset of both chips is done. After one hour, I still have not observed anything wrong. Note that for this test, I had removed the MIM1 board to make sure the CRISA board was working as a stand-alone system. Reconnected the DECs to the SMCS2. First used the OBS to switch on the DECs and configure them to send 256packets/sec. Then, started the test software. In a first time, I can not get a transmission because I'm only catching the EOP and not the DATA_RXED. At the same time I make this modification, I also modify the length of packets read (it was 256 words, it is now 512). After I have done this, I can see unexpected interrupts received (but, I have been using the DEC for around 30 minutes which is the normal time to have them in this state). No time to cool the DECs today so, I'm connecting the SMCS together again. DECs are off. I can also see the unexpected interrupts with the test sofware !!!! Removed MIM1. Still the same unexpected interrupts. Note, I'm sending around 481 packets/sec on each links (packets are 513 words long: (start address is 0x400 and end address is 0x600)). For SMCS2, I have 65701 unexpected interrupts/sec (=136 unexpect interrupts/packets) I remember that the packets shall not be longer than 512 words so I have reduced the size of packets to 512 words (start address is 0x400 and end address is 0x5FF) Still the same. Reduced the size to 256. Still the same. Reading 512, transmitting 256 words. No change. I tried to stop testing the DATA_RXED bit in the interrupt register (it was actually not done in the OBS). Then, only the first packet is transmitted but I observe also unexpected interrupts. I do not understand why packets are not transmitted in this case since we are expecting packets longer than what is sent, checking the EOP shall be enough. 04/08/04 ******** Started the test software again in the morning. It works fine (note that it is the same version of the software that had the unexpected interrupts yesterday). Started at 10:06 (PC time) After one hour, it is still running ok without any unexpected interrupt. Switch-off the rack. (Note, I let smcs1 connected to smcs2) Put MIM1 back. Start OBS, Switch-on both DECs. Stop OBS (leave DECs on). Start test SW (smcs1 connected to smcs2) with DECs on and not transmitting (since not connected) at 11:18. At 11:48, I observe the unexpected interrupts. At 11:51, unplugged the power cord of the DEC external supply and leave the test SW running At 11:52, we still observe the unexpected interrupts Note that it seems that the communication is quite often lost (or at least, packets stop being transmitted). It happens more often on the link3. At 12:21, we still observe the unexpected interrupts. Switch-off the rack Switch-on the rack 5 sec after Start the test SW again. At 12:23, we still observe the unexpected interrupts. At 12:23, switch off At 12:44, switch on, start Test SW (DECs are off) At 12:45, no unexpected interrupts At 13:56, there are unexpected interrupts (although the DECs have never been switched off) At 13:57, Switch off At 14:29, switch on, start Test SW (DECs are off) At 14:30, no unexpected interrupts At 15:30, there are unexpected interrupts Switch off Crisa board out of the rack and connected to an other power supply. At 16:45, switch on, start Test SW At 17:09, interrupted the test, will continue tomorrow 06/08/04 ******** Crisa board out of the rack and connected to an other power supply. At 10:02, switch on, start Test SW At 10:49, stopped the test and started a new version of the test SW where EDAC interrupts are disabled At 10:50, restarted Test SW At 11:20, stopped the test to test the grating Crisa board back in the rack, connected to STM grating to test the new inductosyn readout. It is working fine. There is no need to invert the sign of the inductosyn anymore. While performing a long duration test with both DECs on, with a sequence performing steps on the grating, the grating hit the hard stop at the time the communication with the DECs were lost (all happened at the same time). In the grating position readout, we see a jump of 16600000. (Which can be generated by the software trying to cancel an overflow of the period counter). Connected the temperature sensors and start a new test where they are measured. Actually, I forgot to connect FPU_T1 and FPU_T2 but after 15 minutes, their measure is said to be valid. Obviously, the analog HK is wrong, DMC_REF_5_VOLT shows ramp that seems to be the voltage measured on some temperature sensor. It looks like the hk index is wrong. Note that we have already observed this behaviour in Garching. It also happens after 'some' time. Added the move to central position at the end of the home grating. Moved the grating in the clean room on the cryostat. Connected. The inductosyn readout is a little more noisy (6-8 units instead of 1-2). 09/08/04 ******** Tried again the Test SW with the external power supply. After one hour, we still do not observe any unexpected interrupts. JMG soldered a thermocouple on the mezzanine board (above the SMCS2) chip. And PD added two heaters in the box. Once the SMCS2 chip reached 35°C, we observed the unexpected interrupts. During cool down, I tried when it was 30°C and it worked fine. 14:34, started OBS in the rack with temperature sensors simulator connected. Enabled temperature sensors measure. 14:34, SMCS2 T° = 26.1° 14:38, SMCS2 T° = 29.1 14:44, SMCS2 T° = 31.6 14:54, SMCS2 T° = 34.8 15:00, SMCS2 T° = 36.3 15:05, SMCS2 T° = 37.2 Interrupted the test to upload a new version with hk index error detection 15:16, SMCS2 T° = 38.1 15:23, SMCS2 T° = 38.7 Switched-on BOLC (lost the communication DMC-DPU), and both DECs SMCS2 is not able to connect to DECs (connection with BOLC is fine) 16:00, SMCS2 T° = 40.7 The HK index just started to give wrong values. Switch-off Reconnected grating after the cryostats has been closed. Start OBS The grating position shows jumps of 4194000 (bit 22). I can see 50 jumps a second during 1 minute then they disappear. While homing, I also saw some smaller jumps. 10/08/04 ******** Made a new version of the test SW where SMCS1 is receiving data from SMCS2. Started a long duration test at 14:30 (T° = 28.7°) At 16:24 (T° = 37.9°), no unexpected interrupts have been reported. Started the old test SW, the unexpected interrupts are there immediately. 11/08/04 ******** Big connections problems: DPU comm is lost almost immediately after the DEC have been switched on and the DEC connection is never reached. The connection with BOLC is always fine ! It is not lost when DEC are switched on. Once the DECs are on, it is possible to reconnect to DPU but the communication gets lost a little time after. Connected the red DEC directly to the PC. We can get the connection and keep it. To summarize: - DEC and BOLC are using the same FPGA for 1355 - BOLC can communicate with the CRISA board perfectly - DEC can not (can not even connect right now) - DEC FPGA is working fine since it can communicate with the PC - the only difference can be: -- the cable -- the grounding Testing the cable used for BOLC (the blue one with and additional black wire): The pins 3 are connected together. On the Nexans cable, pins 3 are not connected On the grey 1355 cable, pins 3 are connected. We tried to connect CRISA and DEC using the grey cable but it does not help. We open the BOLC Sim box to see where is the pin3 connected. The 220V ground is not used and the BOLC is not grounded to other units. There are also capacitors on the LVDS (10 pF). Pin3 is connected to the 0V. On the DEC, pin3 is also connected to 0V. On the CRISA board, pin3 is also connected to ground. On the Red DEC base, we have unsoldered pin3 of the nominal 1355 connector Tried to reaffect the links such as BOLC is on link3 of smcs2. We were unable to connect. Crisa board in the test box and DEC on the Tester. Unable to connect to DEC. Connected the DEC on the BOLC link. We could connect ! Connected the BOLC on DECs link. We could not connect !!! It looks like it is impossible to connect on link 2 or 3 ! Ran the test SW. It runs ok (link3 has transmitted 587 packets and then stopped but, at least, it means that we could have the connection once. link2 is working fine). Crisa and DEC back in the rack. Connected Blue DEC on BOLC link. We could connect (the OBS does not make the difference between BOLC and DEC) Connected Red DEC on Blue DEC link. Could not connect. 17/08/04 ******** Removed mezzanine board and sent it to CRISA. 18/08/04 ******** Summary of the 1355 communication lost problems. *********************************************** Status before: ------------- Every time we switch on the BOLC sim, the soldering iron or some power supply, we lose the 1355 communication between DPU Sim (PC) and the DMC. During some period, we also observed communication loss between DEC and the DSP board. Observations: ------------ Connected oscillo probes at the output of the receiver LVDS on the CRISA board. At the time we switch on the BOLC sim, we observe a lot of noise at this output (see graph attached). The SMCS state machine can not handle this noisy signal and enter an unknown state. Once the short connection has been added to the CRISA board (see below), this noisy signal has almost completely disappeared (some shorter peaks still appears at some time). This has greatly decreased the number of disconnection but we still had some. At this time, we have decided to replace the cable between DPU Sim (PC) and DMC. We were using a non shielded cable and we replaced it by a Nexans cable. Summary of modifications done: ----------------------------- On the CRISA board: Added a short connection between the chassis of the CRISA board and the ground (0V). The connection is done between one of the 1355 connector and a grounding point nearby. On the BOLC sim: In order to use the Nexans cable, we have connected the chassis of the 1355 connector to the ground. Note: In order to have a 1355 connection, the grounds of each board must be at the same voltage. Previously, they were connected by the pin3 of our blue cable but the pin3 is not used in Nexans cable. In Nexans cable, this connection is done by the outer shield. The cables: BOLC-DMC: Replaced the blue CSL made cable (with pin3 connected) by the Nexans one (with chassis connected). DMC-DPU Sim: Replaced the grey CSL made cable (without shielding) by a Nexans one (with shielding) Status after: ------------ During 4 hours, we have been able to switch on any source of noise at any time without observing any loss of communication. We will keep this configuration active to make longer observation. Since we do not have the CRISA mezzanine board, we have not yet been able to test the communication between DEC and DSP board. 19/08/04 ******** Tried to reproduce a bug seen in Garching where, after a chip reset, the DMC re-execute the last command received from DPU. Could not reproduce it. Removed the compilation flag CRISA_DRIVERS Connected Blue DEC to link 2 of SMCS1 to test the heater current Can not see any current in the hk. The Ref0V is wrong (increasing all the time) and none of the other analog hk is updated. Worked on the grating open-loop mode with BM. We have increased the number of steps to 16384. It is working well enough. We again saw some big jumps in the inductosyn readouts. We cooled down the MIM1 FPGA with a cold spray and it disappeared. 20/08/04 ******** Modified the code of the cal src to use the MIM1 extension board. Working fine except the src 2 reading that is not always correct and the pos/neg pulse that is not always complete. To be checked on monday. The cal src open-loop is not working fine (the output goes back to zero after 250ms). 25/08/04 ******** Cal Src "calibration": In measure only mode (small 100mV pulse every 20sec, measured made using big gain) resistor value read by Src1 read by Src2 10 101963 103063 20 202356 202842 30 302513 302719 40 403550 402547 50 503510 502412 60 603667 602145 70 704416 702006 In open loop with 0x1000 (pulses of 1.25V) (measured made using small gain) 10 102611 104355 20 203033 204168 30 303782 303954 40 405457 403749 50 506780 503419 60 607942 603659 70 708636 703099 02/09/04 (at Garching) ********************** Modified the "set param both" procedure. We now wait that both DECs are connected again before sending the parameters. It seems that it could happen that only the blue DEC was connected when we were trying to send the parameters to both arrays 06/09/04 (at Garching) ********************** GRATING INDUCTOSYN ADJUSTMENTS Connection of the grating to DMC. Oscilloscope on sine and cosine signals (after amplification). Strange sawtooth signals are observed on the sine and cosine signals when DMC is switched on (and already before grating switch on). Amplitude of these sawtooth signals : ~10 V pkpk Frequency : ~52 kHz but different frequency for sine and cosine. A second MIM1 board is used separately from DMC and only the inductosyn amplifiers are powered. Everithing is disconnected between DMC and FPU. The sawtooth signals are still there. After analysis, amplifier oscillation is suspected and capacitors C91,C92 C93 and C94 (1.5 nF) are removed from MIM1. Then oscillations (sawtooth signals) disappears from sine and cosine signals, which are now nominal. Adjustment of sine and cosine amplitude by tuning the excitation amplitude. Adjustment of sine and cosine phase to be in phase with the reference signal. CHECK OF GRATING OPERATION : Digital noise < 5 digits For the grating QM, you must invert the sign for the nominal inductosyn. PID parameters : 6-50-15-3-250-2710-5 Ramps and homing OK Steps OK Test of the LL : Nominal LL is in locked position but does not open when sending command. Harness is checked and an open line is found on the LL motor power lines P11/16-17. Redundant LL is connected (P111 replacing P11) and open command is send (status cannot be checked as redundant LL position sensors are not operational). Nominal LL is reconnected (P11) and is now in unlocked position. After measuring the harness again, it seems that continuity on P11/16-17 is now nominal (~15 Ohms). Actually, a bad contact on P11 connector is suspected. Nominal LL is then tested again and lock and unlock is now operational and nominal. CHECK OF CHOPPER OPERATION : Chopper operation nominal : chopping test between -13434 and 13434 at about 5 Hz (simpleChop.seq sequence) CHECK OF FILTER WHEELS SENSORS : Switch on SPEC FW 2-58-0 DMC_FW_SPEC_CTRL bit 29 = 1 (FW in position B) Hall sensors values nominal Switch on PHOT FW 2-59-0 DMC_FW_PHOT_CTRL bit 29 = 1 (FW in position B) Hall sensors values nominal CHECK OF TEMPERATURE SENSORS : 2-95-0 to switch on Nominal values CHECK OF CALIBRATION SOURCES : 2-68-0 to switch on CS1 2-72-0 to switch on CS2 Resistances about 0.7 Ohms (DMC_CS1_RES_VALUE and DMC_CS2_RES_VALUE around 7000) 2-91-0 2-93-0 Small heating OK 07/09/04 (at Garching) ********************** HEATING OF THE CALIBRATION SOURCES 2-68-0 2-91-0 2-70-1-be6e0 2-72-0 2-93-0 2-74-1-127690 src1 at: 70°K = 78ohms = 780000 = 0xbe6e0 src2 at: 90°K = 121ohms = 1210000 = 0x127690 Warning : these values are based on proto source calibration data (QM calibration data not available) MEASUREMENTS OF DETECTORS GROUNDING : Isolation measurements between pin 33 and ground (cryostat ground or DMC connector housings) for connectors : P151 and P162 (Red DEC) P51 and P62 (Blue DEC) Normally, - Red DEC is connected to ground on FPU side and floating on DMC side - Blue DEC is connected to ground on DMC side and floating on FPU side Measurements : Note : first trial failed because all short circuit connectors must be removed on both DEC's before measuring. P151/33-GND : ~700 Ohms P162/33-GND : ~700 Ohms P151/33-P162/33 : ~1400 Ohms Normal values as we are measuring the harness resistivity. P51/33-GND : >1 MOhms P62/33-GND : >1 MOhms P51/33-P62/33 : ~1400 Ohms OK, blue DEC is well isolated from ground into FPU but both groud lines of groups 3 and 4 are connected together into the FPU (actually, all 25 FEE modules are connected to the same housing) Therefore, groups 3 and 4 cannot be connected together to DMC (grounds are connected in DMC trough amplifiers and must be independent). One solution is to remove the ground connection on one of the two groups (group 3 is chosen as the connection is made on the backplane and is suspected to be less efficient than for group 4, where ground is connected on the front panel). MEASUREMENTS OF RED DEC HOUSEKEEPING : Connection of Red DEC to FPU Switch on detectors : 2-19-0 2-21-0 Group 1 housekeeping : IDDA : 39515 = 51.5 µA IDDD : ~32767 = ~0 µA ISS : 0 = -250 µA (saturation) ?? The value of ISS is not normal (should be around -60 µA) but this phenomenon has been already measured by MPE with their own setup. Group 2 housekeeping : IDDA : 65535 = 250 µA (saturation) IDDD : ~32767 = ~0 ùA ISS : 0 = -250 µA (saturation) OK, after verification with MPE. These values are normal as MPE has measured 280 µA for IDDA with this group. All housekeeping voltages are indeed nominal, which means that currents are below the limit (about 300 µA). After switch off and switch on (and disconnection-reconnection of the harness), measurements of group 1 housekeeping again : IDDA : 60 µA IDDD : ~0 µA ISS : -69 µA Which are normal values. So high current on ISS disappeared but this also has been already observed by MPË. PIXELS MEASUREMENTS AND ANALYSIS OF RAMPS Red DEC connected only. Note : following pixel numbers are given from 1 to 25, 1 being the first pixel of group 1 (or group 3). Pixels output signals are put in housekeeping diagnostic and observed in real time. DEC settings : 256 readout per ramp (1 ramp per second). Observation of dummy pixels : 6-50-26-4-20-100-18f-c8 BiasR = 50 mV Obserations : - Dummy of module 1 and 14 : Ramps are good (no oscillations) but different signals, 14 has lower signal than 1 - Dummy of module 14 and 15 : Signal of 15 is still lower than 14 - Dummy of module 1,5,8 and 18 : All signals are similar Observation of real pixels : 6-50-26-5-20-100-183-c8-29 BiasD = 10 mV Smalest capacitor Chopper switched between 0 and 7.6 deg (2-53-1-6146) - Pixel 4 of several modules : Signal can be observed when chopper watches the calibration source. But large oscillations at beginning of the ramps. Deviation of supply lines on data connector (J55) (for group 1) to evaluate the noise on the supply lines. Observations : VDDA 64801-64810 VDDR 50693-50720 LVL 26395-26400 ZERO-BIAS 37636-37645 GND 38072-38082 CASCP 37073-37080 CASCN 26385-26392 BiasD = 10 mV BiasR = 10 mV (TBC) 1 digit = 93 µV So, oscillation/noise is lower than 1 mV Fichier : voltagesRedDec.txt Connection of group 3 on Blue DEC (short-circuit connectors removed for group 4) Observation of dummy pixels : 6-50-27-4-20-100-18f-c8 BiasR = 50 mV Observations : - Ramps are good and all similars for different pixels Observation of real pixels : 6-50-27-5-20-100-183-c8-2e1 BiasD = 180 mV Smalest capacitor Chopper switched between 0 and 7.6 deg (2-53-1-6146) Observations : - Signal can be observed when chopper watches the calibration source but oscillations are also present. - Changing the bias for dummy pixels (BiasD) has an influence on the amplitude of the oscillations. Setting a BiasD close to 0 removes the oscillations on real pixels. Note : this is an observation only, not a solution. Diag HK in the current (5.015) version of the software: ID ID(hex) DEC Module Pixel 621 26d Blue DEC 1 DUMMY n.a. 622 26e Blue DEC 14 DUMMY 623 26f Blue DEC 2 DUMMY n.a. 624 270 Blue DEC 15 DUMMY 625 271 Blue DEC 3 DUMMY 626 272 Blue DEC 16 DUMMY 627 273 Blue DEC 4 DUMMY n.a. 628 274 Blue DEC 17 DUMMY n.a. 629 275 Blue DEC 1 4 n.a. 630 276 Blue DEC 14 4 631 277 Blue DEC 2 4 n.a. 632 278 Blue DEC 15 4 633 279 Blue DEC 3 4 634 27a Blue DEC 16 4 635 27b Blue DEC 1 NC 636 27c Blue DEC 14 NC 637 27d Red DEC 1 DUMMY 638 27e Red DEC 2 DUMMY 639 27f Red DEC 3 DUMMY 640 280 Red DEC 4 DUMMY 641 281 Red DEC 5 DUMMY 642 282 Red DEC 6 DUMMY 643 283 Red DEC 7 DUMMY 644 284 Red DEC 8 DUMMY 645 285 Red DEC 8 3 646 286 Red DEC 14 4 647 287 Red DEC 5 4 648 288 Red DEC 15 4 649 289 Red DEC 8 4 650 28a Red DEC 18 4 651 28b Red DEC 11 4 652 28c Red DEC 21 4 08/09/04 (at Garching) ********************** Short functional test from SCOS2000 : Successful Procedure reminder: To select pixels in diag hk (Please always select 9 pixels): --------------------------- 6-50-13-10-ID1-ID2-....-ID9-FFFF (IDs must be entered in Hex) 2-76-1-1 to synchronize Diag HK on blue DEC readouts 2-76-1-2 to synchronize Diag HK on red DEC readouts To visualize : d:\prj\Pacs\TestPlan Tools\View HK diag DEC oscillation.bat Note : the IDs in the file will not match the one you have requested Then 'Shortcut to hkdiag.xls' Remove the last line and save the file as text in d:\prj\pacs\tests garching, Then, to visualize in Matlab: type 'resdec' To command bias: --------------- For Blue DEC: 2-12-0 to switch on wait 5 sec 2-14-0 to switch on CRE wait 15 sec 6-50-27-5- CLK_PER_READOUT - READOUT_PER_RAMPS - CRE_CTRL_REG - BIAS_R - BIAS_D 2-16-0 to send the parameters Usual values: CLK_PER_READOUT - READOUT_PER_RAMPS - CRE_CTRL_REG - BIAS_R - BIAS_D 1sec ramps 0x20 0x100 ... Smallest capa ... ... 0x183 Biggest capa ... ... 0x18F 10mV Bias 29 50mV Bias c8 100mV 199 200mV 333 HEATING OF THE CALIBRATION SOURCES ---------------------------------- Calibration sources: 2-68-0 2-91-0 2-70-1-c3500 (=80ohms) 2-72-0 2-93-0 2-74-1-f4240 (=100ohms) To chop: ------- 2-12-0 to switch on Blue DEC 6-50-27-2-20-100 2-16-0 to send the parameters to Blue DEC 2-10-1-1 to sync on blue DEC 6-60-bolcchop to upload sequence 2-49-0 to switch on chopper 2-51-0 to enable chopper 2-5-0 to start sequence (it lasts 10 hours) 2-6-0 to abort All other commands are rejected during the sequence To send parameters to Red DEC: ----------------------------- 2-19-0 to switch on Red DEC 6-50-26-2-20-100 2-23-0 to send the paramters to Red DEC To save diagnostic hk for later analysis: ---------------------------------------- 2-77-0 to stop hk diag delete d:\prj\pacs\Simulators\SimDPUV26\ASW\HkDiag.dat 2-76-1-XX to start hk diag again 2-77-0 to stop hk diag d:\prj\pacs\Test Plan Tools\View HK Diag XXX.bat to convert it 'd:\prj\pacs\Test Plan Tools\Shortcut to HkDiag.xls' 'Save As' .... INVESTIGATION ON THE ORIGIN OF OSCILLATIONS ON CRE SIGNALS Calibration sources: 2-68-0 2-91-0 2-70-1-c3500 (=80ohms) 2-72-0 2-93-0 2-74-1-e09c0 (=92ohms) Power supplies monitoring connector connected to bridge power supplies to selected modules 1-7 6-50-26-5-20-100-183-c8-0 BiasR = 50 mV BiasD = 0 mV Small capacitor Test with different BiasR - BiasD VoltagesRedDec2.txt: ------------------- BiasR = 50mV BiasR = 10mV BiasR = 0mV Content: 644 284 Red DEC 8 DUMMY 645 285 Red DEC 8 3 637 27d Red DEC 1 DUMMY 638 27e Red DEC 2 DUMMY 639 27f Red DEC 3 DUMMY 640 280 Red DEC 4 DUMMY 641 281 Red DEC 5 DUMMY 642 282 Red DEC 6 DUMMY 643 283 Red DEC 7 DUMMY VoltagesRedDec3.txt: ------------------- BiasD = 10mV BiasR = 10mV Changing the capacitor: 1. smallest 2. biggest 3. biggest (with chopping) 4. smallest (with chopping) Same content as previously TESTS TO OBSERVE SIGNAL ON BLUE DEC PIXELS BlueDecPixels1.txt: ------------------ BiasR = 0mV BiasD = 200mV (333h) Smallest capacitor Content: 630 276 Blue DEC 14 4 632 278 Blue DEC 15 4 634 27a Blue DEC 16 4 1. chopper on one source, BiasD = 200mV 2. chopper on one source, BiasD = 180mV 3. chopper on one source, BiasD = 100mV BlueDecPixels2.txt: ------------------ BiasR = 50mV BiasD = 200mV (333h) Smallest capacitor chopper on one source Content: 630 276 Blue DEC 14 4 632 278 Blue DEC 15 4 634 27a Blue DEC 16 4 BlueDecPixels3.txt: ------------------ BiasR = 0mV BiasD = 200mV (333h) Smallest capacitor 128 readouts/ramp Content: 630 276 Blue DEC 14 4 632 278 Blue DEC 15 4 634 27a Blue DEC 16 4 1. chopper on one source, BiasR = 50mV, BiasD = 200mV, 2. chopper on one source, BiasR = 0mV, BiasD = 200mV, 3. chopper on one source, BiasR = 0mV, BiasD = 220mV, 08/09/04 (at Garching) ********************** état DEC/MEC ON , DEC ON, CRE OFF housekeeping blue heater current = 0 observation blue T° augmente ( installation MPE ) déconnecter J78 => T° redescend la forme du graphique ( premier ordre ) suggère que c'est bien un échauffement et pas du bruit sinon on aurait eu un changement brusque au moment de la déconnection => investiguer par quel chemin ( non prévu ) du courant peut parcourir le blue heater work-around: laisser J78 déconnecté ( perte de blue heater et T° sensors ) Coupure de la mise à la masse du groupe 3 derrière le backplane tous les groupes peuvent maintenant être connectés Démontage carte Supply 1 modification du montage pour alimentations sans feedback ( remote sensing ) but : se ramener à la même configuration que MPE - alimentations en manuel et vérifier que les interactions ( "oscillations" ) sont absentes attention : les réglages des tensions d'alimentation ont été modifiées pour compenser les chutes de tension dans le harnais qui ne sont plus compensées automatiquement alim mesure non compensée mesure après réglage VSS -2.83 -3.00 VDDA +2.43 +2.50 VDDR +1.20 +1.20 préparation HK diag ID ID(hex) DEC Module Pixel 641 281 Red DEC 5 DUMMY 642 282 Red DEC 6 DUMMY 643 283 Red DEC 7 DUMMY 644 284 Red DEC 8 DUMMY 645 285 Red DEC 8 3 646 286 Red DEC 14 4 647 287 Red DEC 5 4 648 288 Red DEC 15 4 649 289 Red DEC 8 4 6-50-13-10-281-282-283-284-285-286-287-288-16D-FFFF ( note 16D should be red readout counter ) ( note use another decodong script ) ( problems with decoded hkdiag.xls ) ( files HKdiagwrongjmg.dat hkdiagwrongjmg.xls ) ( retry with standard list ) 6-50-13-10-281-282-283-284-285-286-287-288-289-FFFF seems OK but - bug FPGA => spurious values every 16 records - columns 1-2-3-7 constant at 65535 ( file hkdiagjmgfm09092004A.txt ) try again with this list 644 284 Red DEC 8 DUMMY 645 285 Red DEC 8 3 646 286 Red DEC 14 4 647 287 Red DEC 5 4 648 288 Red DEC 15 4 649 289 Red DEC 8 4 650 28a Red DEC 18 4 651 28b Red DEC 11 4 652 28c Red DEC 21 4 6-50-13-10-284-285-286-287-288-289-28a-28b-28c-FFFF power on red DEC 2 19 0 power on red CRE 2 21 0 change red DEC param list 6-50-26-4-20-100-18f-c8 BiasR = 50 mV 2-23-0 sync on red 2-10-1-2 start HK diag sync on red 2-76-1-2 stop HK diag 2 77 0 save diag HK file To visualize : d:\prj\Pacs\TestPlan Tools\View HK diag DEC oscillation.bat Note : the IDs in the file will not match the one you have requested Then 'Shortcut to hkdiag.xls' Remove the last line and save the file as text in d:\prj\pacs\tests garching, Then, to visualize in Matlab: type 'resdec' Usual values: CLK_PER_READOUT - READOUT_PER_RAMPS - CRE_CTRL_REG - BIAS_R - BIAS_D 1sec ramps 0x20 0x100 ... Smallest capa ... ... 0x183 Biggest capa ... ... 0x18F 10mV Bias 29 50mV Bias c8 100mV 199 200mV 333 ================================================== stop because problem with HK diag file format ================================================== MPE doing tests with Scos2000 crash ( lost DPU ) when sending "power on chopper" while BOLC sending science and SPU processing it and cal src control ON last test with scos2000 : start everything ( until BOLC sends science data and SPU processes them ) power on grating enable grating move grating relative 1000 oops - MPE forgot to change inductosyn polarity - grating status shows "at limit" and power supply chart goes off limit ( DMC current > 1A ) power off grating and start again properly - do homing - OK ( no crash ) power off grating power on and enable chopper still no crash power off and release system to CEA and CSL ================================================== 18:00 connect to EGSE ( see different HK lists and files names above ) power off red as we have cut the grouning link for group 3 on the back panel, we can reconnect the group 4 DEC to the FPU found P57 plugged on J157 this could explain a lot of above problems power on blue 2 12 0 start diag on blue 2 76 1 1 cre on blue 2 14 0 try to reproduce yesterday's files { reminder 622 26e Blue DEC 14 DUMMY 624 270 Blue DEC 15 DUMMY 625 271 Blue DEC 3 DUMMY 626 272 Blue DEC 16 DUMMY 630 276 Blue DEC 14 4 632 278 Blue DEC 15 4 633 279 Blue DEC 3 4 634 27a Blue DEC 16 4 635 27b Blue DEC 1 NC } 6-50-13-10-26e-270-271-272-276-278-279-27a-27b-FFFF 6 50 27 5 20 100 183 29 333 ( bias_r 10mV bias_D 200mV ) 2 16 0 list from group 4 ( connected to the FPU ) saturated ramps 6 50 27 5 20 100 18F 29 333 ( bias_r 10mV bias_D 200mV large cap ) 2 16 0 OK ! 6-50-13-5-276-278-279-27a-FFFF start diag on blue 2 76 1 1 flat 2 49 0 switch on chopper 2 51 0 enable chopper Chopper switched between 0 and 7.6 deg (2-53-1-6146) OK ! ramps on pixels HKdiag.dat not refreshed any more HK.dat still OK stop / start HKdiag does not help exit and restart sim DPU does not help call AM : lost link with blue DEC 2 89 0 reset SMCS2 OK ! 6 50 27 5 20 100 18F c8 333 ( bias_r 50mV bias_D 200mV large cap ) 2 16 0 not a lot of change 6 50 27 5 20 100 183 c8 333 ( bias_r 50mV bias_D 200mV small cap ) 2 16 0 OK we see the "oscillations" 6 50 27 5 20 100 183 0 333 ( bias_r 0mV bias_D 200mV small cap ) 2 16 0 less oscillations change parameters no PACK 1 6 103 1 to revive DPU status ? 2 15 0 OFF blue CREs 2 13 0 OFF blue DEC =========================== 20:20 cal sources stables T° atteinte chopper fixe sur source de calibration =========================== power on red 2 19 0 cre on red 2 21 0 637 27d Red DEC 1 DUMMY modified 638 27e Red DEC 2 DUMMY n.a. 639 27f Red DEC 3 DUMMY n.a. 640 280 Red DEC 4 DUMMY n.a. 641 281 Red DEC 5 DUMMY modified 642 282 Red DEC 6 DUMMY n.a. 643 283 Red DEC 7 DUMMY n.a. 644 284 Red DEC 8 DUMMY modified 645 285 Red DEC 8 3 modified 646 286 Red DEC 14 4 unmodified 647 287 Red DEC 5 4 modified 648 288 Red DEC 15 4 unmodified 649 289 Red DEC 8 4 modified 650 28a Red DEC 18 4 unmodified 651 28b Red DEC 11 4 unmodified 652 28c Red DEC 21 4 unmodified look at modified dummy pixels 6-50-13-10-27d-281-284-285-287-289-28a-28b-28c-FFFF start diag on red 2 76 1 2 lost HK diag lost DPU/DMC link power off DMC restart all 6 50 26 5 20 100 18F 0 0 ( bias_r 0mV bias_D 0mV large cap) 2 23 0 up going ramps 6 50 26 5 20 100 18F 29 0 ( bias_r 10mV bias_D 0mV large cap ) 2 23 0 down going ramps now to look at real pixels 6-50-13-5-285-286-287-288-FFFF start diag on red 2 76 1 2 6 50 26 5 20 100 18F 0 52 ( bias_r 0mV bias_D 20mV large cap ) 2 23 0 small effect 6 50 26 5 20 100 183 0 52 ( bias_r 0mV bias_D 20mV small cap ) 2 23 0 big saturation 6 50 26 5 20 100 183 0 10 ( bias_r 0mV bias_D XmV small cap ) 2 23 0 idem 6 50 26 5 20 100 18F 0 52 ( bias_r 0mV bias_D 20mV large cap ) 2 23 0 flat 6 50 26 5 20 100 18F 0 80 ( bias_r 0mV bias_D XmV large cap ) 2 23 0 flat 6 50 26 5 20 100 18F 0 0 ( bias_r 0mV bias_D 0mV large cap ) 2 23 0 2 77 0 look at unmodified real pixels 6-50-13-5-286-288-28a-28b-FFFF start diag on red 2 76 1 2 flat 6 50 26 5 20 100 18F 0 39 ( bias_r 0mV bias_D 10mV large cap ) 2 23 0 flat 6 50 26 5 20 100 183 0 39 ( bias_r 0mV bias_D 10mV small cap ) 2 23 0 no light ! ( power off => cal src and chopper off ) =============================== 21:30 OFF for the night =============================== 10/09/04 ******** morning BOL tests with SCOS2000 crash problem of yesterday caused by DPU remaining problem today : sometimes commands not executed by BOLC sometimes cured by DMC reset SMCS sometimes not at one occasion DMC CPU load increased to 32% history file will be generated and sent to AM CEA found that the DMC fans had not been connected => this could be enough to explain th eproblems =============================== restart in local mode 10/9 at 14:30 ( fans on ) Calibration sources: 2-68-0 2-91-0 2-70-1-c3500 (=80ohms) 2-72-0 2-93-0 2-74-1-e09c0 (=92ohms) 2 49 0 switch on chopper 2 51 0 enable chopper Chopper switched between 0 and 7.6 deg (2-53-1-6146) check HK : OK power on red 2 19 0 cre on red 2 21 0 diag HK list for red + which pixels + which supply scheme ========================================================= 637 27d Red DEC 1 DUMMY modified 638 27e Red DEC 2 DUMMY n.a. 639 27f Red DEC 3 DUMMY n.a. 640 280 Red DEC 4 DUMMY n.a. 641 281 Red DEC 5 DUMMY modified 642 282 Red DEC 6 DUMMY n.a. 643 283 Red DEC 7 DUMMY n.a. 644 284 Red DEC 8 DUMMY modified 645 285 Red DEC 8 3 modified 646 286 Red DEC 14 4 unmodified 647 287 Red DEC 5 4 modified 648 288 Red DEC 15 4 unmodified 649 289 Red DEC 8 4 modified 650 28a Red DEC 18 4 unmodified 651 28b Red DEC 11 4 unmodified 652 28c Red DEC 21 4 unmodified look at unmodified real pixels 6-50-13-10-286-288-28a-28b-28c-285-287-289-284-FFFF start diag on red 2 76 1 2 6 50 26 5 20 100 18F 0 199 ( bias_r 0mV bias_D 100mV large cap) 2 23 0 ramps ! add bias_R 6 50 26 5 20 100 18F cc 199 ( bias_r 50mV bias_D 100mV large cap) 2 23 0 6 50 26 5 20 100 18F 199 199 ( bias_r 100mV bias_D 100mV large cap) 2 23 0 change cap 6 50 26 5 20 100 183 0 52 ( bias_r 0mV bias_D 20mV small cap) 2 23 0 6 50 26 5 20 100 183 0 a4 ( bias_r 0mV bias_D 40mV small cap) 2 23 0 add bias_R 6 50 26 5 20 100 183 cc a4 ( bias_r 50mV bias_D 40mV small cap) 2 23 0 View HK diag DEC oscillation.bat file hkdiagjmgfm10092004A.txt same CRE parameters look at modified real pixels 6-50-13-10-285-287-289-27d-281-284-28a-28b-28c-FFFF start diag on red 2 76 1 2 saturation 6 50 26 5 20 100 183 0 52 ( bias_r 0mV bias_D 20mV small cap) 2 23 0 saturation 6 50 26 5 20 100 18F 0 52 ( bias_r 0mV bias_D 40mV large cap) 2 23 0 flat 6 50 26 5 20 100 18F 0 a4 ( bias_r 0mV bias_D 40mV small cap) 2 23 0 ramps 6 50 26 5 20 100 18F 0 199 ( bias_r 0mV bias_D 100mV small cap) 2 23 0 ramps with bias_R 6 50 26 5 20 100 18F cc 199 ( bias_r 50mV bias_D 100mV small cap) 2 23 0 View HK diag DEC oscillation.bat file hkdiagjmgfm10092004B.txt check with Matlab that files are OK power off replace card into frame connect all 4 supply groups to the FPU ( note J78 disconnected to avoid heating blue housing ) quick functional test ( repeat last set of measurements ) Calibration sources: 2-68-0 2-91-0 2-70-1-c3500 (=80ohms) 2-72-0 2-93-0 2-74-1-e09c0 (=92ohms) 2 49 0 switch on chopper 2 51 0 enable chopper Chopper switched between 0 and 7.6 deg (2-53-1-6146) check HK : OK power on red 2 19 0 cre on red 2 21 0 look at unmodified real pixels 6-50-13-10-286-288-28a-28b-28c-285-287-289-284-FFFF start diag on red 2 76 1 2 6 50 26 5 20 100 18F 0 199 ( bias_r 0mV bias_D 100mV large cap) 2 23 0 ramps ! add bias_R and change cap 6 50 26 5 20 100 183 cc a4 ( bias_r 50mV bias_D 40mV small cap) 2 23 0 View HK diag DEC oscillation.bat file hkdiagjmgfm10092004C.txt same CRE parameters look at modified real pixels 6-50-13-10-285-287-289-27d-281-284-28a-28b-28c-FFFF start diag on red 2 76 1 2 ramps 6 50 26 5 20 100 18F 0 199 ( bias_r 0mV bias_D 100mV small cap) 2 23 0 ramps with bias_R 6 50 26 5 20 100 18F cc 199 ( bias_r 50mV bias_D 100mV small cap) 2 23 0 View HK diag DEC oscillation.bat file hkdiagjmgfm10092004D.txt power off reconnect Scos2000 and programmable power supply 06-10-04 (at Garching EC-FM-AM) ******** FPGAs of the Red Base board have been replaced by version 2.0. No more spike in the HK Supply HK is OK there is a small error on the temperature measurement to be investigated later Diag HK in the current (5.016) version of the software: ID ID(hex) DEC Module Pixel 621 26d Blue DEC 1 DUMMY n.a. 622 26e Blue DEC 14 DUMMY 623 26f Blue DEC 2 DUMMY n.a. 624 270 Blue DEC 15 DUMMY 625 271 Blue DEC 3 DUMMY 626 272 Blue DEC 16 DUMMY 627 273 Blue DEC 4 DUMMY n.a. 628 274 Blue DEC 17 DUMMY n.a. 629 275 Blue DEC 1 4 n.a. 630 276 Blue DEC 14 4 631 277 Blue DEC 2 4 n.a. 632 278 Blue DEC 15 4 633 279 Blue DEC 3 4 634 27a Blue DEC 16 4 635 27b Blue DEC 1 NC 636 27c Blue DEC 14 NC 637 27d Red DEC 1 13 638 27e Red DEC 1 14 639 27f Red DEC 1 15 640 280 Red DEC 1 DUMMY 641 281 Red DEC 5 13 642 282 Red DEC 5 14 643 283 Red DEC 5 15 644 284 Red DEC 5 DUMMY 645 285 Red DEC 18 NC 646 286 Red DEC 18 1 647 287 Red DEC 18 2 648 288 Red DEC 18 3 649 289 Red DEC 18 13 650 28a Red DEC 18 14 651 28b Red DEC 18 15 652 28c Red DEC 18 DUMMY We can see ramps on all the real pixels but not on the DUMMY. Swapping the board shows that it comes from the new FPGAs. Note: param to have ramps on DUMMY pixels: 6-26-5-20-100-18f-0-c8 (50mV BiasR) Connected Francis' distribution board. There, we can see that pixels 1-2 are actually found at position 2-3. During the night, Eric found a problem in the VHDL code (all pixels position are offset by one, so the dummy pixels which is the last one is out of the packet). 07-10-04 (at Garching EC-FM-AM) ******** FM replaces 1 data and 1 supply board. Francis' distribution board is connected. MVb and FM checks the voltages: OK. Spare boards don't have labels. FM labels them. We still see the oscillation on the real pixels with the new supply boards. We have monitored the supply voltages and they are stable (some white noise but not correlated to the reset). (an e-mail has been sent with the graphs by AM today) EC burns new FPGAs (1 base and 1 data version 2.1) and replaces them in the board. Many commands are not interpreted. We have no idea why. We tried with two different boards: no change. We went back to version 2.0 and we also observed the problem. We finally found a quite stable set-up with : base FPGA v2.0 and data FPGA v2.1 and using a new cable between crisa and DEC. Note that we also introduced a 1ms delay between all the commands in the OBS. It seems that some command are still not interpreted but we observed only 1/120 failing. FM puts the original supply and data boards back in the rack. DEC ON, the Blue DEC temperature is increasing. We measured the voltage on the heater on the Blue DEC. We see 760mV on the voltmeter (and we read the same value in the hk). (EC unplugged the MPE EGSE) FM measures the harness (J78), on the heater side, only the pin 7 and 8 are connected (300kOhms) which is not nominal. while spectro are off, we disconnect the J78 cable, the temperature is decreasing. When we reconnect, the temperature is increasing again. FM makes a connection between the harness connector and the DEC connector only on pin 7 and 8. The temperature is decreasing. FM makes a connection between the harness connector and the DEC connector on all pins except the heaters (7-8-14-15). The temperature is decreasing EC replaces the red base board by the development one because we could not see the CRE_STATUS bit on the group I on this board. When we test it on the Base tester, it works fine. Once in the rack, it shows the same wrong behaviour. EC swaps the data FPGAs. At the same time, he puts new FPGA in the Blue base (data 2.1 and base 2.0). We still see the same problem on the red group I 08-10-04 (at Garching EC, FM, AM) ******** We put the FPGA that was in the Blue Base into the Red Base and test it on the tester : group I looks ok. When we put the board in the rack. It does not work anymore. Same with the blue base board instead of the red. It does not work either. FM takes the supply board of group I out of the rack to test it. EC modifies its base tester to be able to test CRE_STATUS on both group at the same time. 27-10-04 (at Garching EC, FM, BM) ******** Integration of new (modified) supply boards on group 1 and 3 28-10-04 (at Garching EC, FM, BM) ******** Tests with the grating : P=250, I=2710, D=5, Outlimit=3331 (222 mA) 45 deg. = fffe0 -5 deg. = -1c718 Measurement of the range : ~997182 but doesn't seem to touch hard stop at full range position. P=250, I=2710, D=5, Outlimit=452f (300 mA) Measurement of the range : ~1004199 Change of PID parameters : P=39b, I=c350, D=e ==> Parameters from QM qualification campaign at CSL Regulation OK. Outlimit=5c3f (400 mA) Measurement of the range : ~1012574 Tests with PID parameters : P=3e8, I=c350, D=12, Outlimit=5c3f Regulation good and in specifications File : gratingcold_281004d.txt Measurement of the range : ~1012813 Modification of ADC range on data 1 and 3 boards => +0.5V Tests with the chopper : Trials with different PID parameters to optimise and increase the duty cycle. New parameters are : Ki=0x4618560 Kf=0x9c4 rate=0x148 Kicurr=0x668a0 File : choppercold_281004j.txt Modified the CS controller code to have a better accuracy. Tested on resistor : OK 29-10-04 ******** 9-11-04 ******* DMC back in CSL overnight test with 2 DEC sending 256 readouts/sec. Nothing else running (except SimDPU of course). The communication remained stable for 16 hours. 10-11-04 ******** Measurement of excitation amplitude for inductosyn QM on a resistor of 18.5 Ohms : 800 mV/pkpk 214 mV/rms Setup DMC in nominal mode and load: 3 PC to simulate DPU, and 2SPUs Bolc Sim, complete DMC start link with SPU 2-87-1-1 2-86-1-1 start DEC 2-12-0 2-19-0 6-50-26-2-20-100 6-50-27-2-20-100 2-24-0 2-14-0 2-21-0 Send data to SPU 6-50-23-1-0 6-50-24-1-0 BOLC link (send readouts) 2-89-0 2-33-1-09020002 Cal Src 2-68-0 2-91-0 2-70-1-c3500 (=80ohms) 2-72-0 2-93-0 2-74-1-e09c0 (=92ohms) Temp Sensors 2-95-0 Grating 6-50-32-1-1 2-38-0 6-50-15-3-250-2710-e 2-40-0 2-44-1-0 2-42-1-e000 start hk diag 2-76-1-0 sequence 6-60-gratstepslong 2-10-1-1 2-5-0 The communication with Both DEC and BOLC was lost only a few seconds after BOLC started to send readouts ! Start again the same test with first starting only communication on SMCS 2 (DECs + BOLC) OK for 15 minutes Start all the other functions Did not start the sequence yet BOLC link lost (note, packets are not transmitted but no disconnection has been detected by OBS) chip reset, link ok After 4 hours, 3 links are lost at the same time. chip reset => communication started again 12-11-04 ******** With JTAG connected, we observed that, when the transmission are stopped, a transfer was in progress on all of the links but non of them has reached the EOP. The 3 receive FIFO of the SMCS are full (in the ISR. (note: that is the ISR value of the moment where the communication where lost). After clearing the ISR, FIFO is still full on BOLC link. Note: I have added counters to check that every interrupt is reported to the high level software. It seems it is OK. Note: when the CPU load is very high (around 80%), nominal hk is not transmitted anymore to DPU but commands and PACK are going through. Other test, BOLC link lost after 2 hours: Current address counter is at the end of a packet. The ISR has been signaled, the smcsDrv has been signaled but the BolRec is still waiting for the event. The only explanation is that Virtuoso has lost the event !!!! Is it due to a too high CPU load ? Now, we will try to swap smcs1 and smcs2 links and make a long duration test over the week end. NOTE : big-little endian must be inverted too ! autoreconnect too ! Procedure to start the OBS from the EZ-ice: emulator reset F4 wait 15 sec F4 change PMBANK1 to 0 Load onboard.exe change PC to 8032 F4 start comexe start simDPU for ASW At 19:20, I could start a test with SMCS1-SMCS2 swapped. CPU load is 62% which is equivalent to what they have in Garching. 16-11-04 ******** The test is still running !!!!! but, after 1 hour, SPU red link has been lost (it was on smcs2) after 23 hours, SPU blue link has been lost (it was on smcs2) CPU load was then 50% Rewired nominally. Back to OBS 5.019 with nominal link affectation. Disabled EDAC (in hk.c and in smcs.s) We lost the link with DECs only a few minutes after startup. We build a small test software (option 8 in 'testSpacewire') to send packets to 3 links of smcs2 at the same time. On the crisa board, a test software is also running, simply receiving packets on all links. Started a test at 15:20. At 16:20, it is not transmitting but I don't know what happens Started a new test at 17:08 After a while (don't know exatly when), the third link stopped and a few minutes later, first link and second stopped at the same time. We will have to make the same test on smcs1 17-11-04 ******** OBT probs: HK diag at 1 KHz: 1c5, f2 We see increment of OBT between 520 and 543 instead of 512 power consumption: SUSW only : 580mA Started a test for connection loss: same as yesterday but on smcs 1. Started at 17:55 18-11-04 ******** At 9:22, all 3 links are still running. So it loos like we really have a problem on smcs2. Now working on the 'chopper/FW not moving when timing FPGA in photometry mode' Commands: 6-50-29-6-0-0-0-1a-977602a-33 2-10-1-4 2-11-0 2-59-0 2-66-1-0 2-66-1-1 for hk pos sensors photo wheel (563-569) 6-50-13-3-233-239-ffff If you send many times 2-10-1-4 and 2-11-0, after a while, the analog hk is wrong Started a new test in which we sent only 3 times the 2-11-0 (write fpga parameters) and we lost the ISR !!! There are already 7 nops between each write instruction !!! OBT: On the OBT cable, on the connector close to the OBT box, we desoldered pin6 and soldered it on pin5. Only one signal of the differential signal was sent to DMC. The other one was floating. Now OBT counter is correct (tested in hk diag and in photo packets). Placed the new FPGA in EM1. OBT looks ok. We can switch-on DEC (but, we observed that the communication is lost only a few seconds afterwards !!!) Grating makes a strange noise FW moves but free running. It seems that the output is written once and then, it is free running. CS are working fine. When writin FPGA parameters many times, it shows the same problem as previous version (ISR lost or HK index lost) 19-11-04 ******** Previous FPGA back in MIM1 with a small fan. Started a test at 12:35. Note that we lost the communication with DECs very fast once again. At 15:25, it seems that MIM1 is still working fine (analog hk + ISR OK) At 15:51, started a new test at nominal CPU load. Note : At startup, CRE_ST_1 and 3 are 88 CRE_ST_2 and 4 are 8C After initialization: CRE_ST_1 and 3 are 8089 CRE_ST_2 and 4 are 808D !!! After a master reset, it is still the same ! It seems it is a problem on the mezzanine interface: bit 19 stays at 0 all the time 22-11-04 ******** All smcs2 links have been lost. Mim FPGA is still ok (ISR OK, analog HK ok). Francois has cleaned the Crisa board (ultra-sonic cleaning). Now, bit19 is functional. Once the SMCS2 warms up again, bit19 goes back to zero again !!! With the fans on, it works fine again. Pierre has inspected the board but did not find anything. 23-11-04 ******** Today, the power supply does not work !!!! Fuse broken (twice). We use another one. synchro test between DESTRUCTIVE-READOUT and READOUT_COUNTER ( the readout-counter has to be at 0 at the time of the destructive-readout ( start @ 17h00) 24-11-04 ********* 15h00 ( after 22 hours of connection ) the readout-counter and the destrucive-readout are still synchronised. Latest MIM1 FPGA version in EM1. ISR = 8192Hz OK SWON DECs OK MASTER RESET DEC OK FW PHOTO OK GRATING NOK (inductosyn readout errors) OBT COUNT on internal sync OK OBT COUNT on blue DEC sync OK OBT COUNT on red DEC sync OK OBT COUNT on BOLC sync OK 25-11-04 ******** Testing FPGA reset when writing FPGA parameters Added nops before and after first write : NOK without nops : NOK only Ctrl reg : OK ? Phase Inc + Ctrl reg : NOK Phase Inc only : NOK Set PLL free running before and back after : NOK Reset PLL after : NOK All but phase Inc : OK 26-11-04 ******** FPGA 7.02 in. 2 corrections in the software : one to detect the period jumps and the other to detect the complete turn jumps. I had many troubles when adding 2 integers at the end of the grating structure (The inductosyn readout was then completely wrong). I moved these 2 entries in the Mim structure and it went fine. I could not understand what was wrong (compiler bug ?) At 17:34, started a test without any fans to test the MIM FPGA thermal behaviour. 29-11-04 ******** Stopped the test at 9:38, FPGA shows the same behaviour as friday evening. the 'chopper does not move in photo mode' problem: We had to write 7-8 times the parameters before it fails The chopper does not move but, at that time, the FW could move The parameters we are using: 0-0-0-1a-0x0977602A-33 (internal sync) ISR is still at the right frequency, analog hk is working. OBT counter is incrementing correctly => we receive the sync. The chopper can be commanded in open loop (no need for a sync) Started a new test, same config. I could send the 'set timing FPGA params' 50 times without any problem. Once I re-send the 'synchronize on det' on BOLC (2-10-1-4), the hk analog stopped (including chopper pos that was blocked at 4887) The same happened again later. Note that both commands call WriteFpgaParameters. The only difference is that with the 'set timing', the synchro source is 0, while with 'synchronize on det', the sync source is 5. So, it seems that there are 2 types of crashes: 1. Looks like a problem related to synchro but I don't know what 2. When writing the synch Now, I modify the default params such that sync src is 5 (so, 2-11-0 and 2-10-1-4) are equivalent. I could reproduce twice a wrong behaviour with: 2-11-0 2-10-1-4 2-11-0 2-10-1-4 Note: in 'set timing param', there was a filter on sync src selection (& K_BMASK_TIMING_ACCESSIBLE_BITS) that should have been applied to ctrl reg. I corrected it but it does not help. I think the problem happen when we change the sync src. Once I was not able to crash it, going to a blue dec sync src and then back to bolc sync src crashed it immediately. 30-11-04 ******** Tested the Cal Src ripple: both sources on, open mode, voltage = 0x1000 (1.25V), both resistor = 56 ohms CS1 shows a peak to peak noise of : 16 mohms CS2 : 16 mohms both sources on, open mode, voltage = 0x2800 (3V), both resistor = 82 ohms CS1 shows a peak to peak noise of : 7 mohms CS2 : 6 mohms both sources on, open mode, voltage = 0x2800 (3V), CS1 resistor = 99 ohms, CS2 = 82 ohms CS1 shows a peak to peak noise of : 5 mohms CS2 : 5 mohms Note that when MIM2 is not in the rack, the processor stops (reset ?) when we switch on DECs !!!! 10-01-05 ******** Checked the capacitor select swapping: this is only the mapping in the SUM between the software names and the harness names that is wrong. Checked the blue/red dec relay swapping: everything is now according to the plans. The first wiring was incorrect. Some capacitors on the LVDS have been removed (ask EC) 11-01-05 ******** Replaced the red DEC relay on the PSU by a transistor. We made a test with 1000 switch on/off of the red DEC: 4 switch on did not lead to a connection. The communication with DPU has not been lost ! which is a very good improvement. On the red DEC, we have put the capacitors on the LVDS back (ask EC) On the blue DEC relay, we have added a self-inductance. We made a test with 700 switch on/off of both DEC: once, there has been an error on both link. They both connected, transmitted 250 packets and then, an error occured on the SMCS2 (error signaled on the 3 links) 12-01-05 ******** Ran this test overnight: ChkClearReport() Repeat(5000) #switch on both DECs and send default parameters to both Trig(12, 0, 0) WaitTime(1, "") Trig(19, 0, 0) Write2(26, 0x40, 0x100) Write2(27, 0x40, 0x100) WaitTime(10, "") ChkBit(204, 19, 0) ChkBit(201, 19, 0) WaitTime(4, "") Trig(24, 0, 0) WaitTime(6, "") ChkBit(204, 19, 0) ChkBit(201, 19, 0) #switch off both DECs and send default parameters to both Trig(13, 0, 0) WaitTime(1, "") Trig(20, 0, 0) WaitTime(3, "") ChkBit(204, 19, 1) ChkBit(201, 19, 1) ChkReport() EndRepeat() ChkReport() score : 13060/13080 (the test was not finished) Failures: 6x After the SET_PARAM_BOTH, the red DEC connection was lost 3x After the SET_PARAM_BOTH, both DEC connection were lost 6x After the SET_PARAM_BOTH, the blue DEC connection were lost 1x Before the SET_PARAM_BOTH, both DEC connection were lost 01-03-05 ******** The EM board is back from CRISA. The mezzanine has been upgraded (new PCB). At crisa, a test has been run where the communication on the SMCS2 was tested for 15 hours without a single failure. Note that they had the CPU board alone with a very noise-insulated power supply. Run the communication test set-up as run at CRISA (here, the CPU board is in the EM rack). After 5 minutes, the communication stops ! Tried 4-5 times, always the same. New FPGA 7.1 in MIM1. with OBT (131072 Hz) connected, ISR frequency is 16384. When we disconnect the OBT, ISR frequency is slowly decreasing to 16324 and then is not stable (slow oscillation) Reconnected the OBT, the ISR frequency goes back to 16384 again. Reconnected the mechanisms and the red DEC. run func_grat.txt OK, the grating is still noisy but functionnaly it is ok. run func_chop.txt OK, the chopper does not have good controller parameter byt functionnaly it is ok. run mimfpga.txt It seems that the fpga does not react correctly to the writing of its parameters: the chopper is not moving every time. At 18:18, started a 1355 test on the SMCS1 (3 links transmitting from the PC, and the crisa board receiving). After a few minutes, all links are lost !!! (it has never happened before on this SMCS chip !) 02-03-05 ******** run mimfpga.txt (switching from spectro to photo and changing the synchro sel while moving the chopper). The analog hk is not working fine (all the mux addresses are correct but mux0-2 sequence that is corrupted). Nicolas explained that it is not required to write phase_inc in the PLL if we stay within nominale frequencies (for BOLC 1 - 40 Hz). When we removed the writing of synchroSourceSel, everything worked fine !!! 03-03-05 ******** When in photo timing mode, the DEC HK is messy. When going back to spectro mode, everything is fine. I think that the HK is messy because the packets are not transmitted correctly (We receive 336 packets every 2 sec). We still observe jumps in the inductosyn readout. When looking at the signals out of the inductosyn, we see the ripple clock with variable length happening at variable time (it is seldom in phase with the 16 bit counter overflow !) We tried to put another AD2S80, but we observe the same behaviour. With the CRISA board in an external box and with an external power supply, we try the 1355 communication test program. First we connect to the 3 links of the CPU board. Set-up: Elix Power supply PC ground connected to the PS GND 0V of 15V connected to 0V of 5V EZ-ICE PC not grounded After a few minutes, it fails !! Same test with EZ-ICE PC grounded After a few minutes, it fails !! Note : always link1(PC) is stoped first 04-03-05 ******** Same test VCC SMCS232 measure : No communication : Noise < 100mV pp With communication: Noise < 100mV pp New test : data trnsfer rate (1000 packets every 4 sec insteed of 1000 packets every 0.7sec) after 1/2 hour it fails (link1 first) New test : CPU board without mezanine connected to both AVM DECs ( 256 Hz) and BOLC SIM (40 Hz) Test start at 15h00 07-03-05 ******** 10h30 Communication still running both DECs and BOLC still connected. Test stopped by EC. on DEC base AVM chassis connected to the groud ( no a short connection ) but line 3(internal shiel) on space wire cable not connected to ground 11h30 Mezanine back on the board mezanine connected to both AVM DECs ( 256 Hz) and BOLC SIM (40 Hz) Test start at 11h30 14h10 still running 15h30 still running , test stopped by EC New test CRISA board back in EM rack connected to DECs AVM and BOLC SIM when connected => transfert ( not com) lost with J02 - AVM DEC 1 inverting J02 with AVM DEC 0 when connected => transfert ( not com) lost with J02 - AVM DEC 0 inverting J02 with BOLCSIM Test stopped by EC EC and AM : try to boot the CPU bord => FAILED New test CRISA board back in EM rack connected to DECs AVM and BOLC SIM mezanine connected to both AVM DECs ( 256 Hz) and BOLC SIM (40 Hz) Test start at 18h00 08-03-05 ******** 09h00 1 link lost( no more connection ) on 1 DEC AVM !!!! New test : EC : boot cpu board => FAILED 09-03-05 ******** Tightened the 2 FPGAs on the Crisa board => SUSW is now working again. The set-up goes into our office. First set-up: The PC EGSE is connected to link 1 of the rack. No other connection (Dec blue base is not inserted) => DPU-DMC connection is OK (tested for 2 minutes) Then, connected red DEC to J06. => OK, dectest.txt => 10 on/off => OK Then, connected blue DEC to J05 and BOLC sim to J04 => OK, dectest.txt => 55 on/off, one failed on both DEC (but the anti-virus started at that time !) Started a new test At 17:50, switched on both DEC and BOLC, to see if the connection is stable 10-03-05 ******** 9h00 connection still running ( both DEC + BOLC ) 17:00: still running !!!! 23h MIM1 mezzanine back in on MIM1 and BOLC sync is now connected. at 17:30, start a new test 11-03-05 ******** At 14:30: The test is still running, all links are still connected !! (21h) Connected the EZ-Ice PC, the 3 links of the mezzanine board are connected to the EGSE PC. The 3 links of the CPU board are not connected. At 14:52, started a test At 20:00, it was interrupted when I tried to remove the EZ-ice restarted on friday at 20:02, this time, I could remove the Ez-ice 14-03-05 ******** Still running. 15-03-05 ******** 9h30 Still running : 84Hours !!!!!!!!!!!! world record Test stopped by EC DECMEC modification by PD : DSP BOARD fan removed DEC switching on relays modification ( MOSFET -> Relays with protecting self ) AD2S80 changed ( actual replaced by the one with the best accuracy ) 14h00 : start script dectest.txt with these new modifications It fails , reddec don't start verification of the modifications done. 14h30 : problem solved ( bad connection ) 16h30 : stoped by EC result : 1601/1608 16h40 : start Again ( script modification : BIP when failed ) 16-03-05 ******** 15h45 socre : 19733/19794 15h46 : no more command executed !!!! neither ON , neither OFF One dec ( dec blue) still ON and connected ( no space wire disconnection ) connection between DEC/MEC and PC KO ( HK no more refreshed ) 16h28 EC + JMG DEC/MEC sitch OFF DEC/MEC switch ON start SimDPUv26script start destest script switching DEC B / DEC R on/off running again. 17-03-05 ******** 11:00 score : 15879/15906 Connected the mechanism box again to run all the tests concerning the chopper. Tested the chopper with an inductance = 0x151. It has no influence (note that, in warm, it should ne make any difference; this is what we wanted to test after the small modification we have made) a_mec.txt : failed, the chopper moves whatever is the synchro source func_chop.txt : OK func_test.txt : OK 18-03-05 ******** Both the grating and chopper warm parameters are now by default in the code Corrected SPR 0250-0285-0286 Back in the basement !! Test setup: PC EGSE 3 links connected to 3 links of CPU board on the mezzanine: J04: BOLC J05: a DEC from AVM J06: red DEC (from EM) We run the test software to listen on all links. After a few minutes, link0 is lost and the BOLC link never got the connection Same set-up with BOLC and both DECs off. link0 is lost after a few minutes !! 21-03-05 ******** Mounted the test setup on a wheel-table in the basement. Same grounding. Both DECs are currently out of the box for checking by EC. The test fails after a few minutes. Moved the same setup in our office. The test failed after a few minutes too. EC tightened the FPGAs in their sockets again (on the Crisa board). The test failed again after a few minutes. BM re-tightened the FPGAs in their sockets. (Note that in the mean-time FM took out most of the date boards). The test failed after 10 minutes (a little longer than before). Started a new test again at 11:50 After 1 hour, link2 has been lost Dismounted the FPGAs sockets on the CRISA board. 2 pins seem to be stuck. PD unstuck them. Note that we are not sure this is a permanent repair. It seems very fragile and they can stuck again. BM tightened very loose by hand. At first try, the SUSW did not boot at all. The cooler was not tightened enough. BM retightened 3/4 turns. 16:44, started a new test. After 40 minutes, it failed ! Swapped cables on J02/J03. Restarted at 17:29 Still running at 18:49 (removed JTAG) In the morning, link2 is lost 22-03-05 ******** The chopper has been removed and sent to MPIA. Started the same test to see if it is always the same link that fails. link2 failed In these 2 tests, link2 was connected to J02 Titghtened the FPGA again. Started a test at 10:45 Still running at 17:39, stopped it to test the SMCS2 Note: a blue soft matter has been extracted from J01. It is still in all other connectors. Connected 3 links on SMCS2 links. Started a test. After a few seconds, link0 is lost (this is the one connected to J06) Tightened again. Started a test. After a few seconds, link0 is lost (this is the one connected to J06) Added a second screw on J05 and J06. Same result. 24-03-05 ******** Started the test on SMCS2 links. ( DMC is cold ) link0 is lost (this is the one connected to J06) 25-03-05 ******** I want to run mimfpga.txt with the old version of the software (the one that is still writing in sync_src_sel) BOLC is off and none of the DECs are in the rack (this means that there is no sync signal). Everything looks fine ! Once BOLC and DECs are connected, we can see that it fails again. Now, we modified mimfpga.txt such that, in photo mode, it writes 0x23 in the ctrl reg instead of 0x33. Then, everything works fine again. So, it seems that the problem is not related to the writing in the register but is really related to the synchro source selection itself. Put new FPGA MIM 8.0 vAD2S80 on MIM1. At first glance, it looks OK, we did 2 homings and a few moves while monitoring the inductosyn readout as provided by the FPGA and we haven't seen any jump. It should be tested toroughly before removing the software patches. 30-03-05 ******** DSP QM board in EM DMC ( PD ) 16h45 power ON DMC conso 640mA unable to load completly the OBS. 17h00 communication test ( with ADSP 21020 Emulator and the 3 links receiver 1-2-3 motherboard) test1 connection with link0 dirrectly lost test2 connection with link0 dirrectly lost 17h10 communication test ( with ADSP 21020 Emulator and the 3 links receiver 4-5-6 mezzanine) test1 connection with link1 ans link0 dirrectly lost test2 connection with link0 dirrectly lost 31-03-05 ******** Tested again with the QM board in the EM1 rack. It fails after a few seconds. With the QM board in its box => it works fine for 1 hour Tried with 2 different power supplies => it works ok QM board back in the rack (the rack contains MIM1-2-3 and a few DEC boards) Removed MIM1-2-3 from the rack Link 0 is lost after a few seconds Connected directly the power supply instead of the DC/DC (the PSU board is still in the rack). Link is lost after a few seconds. AGND and DGND connected together on the backplane very close to the CRISA board. Link is lost after a few seconds. PSU board removed , +15 and + 5 provided directly at the backplane Link is lost after a few seconds. DSP BOARD back in the small rack => it runs ( IRQ 3 loaded by opto. (=> 0 on the IRQ 3 line) ) Power consumption when DSP board alone in the rack and SUSW running (no PSU, no MIM and no DEC) => 830mA on the 5V When SUSW is halted by EZ-ICE : 400mA The rack without the DSP board: 0mV We usually observe the same consumption when the board is in its box. Note that we observed 560mA on the 5V in the box but we have not checked if SUSW was running at that time We had a lot of problem to boot the board when in its box. Finally, we discovered that +15V was also connected to -15V return. 01-04-05 ******** 09h30 Inspection of the "small" dsp box connector : +15 missplaced ( -15RTN ) => correction by PD et EC. 10h15 communication test started ... OK test stopped by EC . 10h16 communication test started again ... OK 10h30 DSP board back in the EM box. communication test started ... KO after a few seconds 10h45 Switch cable link0 ans link2 on DSP board communication test started ... KO after a few seconds (link0 failed first) 10h45 Switch cable link0 ans link2 on PC board communication test started ... KO after a few minutes (link0 failed first) 11h34 Test with only link1 and 2 (link0 disabled in testSpaceWire.exe) 14:27 still running, interrupted to try with all links with another board/PC Also tried with another PC but the behaviour was exactly the same (link0 lost !). Back to the first PC Monitoring of 5V supply voltage in the rack: - we can see the SMCS interrupts in the 5V (200mV peak to peak) Monitoring of 5V supply voltage in the box: - same level as in the rack Note: link0 was lost while DSP board was in the box !! 05-04-05 ******** We now have another power supply with an OVP on 5V and the -15V in the small box. Note that, according to crisa ICD, the -15V is not mandatory (not used) Test1: PC EGSE with 3 links all connected to the mezzanine board. Crisa board in its box. EZ-ICE connected. The shunt that is used to measure the current is in the '-' line. The 0V is not connected to ground Started at 11:34 AM Power consumption on 5V : 990mA when 3 links running At 14:14, it is still running (2h40) Test2: PC EGSE with 3 links all connected to the mezzanine board. Crisa board in its box. EZ-ICE connected. The shunt that is used to measure the current is in the '+' line. The 0V is connected to ground Started at 14:45 Power consumption on 5V : 1A when 3 links running JMG measured the 5V on the board and, at the same time, the board reset. (5V was 4.7V during operation) Removed the fuse on the 5V to reduce the voltage drop. We now have 4.78V on the board 4.83V on the backplane => 0.17V in the wires + connector (at the backside of the box) Replaced the wires => we now have 4.9V on the board About the reset of the board when we measure the 5V: - it depends on what is the first pin you touch (ground or 5V) - it resets the board if the voltmeter is close to the power supply lines of the board; if it is on the other side, it does not reset. Removed the connector, and twisted the +15V, -15V, 0V Tested the communication for 3 minutes => OK Test3: PC EGSE with 3 links all connected to the mezzanine board. Crisa board in the rack EM1. EZ-ICE connected. In the rack, we have only: - the crisa board - a power supply connector (replacing the PSU board) with +5V, +15V, -15V (the same connections as in the Test1) 5V on the board is 4.9V (the same value as in the box) Started at 16:20 Link0 lost at 16:55 Test2 started again at 17:46 Still running at 9:20 06-04-05 ******** Test4: PC EGSE with 3 links all connected to the mezzanine board. Crisa board in its box. EZ-ICE connected. We introduce white noise (amplitude = 4V peak to peak ) on each of the pin of the backplane connector to see if we can crash it that way. Note, we use a 1Kohm resistor between the noise generator output and the backplane connector. With the white noise : no crash, no communication loss With a square 4V 11KHz signal : no crash, no communication loss With a 'square' 4V 15MHz signal : no crash, no communication loss 3 hours of continuous operation 30mV between '-' of the power supply and the ground plane. 4.9V between - of the power supply and + on the board 4.87V between ground plane and + on the board Test3 again: PC EGSE with 3 links all connected to the mezzanine board. Crisa board in the rack EM1. EZ-ICE connected. In the rack, we have only: - the crisa board - a power supply connector (replacing the PSU board) with +5V, +15V, -15V (the same connections as in the Test1) 4.86V between ground plane and + on the board Monitoring the 5V on one of the drivers of the link that is not used on the DSP board: We see 400mV peak to peak when an SMCS interrupts occurs When monitoring the 5V on the mezzanine board, we see 1.5V peak to peak when an interrupt occurs! Otherwise the noise level on Vcc is < 200 mV p/p ( high frequency, spikes, clock ) test ran for 1 hour then link 2 was lost restart at 17:28 crashed at 18:03 07-04-05 ******** Inserted new MIM FPGA in MIM1 (version 8.02) Test 07-04-05-01: QM board in the rack with PSU, MIM1, MIM2, MIM3. DPU-DMC is the only connection Tried to upload the OBS. The communication DPU-DMC is lost during the upload of the OBS ! Test 07-04-05-02: QM board in stand-alone box. DPU-DMC is the only connection. Upload OBS : OK Burned EEPROM : OK Received HK during 2 hours Test 07-04-05-03: QM board in the rack with PSU only. DPU-DMC is the only connection Unable to start the OBS (or to get the communication with DPU) (Tried 2x) Test 07-04-05-04: QM board in the rack with external PSU. DPU-DMC is the only connection OBS started, communication with DPU: OK (Tried 3x) (Tested only for a few minutes) Test 07-04-05-03 (again): Now, it worked 3 times in a row ! Test 07-04-05-05: Same setup as test 03 with MIM1 inserted. FAILED: can not start OBS or communication with DPU FAILED Test 07-04-05-06: Replaced MIM1 with FPGA 8.02 with an old MIM with FPGA v7.01 the OBS can be started and the communication with DPU is OK. When I switched on blue DEC (that is not in), the communication has been lost The EM board has been received Test 07-04-05-07: Replaced QM DSP board by EM board ( in card cage with EM PSU and MIM1/FPGA V8_0 The OBS can not be started from EEPROM. Note that I have noticed new labels on the EEPROM or PROM ? Uploaded complete software and burned EEPROM. the OBS can be started and the communication with DPU is OK. When I switched on blue DEC (that is not in), the communication IS NOT lost (it was lost everytime with the QM board). Test 07-04-05-08: Start from EEPROM : OK test 8/4/05 JMG measuring interference level SMCS test started 14:30 3 links EM board internal PSU no MIM on main board : DC/DC frequency = 400 mV p/p SMCS frequency = 700 mV p/p install extender in MIM1 location : no impact 15h30 still running stop to install MIM1 on extender restart test SMCS usual procedure note : ElectroStatic Discharge ( human body model : me ) on card cage has no impact - test continues to run 16h30 still running ( no reset no disconnection ) stop to install QM board ( remove MIM1 and extender ) testsmcs started at 17:00 link 0 stopped transmitting after 21000 packets ( link loss, not a board reset ) QM board clearly more problematic than EM board IN SAME ENVIRONMENT preparing to demonstrate resets stop to install extender and MIM1 test 1 : using EZ-ICE loop writing to 20040046 and reading from 2004004A no reset test 2 : testsmcs 3 links ( 4-5-6) link loss after a few seconds or minutes ( tried 3 times ) remove extender and MIM1 : same problems replace QM with EM - installed MIM1+MIM2+MIM3 test smcs 3 links stops after 80000 packets remove MIM1-2-3 do not touch EM DSP restart test smcs 3 links at 17:15 12/4/05 JMG *********** EM DSP in card cage with nominal PSU no MIM has been running testsmcs since 11/4 at 16:15 Vp = 28V; Ip = 0.29A Vcc on board = 4.95V Vcc noise on backplane = 200 mV p/p at DC/DC frequency Vcc noise on DSP board < 100 mV p/p 10:00 change links to 1-2-3 restart test smcs stopped after 200000 packets restart test smcs links 1-2-3 one link stopped after 100000 packets dismount EM DSP and install QM DSP ( nominal PSU no MIM ) Vp = 28V; Ip = 0.27A Vcc = 4.95V on board restart test smcs ( 4-5-6 ) 2 links lost after a few seconds restart test smcs ( 4-5-6 ) 1 link lost after a few seconds briefing with Crisa ******************* connect links 1-2-3 start measurements as requested measurement pin 9 link 3 and Vcc wrt local ground ( not chassis ) power on Vp = 28V Ip = 250 mA Vcc and link output : NOTE : HARDCOPIES are in Atlas\Pacs\Tests\VisitCrisa12-13April2005 noise without EZ-ICE = 50-100mV p/p hard copy "print00" "print01" noise with EZ-ICE = more HF but roughly same amplitude hard copy "print02" "print03" load and start test smcs on links 1-2-3 link 2 ( PC ) lost after 20000 packets hard copy "print04" : Vcc shows 300 mV p/p at 10 MHz edges ( SMCS clock ) connect to link 0 start test smcs link 2 lost after 100000 packets ( still exchanging nulls ) output of link 3 driver hard copy "print05" stop and install EM board ************************* probes : GND ( solder wire on decoupling capacitor ) ; Vcc ; on solder side, FPGA regulator input hard copy "print06" and "print07" before connecting EZ-ICE connect EZ-ICE hard copy "print08" and "print09" noise in the range 50-100 mV ( subjectively slightly less noisy than QM but scope ground not exactly at same place ) driver output has less HF noise but shows spikes at DC/DC frequency test smcs links 1-2-3 hard copy "print10" ( subjectively we do not see any 10 MHz on Vcc ) still running after 300000 packets stop and cut front panel link ***************************** hard copy "print11" "print12" without EZ-ICE - seems to be much more DC/DC noise 100 - 200 mV p/p connect EZ-ICE hard copy "print13" "print14" test smcs started links 1-2-3 about 1 V of common mode interference on link output ( Crisa has seen same order of magnitude during tests ) one link lost after 50000 packets stop and connect on Vcc at other side of board ( Vcc pin of driver ) hard copy "print15" "print16" connect EZ-ICE "print17" "print18" start smcs test "print19" "print20" with links running Vcc noise < 200 mV p/p common mode (?) noise on link at DC/DC frequency one link lost after 50000 packets stop and connect GND to front panel on QM, at 2 places ( main board and mezzanine ) power on without ICE hard copy "print21" "print22" connect front panel to frame "print23" connect EZ-ICE "print24" lost 1 link after 10000 packets "print26" "print27" connect cables to extension board test smcs links 4-5-6 still working after 200000 500000 packets stop to add filtering capacitors ******************************** 2 x 22 µF ceramic multilayer 10V in parallel with existing filtering capacitors near SMCS1 on main board connect cables to links 1-2-3 connect probes hard copy "print28" connect EZ-ICE hard copy "print29" start smcs test on links 1-2-3 hard copy "print30" ( traces of 10 MHz ) lost one link after 50000 packets Vp = 28V Ip = 0.29A Vcc = 4.95V restart test at 16:50 "print00" "print01" "print02" on second disquette ( look at noise shape on Vcc ) still running after 100000 150000 200000 350000 still running at 17:05 stopped for connecting a second PC ( exercise all 6 links ) *********************************************************** 17:20 start with 6 links active try to display Vcc interference in differential mode 30000 100000 200000 load : 800 packets of 256 words /s x 4 bytes x 6 links ~ full speed ( 7 Mbps on all links ) Vp = 28V Ip = 0.29A 17:40 still running switch on/off the lights in the room : no effect switch on/off a lamp on the same mains plug : all links stopped insert MIM1 *********** start again test smcs on all 6 links ( no scope ) Vp = 28V Ip = 0.4A 30000 100000 300000 insert MIM2 MIM3 **************** start again test smcs on all 6 links ( no scope ) Vp = 28V Ip = 0.65A lost link 0 after 40000 packets links 4-5-6 continue at same speed backplane Vcc : 4.91V stop and restart at 18:20 50000 packets and counting 160000 packets : link 0 lost - links 4-5-6 still running removed ICE probe we leave the system ON for the night - links 4-5-6 running full speed - links 1-2 reduced speed In the morning, the 5 links are still running stopped at 08:30 to install EM board stand alone as a test tool 13/4/05 AM JMG ************** Crisa measurements at Vcc on QM board stand-alone ( linear bench power supply ) measure difference between Vcc and GND ( 2 channels ) oscilloscope return connected to chassis chassis connected to GND at power supply GND connected to Earth at power supply "print04" without EZ-ICE connect EZ-ICE start smcs test on links 1-2-3 one link lost after 3000 packets change probe to monitor link 1 output connect EZ-ICE start test program again upper trace : Vcc ; lower trace : link Vcc noise < 50 mV when links stopped, 100 mV when EZ-ICE connected try to remove EZ-ICE connection when links are running : failed try again : success "print08" without EZ-ICE stop to connect links 4-5-6 change probe to monitor link 6 output oscilloscope return connected to chassis connect EZ-ICE start test program again upper trace : Vcc ; lower trace : link links running links stopped when removing EZ-ICE "print09" without EZ-ICE but links not transmitting verification contact resistance between LVDS connectors shells and stiffeners ( in some past QM measurements Crisa got 3 ohms on one connector ) here all connectors have contact resistances < 100 mohm look for a pattern in link loss ( always same or not ? ) ******************************************************** config : QM board stand alone, connect cables as follows : PC link cable DSP link 0 A 3 1 B 2 2 C 1 10:14 connect EZ-ICE and start test smcs ( scope not connected ) 200000 packets connect oscilloscope ground to front panel link 1 lost restart link 1 lost after 120000 packets restart 300000 packets link 1 fails again place QM DSP back in EM1 card cage ********************************** ( changed DSP location without disconnecting LVDS cables ) 11:00 test smcs started ( 3 same links same cables ) second EGSE PC and scope still powered ON but not connected MIM1 + MIM2 + MIM3 boards inserted Vp = 28V Ip = 0.65A EZ-ICE stays connected 80000 packets links 1-2-3 300000 packets 500000 packets lost link 1 stop to connect the 6 links ( to add noise in the system and hincrease probability of disconnect ) + swap links *************************** configuration : PC1 link cable DSP link 0 A 2 ***(swapped at DSP front panel) 1 B 3 ***(swapped at DSP front panel) 2 C 1 PC2 link cable DSP link 0 D 4 1 E 5 2 F 6 test started 11:38 PC 1 link 0 lost ( still link 2 on DSP ) after 40000 packets start again 200000 packets 600000 packets - leave it running during lunch still running at 13:18 stop to change cabling and install flight software ************************************************** PC1 link cable DSP link 0 A 2 1 B 1 ***(swapped at DSP front panel) 2 C 3 ***(swapped at DSP front panel) PC2 link cable DSP link 0 D 4 1 E 5 2 F 6 load and run flight software : OK test DEC ON/OFF commands : OK ( relays operate ) stop to install DEC BASE boards ( red and blue ) and connect LVDS cables ************************************************************************ connect and power ON BOLC simulator start OBSW : OK observe HK : OK ( DEC + BOLC connected and packet counters ) load test sequence "dectest.txt" ( repeat DEC ON/OFF + verify HK ) 14:00 AM ******** run the test sequence Result = 287/300: - 1x1 link not connected after switch-on - 6x2 links not connected after the SET_PARAM_BOTH command When reconnecting, the connection stays stable for only a few seconds on both DECs stop to add capacitors on the mezzanine board (same capa as on the CPU board) ***************************************************************************** 15:06: switch on both DECs, stable connection for at least 2 minutes 15:08: start "dectest.txt" result = 291/300 When reconnecting, the connection stays stable for only a few seconds on both DECs JMG fixes the DECs front panels to the cage with screws. try to capture disconnection event with scope no success ( no special spike on data lines ) try to connect scope on "transmit" LED on DEC base captured disconnection using LeCroy "trigger on dropout" ( hard copies ) seems that link from DEC to DSP stops first ( to be evaluated by 1355 specialist ) 16:50 Crisa leave JMG+AM ****** exchange DSP boards ( mount EM DSP in EM card cage ) start OBSW and connect DECs wait for disconnection 17:10 no disconnection in continuous operation start test sequence 17:20 no failure 17:30 no failure test sequence ended - leave system ON and active for the night ( record HK ) 14/4/05 JMG EC ************** 12:00 DEC links still functional packet counters from HK records in DPU simulator : DEC1 DEC2 BOLC 17228818 17221307 2725813 17229330 17221819 2725893 17229842 17222331 2725973 17230354 17222843 2726053 17230866 17223355 2726133 saved HK file in //atlas/pacs/tests/crisa visit/april14/hk.xls 13:00 stop to install remaining DEC boards after modifications grounding, ADC range - power off decs by DPU telecommand - power off 28V 13:24 power on and start OBSW power ON 2 x DEC by DPU telecommand connected 14:20 still ON Vp = 28V Ip = 1.86A 15/04/05 JMG ************ system still functional ( about 22 hours ) average box temperature ( Pt100 thermometer touching outer walls ) 37°C packet counters : 19946873 19942145 3116734 19947385 19942657 3116814 19947897 19943169 3116894 19948409 19943681 3116974 19948921 19944193 3117054 19949433 19944705 3117134 saved HK file in //atlas/pacs/tests/crisa visit/april15/hk.xls stop for shielding improvements measured some internal components T° right after stop DSP board chips ( SMCS, RAM, ... ) and DC/DC components case T° about 47°C 18-04-05 ******** Placed a metal plate between DC/DC and DSP board. 17:00 EM DSP board in the rack, switched on both DEC, OK for 10 minutes. QM DSP board in the rack, DEC communication lost after 2-3 minutes Improved the contact between the DSP front panel and the rack (we had one single wire and we now use the metal blocks we had on the EM2 rack) 18:23 both DEC and BOLC connected 18:33: both DEC links lost. 19/04/05 ******** power on with scope print00 : QM board, DEC OFF, trace 1 = DPU link, trace 2 = Vcc ( gounded on DSP board ) started DECs at 11:45 print01 02 03 04 05 connection lost at 12:00 remove QM board and insert EM board 16:00 run dectest.txt, DPU link lost !!!!! It was never lost with the EM before !!! 16:10, started again. 2 DEC communication loss on 110 attempts. at 17:20, switched on both DEC and let it run... 20/04/05 ******** 09h00 both DECs still connected and running !! 10h30 stopped by EC . QM board in the rack. Monitoring of LVDS and TTL signals when a disconnection occurs. Nothing is wrong. We confirm that this is the DEC that stops first. AM: Modified the SMCS drivers such that the links are not stopped when a disconnection is detected (since the links of the mezzanine board are in 'autoreconnect', it works fine). 25 packets are lost when a disconneciton occurs. The disconnection is still signaled in the housekeeping. In 3 hours, we observed 120 disconnection between DSP and DECs but thanks to new drivers, the communication is still running. Placed MIM FPGA 8.08 in MIM1. EM board in the rack 21-04-05 ******** New mim FPGA (v8.09 ) in MIM1 The shifted sync src is not yet perfect. NM will issue a new version where the bit is reset once it is read. ====== note from JMG : I do not agree ====== The function has to be consistent with the user manual, and the specification Checked the OBT, it is incrementing very slowly. (OK, the OBT was not connected !) Tried a few scripts, part of the acceptance test procedure. The FW script failed because of the different phase A/B in proto and in QM. We will try to swap the phase at the FW level tomorrow. At 20:00, switched ON both DEC and BOLC and test the communication for the night. No disconnection at all during the night 22-04-05 ******** Swapped the coils cables on the FW such that they have the same behaviour than the QM FW. => FW test OK New MIM FPGA v8.10 Sync src is ok. ====== JMG : want to see screen copy !!! a_soft.txt is not perfect !! ====== JMG : what does that mean ??? 23-04-05 JMG ************ install mim1 on extender to check fpga signals 25-04-05 EC ************ Test SPU temperatures HK. board MIM2 modified by PD inserted in DMC : In HK 0 = 0 Ohms => 32737 = 400KOhms => 1Lsb = 400000/32768 = 12.2 Ohms R1: 2K HK = 110 =>1.32 KOhms => NOT OK R2: 10K HK = 485 =>5.91 KOhms => NOT OK R3:400K HK = 18450 =>225 KOhms => NOT OK ??? ADC -10 / +10 on 16 bits 1 lsb = 20 Volts / 65535 = 0.3 mV on R1 => 110 * 0.3 = 33.0 mV => OK R2 => 485 * 0.3 = 145.5 mV => OK R3 => 18450 * 0.3 = 5.53 V => OK convertion : drive current was 25µA , but it is 20µA => full positive range (10V/0.7) / 20e-6 = 714 KOhms new convertion : In HK 0 = 0 Ohms => 32737 = 714KOhms => 1Lsb = 714000/32768 = 19.96 Ohms R1: 2K HK = 110 =>2.195 KOhms => OK R2: 10K HK = 485 =>9.680 KOhms => OK R3:400K HK = 18450 =>368 KOhms => OK NOTE : Modification must be done in HKdef !!!! Other SPU HK verifications : +DMC_SPU_VCC_CUR PD inject 2Volt => after attenuation 2V*0.7 = 1.4V in HK 4690 * 0.3mV = 1.407 V => OK +DMC_SPU_VCC_VOL PD inject 4.9Volt => no attenuation => 4.9 V in HK 16264 * 0.3mV = 4.88 V => OK +DMC_SPU_VP_CUR PD inject 3Volt => after attenuation 3V*0.7 = 2.1V in HK 6790 * 0.3mV = 2.03 V => OK +DMC_SPU_PSU_P15V PD inject 3Volt => no attenuation => 3 V in HK -1 !!!!! No HK EC : Closing of the DMC and cleaning of the SETUP Begining of the acceptance test (BM) 26-04-05 (EC) ******** 9h00 acceptance test , script writing and testing 17h30 DECblue on & transmitting DECred on & transmitting BOLC on & transmitting 27-04-05 (EC) ******** 9h30 DECblue still on & transmitting DECred still on & transmitting BOLC still on & transmitting 10h30 test stopped by EC script Writing and testin .... 17h30 DECblue on & transmitting DECred on & transmitting BOLC on & transmitting 28-04-05 (EC) ******** 9h30 DECblue still on & transmitting DECred still on & transmitting BOLC still on & transmitting no diconnection during the night ! HK saved on D:\PACS\HK_280405.xls ******************************************************** EM 2 ******************************************************** 23-05-05 (EC ) ******** Configuration : Chassis EM2 one DEC BASE MIM1 PSU ( with regulated power supply ) external power supply for the DECs ( 2 boxes ) Start one test on DEC - DSP QM connection the try to power on the DECs failed PD observed that no signal came out the FPGA problem ( clock missplaced / pcb layout not compatible with new FPGA 8.1 layout ) solution => old fpga 8.0 24-05-05 (EC ) ******** 10h15 Connection started between DSP board and DEC 17h15 7 hours of connection : 10 disconnections recovered by the software : +/- 1/hour 25-05-05 (EC ) ******** 9h00 : DEC seems to be Crazy ( oscillating between ready state and reset state => like CRE CLK problem) No more communication between EGSE PC and DSP board Consumption on 5 volts is too low ! verification : 8 KHZ erratic ( clock desapears ) CLK DSP board is there : no dsp board reset system stoppped by EC modification to the CLK routing on MIM1 was done by PD. original FPGA 8.10 back on the board. 10h00 : New connection test started .... 16h30 : 3 disconnections recovered by the software 26-05-05 (EC ) ******** 10h00 :22 disconnections recovered by the software 26-05-05 (JMG-NM ) ******** 14:00 review MIM FPGA open questions remove Crisa QM for adding AGND/DGND connection on adapter PCB use Crisa board with EZ-ICE to manually access MIM1 FPGA registers test timing subsystem ( the new FPGA now starts in free run at power up ) test different Phase Inc values and verify PLL output and CRE clock frequencies ( note : strictly zero jitter only if PLL divides by a power of 2, otherwise jitter +/- 1 DSP clock ) nominal in spectro : 095217CB ( divide by 27.4, CRE freq = 8192 Hz ) 097b425e 27 8333 09249249 28 8036 08000000 32 7031 10000000 16 14062 08888888 30 7500 recommended : 09249249 for science acquisition in free run ( cre clock will be 8036 Hz ) FPGA bug : register 87 bits 19-20 not functional, bits 17-18 act on CRE and sampling test DEC master reset at all possible frequencies : OK ( and both DECs synchronous ) NOTE : only red DEC power ON is connected on EM2 test select sync source DEC BASE bug : only one functional sync output ( socket pin solder - to be repaired ) MIM FPGA bug : delayed sync : delay not proportional to entered command value MIM FPGA bug : missing /convert pulses ( impact on housekeeping A/D converter control ) 31/5/05 ******* observe erratic FPGA behaviour ( CRE clock interruptions ) found cause : when DSP board running low level software mechanism unknown - can reproduce without power off ( ICE : chip reset then F4 ) - when DSP stops, FPGA output is normal without any recovery action - if FPGA was programmed before ( e.g. sync source sel not zero ), registers are not altered after anomaly ( => anomaly is not caused by spurious FPGA reset ) - try to execute test program with endless loop writing each bit of each FPGA register this does not reproduce the symptom ( only similar case is when writing zero to phase inc but then FPGA cannot recover by itself ) - when DSP is running CSL on board ( high level ) software : no problem observe glitch on /mimsel ( ibbds1csn ) when DSP running low level software test the different timing control bits by manually writing into the control registers - reset PLL by OBT ( test with various OBT frequencies and various phase incr values ) OK - reset counters by sync ( note control bit is one shot, not permanent ) OK 1/6/05 ****** investigate spurious behaviour - disconnect the 2 "write" inputs, /mimsel and /read from the FPGA => problem still there - change FPGA 8.10 to 8.01 => problem still there check Vcc = 4.97V ( OK ) check 2.5V FPGA ( OK ) check with older FPGA version ( V3.1 26/5/2004 - asynchronous design ) ( adapt 18 MHz input ) => problem is not there mount FPGA V8.10 again, reconnect strobes, except IBB_DBWRAN : problem still there stop DSP : problem disappears try to reproduce first tests with DSP QM : exactly same behaviour ( see logbook 16/11/04 ) - enable output drivers of MIM1 board ( switch on board ) => spurious reset when accessing FPGA with EZ-ICE, DSP resets and emulator hangs - disable output drivers of MIM board ( switch on board, as it was since 23/5 ) => reliable operation - can write FPGA registers but not read them next : attempt to disable the reset generator on the DSP board ? 2-3/6/05 ******** NM AD hardware checks and improvements on MIM1 ( filtering of DSP clock ) spurious FPGA behaviour at power ON when DSP running LLSW : cured 6/6/05 ****** spurious DSP reset and emulator hang when reading from MIM1 : still there read from MIM1 disabled ( output buffers ) => OK ( same as 1/6 above ) write ISR using ICE and let it run 7/6/05 ****** modify components in DSP reset section ( see hardware log book ) enable MIM1 output buffers : no spurious resets continue with checks of FPGA timing and sync functions problems with HK ADC control pulses, shifted sync signal and frequency selection control bits 8/6/05 ****** new version FPGA V8.11MOD1 ( not compatible with MIM2 & MIM3 ! MIMDOUT bus is used for debug signals ) reset by OBT : OK phase incr : OK write ISR routine to display sync status as read by software check HK /convert and investigate shifted sync DSP and ISR routine have been working for 4 hours at 1000000 FPGA accesses / second without spurious resets found control bits 3 & 4 shorted ( HK_PWM has same problem as shifter ) reinstall FPGA V8.10 and investigate data bus in/out of FPGA DSP bus on backplane : OK short is on MIM1 board - repaired shifted sync now OK CRE clock and sample clock frequency control OK again /convert missing pulses 9/6/05 ****** dismount card cage to add backplane wiring - LL redundant interface - red & blue DEC ON/OFF create next FPGA revision 17/06/05 ******** ( using FPGA version 8.11 ) Since the 9 MHz clock line has been adapted, we have performed a connection test between DEC - CPU board. After 4.5 hours, there has been no disconnection (we had 2-3 each hour before). Test interrupted because MIM1 needed for QM FPGA test tool design JMG : dismount QM DSP install Crisa EM ( !!! no card guides, use with care !!! ) 19/6/05 ******* JMG power on Crisa EM + MIM1 test board without FPGA disconnect DEC power supply connect EZ-ICE message "initialisation failed" remove MIM1 same message dismount EM board install QM board ( just as a cross check for the emulator ) connect EZ-ICE operation is OK => Alain & Pierre to test the Crisa EM stand alone [ see Crisa e-mail about EZ-ICE : need additional Vcc pin for activation ] 21/6/05 ******* EC + AM + PDX : load OBS and test MIM2 MIM3 error in MIM3 ( also in as built drawings ) 22/6/05 ******* BM : continue integration with prototype mechanisms EC + PDX + FM : verify MIM1 on extender + DSP QM modified = correct operation without resets note : test tool for MIM FPGA QM/FM will use a flat pack socket on MIM1 COPPER SIDE => MIM1 needs to be on extender 24/6/05 ******* AM: install Crisa EM in the stand-alone box: impossible to connect with the EZ-ICE But OBS can boot. Crisa EM in the rack, the OBS can not boot (the SUSW is OK but not the ASW). There are no reset. note : see Crisa mail about EZ-ICE interface ( additional Vcc needed to activate ) Connecting the grating, hall sensors have an incorrect value in the hk. HALL_A -5V 0 5V 3V HALL_B 0V 5V 0 3V HALL_A in HK 16600 32000 -17000 -22500 HALL_B in HK 31800 -17000 -32000 -22500 ADC was configured in 'straight binary' instead of '2's complement' Corrected ( MIM2 hardware ). 4/7 .. 15/7 *********** BM acceptance test grating hardware FM model ( system moved to COE ) 13/07/05 ******** New parameters for Grating Lock and Unlock. To be documented in SUM and release notes Lock (47): bit1: LL1 bit2: use limit switch bit4: LL2 Unlock (48): bit2: use limit switch bit3: LL1 bit5: LL2 According to NM, maybe we need to set both LOCK and UNLOCK to 1 at the same time. Note from JMG : strictly forbidden !!! ( there is a logical lock inside the FPGA : if the DSP sets both bits then the FPGA outputs "close" ) ( this may change for the FM we might want to give priority to the opening function ) control bits in register : addr 0x45 ( MIMCTRL ) bit 10 = lock M1 = close M1 FPGA pin 110 MIM1 J2 pin A23 MIM3 B23 bit 11 = unlock M1 = open M1 FPGA pin 111 MIM1 J2 pin A22 MIM3 B22 bit 20 = unlock M2 = open M2 FPGA pin 126 MIM1 J2 pin B21 MIM3 B21 bit 21 = lock M2 = close M2 FPGA pin 127 MIM1 J2 pin B20 MIM3 B20 18/7/05 ******* JMG - NM reinstall system in development office verify LL step by step error in backplane connections : remove MIM1-MIM3 additional wires, use existing B21 B20 links error on MIM3 inputs used are B21 B22 B23 B24 should be B20 B21 B22 B23 + wrong order inconsistency between MIM1 CSL and MIM1 OIP + wrong FPGA pins => NM to change FPGA pin allocation for M2 open and M2 close => NM to verify compatibility new pin allcation with EM1 MIM1 ( LL2 does'n exist but pins must be NC ) => PDx to change MIM1 EM2 : FPGA pins 16 & 17 are M2 open and M2 close => PDx to change MIM3 EM2 : B21 B22 B23 B24 to be B23 B22 B20 B21 for consistency with backplane and FPGA command bits => PDx to update backplane schematic to show existing lines formerly named spare1 and spare2 now LL2 commands as listed above => PDx to review ( and possibly correct ) layout of MIM1 test vehicle for QM FPGA note : push buttons will not interfere with FPGA as long as inactive outputs are 0 ( FPGA and buttons as wired ) 19/07/05 ******** JMG power up system ( DSP + MIM1 + FPGA 8.12 ) connect EZ-ICE - emulator reset - OK CRE clock present - 8192 Hz - no OBT check availability of 4 LL control bits at MIM3 J2 level : OK bit 10 = lock M1 = close M1 MIM3 B23 bit 11 = unlock M1 = open M1 MIM3 B22 bit 20 = unlock M2 = open M2 MIM3 B21 bit 21 = lock M2 = close M2 MIM3 B20 check period measurement = 0 at power on check phase acc ( residual ) = 0 when unlocked AM check OBS function ********************* 2-38-0 to switch-on grating Unlock both LL: 2-48-1-28 Lock both LL: 2-47-1-12 only "open M1 working", 3 other bits inactive 20/07/05 ******** JMG+NM test timing functions spectro mode, lock on OBT : bit 3 of register 0x87 ( write 0x8 ) OBT phase inc residual high residual low samples difference 131.072 095217CB 04 F39CA3D8 FCEEBBA3 F47F8CB test adjustment procedure : FCEEBBA3 = decimal -51463261 theoretical correction to phase incr = 51463261 / 137 = 375644 = 0x5BB5C phase acc < 5 => pll frequency too low => phase incr must increase new phase incr = 0x095217CB + 0x5BB5C = 0x957d327 OBT phase inc residual high residual low samples difference 131.072 957D327 04 FFFFFFDF F6A82CB8 test with a too high pll frequency OBT phase inc residual samples difference 131.072 95D8E83 04 F9B3B598 05 0311441B BUG : LSW & MSW not read at the same time => sometimes EZ-ICE displays 05 F9.. or 04 03.. test : change OBT frequency OBT phase inc residual samples difference 131.072 957D327 04 FFFFFFDF 04 F6A82CB8 130.072 957D327 05 0957D306 04 FFFFFFDF 129.072 957D327 05 12AFA62D 05 0957D306 957D327 Now connect BOLC SIM ( sync only ) power off - power on - test state is unsynchronised and BOLC sync is present phase acc is read as 0 : OK power up state :BOLC SIM outputs 40 Hz FPGA sync delay = 0 set sync selector to BOLC ( reg. 0x84 = 5 ) check test pin "sync4" should be shifted sync - OK to switch to photo mode : - set phase incr to nominal value 0x0977602A verify CRE clock / IRQ is at 8320 Hz : OK - set sync selector to BOLC ( reg. 0x84 = 5 ) needed ? not from FPGA 8.X display CRE clock - SYNC1 test pin - shifted sync test pin ( drifting ) ( see screen copy ) - set "lock on BOLC" bit 5 of register 0x87 ( write 0x20 ) => waveforms are locked but phase is not 0 ( OK ) - perform PLL_PD_SYNC ( temporary set bit 4 of register 0x87 keeping bit 5 ON - value 0x30 ) => waveforms are locked in phase and stay stable - observe phase acc and perform phase inc adjustment BOLC phase inc residual samples difference 40.00 Hz 0977602A 00 25DBE4C8 00 2F5344F2 theoretical value of HI(phase acc) = 16640 = 0x4100 but only bits 32-39 are implemented max theoretical value ( 1.25 Hz case ) would be 532480 = 0x82000 ( 20 bits ) AM check OBS function ********************* 2-38-0 to switch-on grating Unlock both LL: 2-48-1-28 Lock both LL: 2-47-1-12 Tests OK using control signals at MIM3 J2 site on backplane Software burned in EEPROM set photometer mode again BOLC phase inc residual samples difference 40.00 Hz 0977602A 00 25DBE4C8 00 1C64849E ( 1 cycle difference from 2 hours before ) test adjustment procedure error = 0x25DBE4C8 = 635167944 decimal, divided by 16640 => 0x951B count too much => pll frequency must decrease => phase inc must increase new phase incr = 0x0977602A - 0x951B = 0x976CB0F drift ( there is no more a whole number of CRE clocks in a BOLC period ) error : we should have added, not subtracted ? new phase incr = 0x0977602A + 0x951B = 0x977F545 drift the other way phase acc HI = 4 ! error : wrong divisor ( should be number of DSP clocks in a BOLC period ) error = 0x25DBE4C8 = 635167944 decimal, divided by 450000 => 1411 => 0x583 new phase incr = 0x0977602A - 0x583 = 0x9775AA7 OK BOLC phase inc residual samples difference 40.00 Hz 9775AA7 00 00033d4c 00 097A97F3 next test : change Bolc frequency ( restart high level software from EEPROM ) commands : 2-33-1-0B020020 (20Hz) Le diviseur-1 se trouve dans les bits 5 à 9 2-33-1-0B020040 (13Hz) 2-33-1-0B020060 (10Hz) with BOLC at 13.3 Hz, use high level software to set the phase incr to 0x9775AA7 ( previously adjusted value ) note : as we are using integer divisors, the adjusted phase incr is valid for any available BOLC frequency restore photometer mode ( sync on BOLC ) read LO(phase acc) from HK file value is 636900 decimal this value is the residual after one period at 13 Hz, or 3 periods at 40 Hz the residual at 40 Hz was 0x00033d4c or decimal 212300 = 636900 / 3 this is exactly as expected because the PLL has drifted 3 * longer at 13 Hz 28/07/05 ******** JMG/NM validation of FPGA version 8.16 MIM1 EM2 on extender DSP EM under EZ-ICE FPGA V8.15 and V8.16 are not working ( clock input loads down signal source ) note from NM : made with other programming device 29/7/05 ******* NM/PDe cause of malfunction found - MIM1 board interference with FPGA + buffer component damaged MIM1 repaired and FPGA new copy installed ( still V8.16 ) first part of validation report filled in ( FPGA power on state ) see "non-computer" log book add launch lock model ( 2 DC motors ) to FPA mechanical model and connect redundant interface label LL manual control buttons 01/08/05 ******** NM/JMG second part of validation procedure ( FPGA + MIM1 + DSP + EZ-ICE ) using level 1 interface check table see results in "non-computer" log book install MIM2 and MIM3 in card cage connect FPA mechanical model start from EEPROM OK OBSW started incoming HK BM Grating test with STM. HK diag du SP et de la Position : 6-50-13-3-fa-fb-ffff. Both values are identical !? -> SP = Position. 2-40-0 for starting regulator, nothing happens. In nominal HK, SP is OK. Using the script grat_func, the KH diag is now right but strange movement of the grating when homing and security disable the controller. Restart of the controller "manually" : 2-40-0 and the grating highly oscillate (at high frequency). Homing : 2-44-1-0 and nothing happens. Stop and restart of DMC and now everything is OK (no oscillations and homing and movements OK). I don't know what happened, maybe a wrong command sequence... Test of LL : 2-47-1-12 and 2-48-1-28 -> With only nominal motor connected : OK -> With both motor connected : LL turns a little in wrong direction and stops ! 2-47-1-2 and 2-47-1-8 : redundant motor is turning in the opposite direction compared to nominal. This is a mistake in the cabling : no inversion on redundant. -> correction Retest : No change ? Commands are not giving expected results. 02/08/05 ******** JMG/PDe/AM test LL commands from EZ-ICE : close M1 OK - open M1 OK close M2 OK - close M2 short circuit cause : signal is grounded on MIM2 ( inconsistency with backplane ) correction : cut pin MIM2 J2 B21 Tested all locks combination on the LL prototype: OK Unlock Nominal: 2-48-1-8 Unlock Redundant: 2-48-1-20 Unlock Both: 2-48-1-28 Lock Nominal: 2-47-1-2 Lock Redundant: 2-47-1-10 Lock Both: 2-47-1-12 03/08/05 04/08/05 BM ******************** use of EM2 electronics with FM grating ( ambiant T° testing before cooling ) configuration : QM DSP - MIM1 EM2 + FPGA V8.16 - MIM2 EM2 - MIM3 EM2 - redundant LL switches linear power supplies ( note : 28V ON/OFF manual switch added ) some "jumps" of power-of-2 n° of units observed in grating position HK ( 128, 256, ... ) not critical for functional testing => grating testing continued 05/08/05 NM PDe *************** PDe checked and re-made some MIM1 solder joints connect EM2 to EM mechanisms while cryostat is cooling simplified procedure for grating tests : use "start from EEPROM with script support.bat" execute script name "induct.txt" no jumps observed 06/08/05 JMG+NM *************** 9:30 switch on system for inductosyn testing on EM mechanisms verify AD2S80 reading waveforms use script to check for jumps manipulate grating by hands ( you have to be convinced and you will find some positions and rotation speeds where you will see jumps in the HK ) found error in GRATINGRD waveform ( was OK in previous FPGA version - was changed when converting from schematic to VHDL ) FPGA 8.17 to be produced the following modifications will be tested : - synchronous bus interface for DSP writes - correct polarity GRATINGRD 07/08/05 JMG+NM *************** install FPGA V8.17 restart OBSW and perform same tests as yesterday use "start from EEPROM with script support.bat" execute script name "induct.txt" start DiagHkRTViewer( icon on desktop ) display LSB, /INH, DIR, RIPPLE CLK note diagnostic HK worked for 1 hour without any error or transmission loss between DMC and DPU SIM try to produce marginal cases Ripple clock at the same time as inhibit ( unavoidable, see data sheet ) ( need > 10 end to end moves at high speed to catch one sample ) => case exists but is a rare event and potential error of 65536 handled by software possibly recommend users to avoid 0xNN0000 as static position for the grating stop OBSW and connect EZ-ICE execute again low level interface checks ( writing into registers ) OK ( see paper log book ) but some functions not tested because MIM boards are inserted execute lock on BOLC procedure see this log date 0/7/05 - lock - measure phase error - adjust phase incr set photometer mode BOLC phase inc residual samples 40.00 Hz 0977602A 4100 2f5344f2 4100 25dbe4c8 test adjustment procedure divisor should be number of DSP clocks in a BOLC period error = 0x25DBE4C8 = 635167944 decimal, divided by 450000 => 1411 => 0x6e4 new phase incr = 0x0977602A - 0x6e4 = 0x9775946 OK phase acc oscillates between 40ff.fnnn and 4100.0nnn jitter on PLL output is 1 DSP clock = 55 ns 8/8/05 BM ********* connect to FM grating FPGA V8.17 fails to operate the grating revert to FPGA V8.16 test LL operation adjust Inductosyn voltages ( pots on MIM1 and MIM3 ) 9/8/05 JMG NM AM **************** check DAC data sheets : timing variations exist between AD and Maxwell discuss FPGA improvement & increasing wait states from 4 to 6 for EM/QM/FM compatibility dismount MIM1 and MIM3 for measuring the adjustable resistors 10/8/05 ******* change DSP board initialisation for 6 wait states MIM accesses ( not tested ) 11/8/05 BM AM ************* update code for chopper transitions JMG : 6 WS modif is not working NM : prepare and simulate FPGA V8.18 JMG : 6 WS operation OK when writing to the DMPSC register by hand via EZ-ICE dismount MIM1 to install FPGA V8.18 note : mechanisms will NOT work with 4 WS any more test FPGA V8.18 OK with 6-7-8-9 WS spurious double write with WS >= 10 to be tested further with EM mechanisms ( go back to V8.16 if any problems or work with FM ) 23/8/05 JMG *********** install copy of FPGA V8.18 made with new version of software ( to validate QM version ) 24/8/05 JMG *********** using MIM tets plan document to validate wrt FPGA 8.18 power up state checking ( see paper log book ) OK level 1 tests using EZ-ICE OK 25/8/05 JMG *********** level 2 tests using test program and EZ-ICE document usage of shifted sync for mechanism trigger tets program displays mechanism trigger condition on spare control bit maximum phase shift = N-1, N is number of samples in sync period examples in nominal locked conditions - power up case ( internal 256 Hz ) : N = 32 - nominal DEC mode ( 256 Hz, 32 clocks/readout ) : Nd = 32 - other DEC mode ( X clocks/readout ) : Nd = X - nominal BOLC mode ( 20 Hz ) : Nc = 416 - BOLC simulator at 40 Hz : Nc = 208 - other BOLC mode ( parameter X = 0..31 ) : Nc = 208 * ( X+1 ) due to internal FPGA delay of 1 sample in spectro mode, programming N-1 triggers the mechanisms just after the sync positive edge programming N-2 triggers the mechanisms just before the sync positive edge ( actually just before the next one ) programming a value >= N makes the shifted sync and the mechanism trigger disappear !!! ( valid for nominal phase incr and sample frequency selection, for other cases recompute using FPGA manual ) test with BOLC ( see scope screen copies ) - phase incr = photo value 0x89<=0x977602A - sync source = BOLC 0x84<=0x5 - lock PLL to BOLC 0x87<=0x20 display mechanism trigger for various values of phase shift ( 0x85 ) the 1 sample delay is also there except for the value 0 of phase shift programming 0 triggers the mechanisms just after the sync positive edge programming 1 triggers the mechanisms 2 sample periods later programming 2 triggers the mechanisms 3 sample periods later etc... programming N-2 triggers the mechanisms just before the sync positive edge ( actually just before the next one ) 0xCF is the maximum value ( 0xD0 = N = 208 kills the shifted sync ) OK 25/08/05 AM *********** Modified the OBS to have 6WS when accessing the timing FPGA (version burned in EEPROM) functionnal tests to validate the timing FPGA: - Grating homing: OK - Chopper: the position readout is not in the same range as before (central position and range have changed) - FW: When moving the FW, the grating moves (and not the FW) - Cal src: unable to test (mezzanine board is missing) - Temp sensors: no tested yet (missing simulators) 25/08/05 JMG ************ SELFW MIM1 U20/14 OK according to drawings should go to MIM1 J2/B4 MIM3 J2/B3 SELPHOT MIM1 U20/13 OK according to drawings should go to MIM1 J2/B3 MIM3 J2/B2 backplane tested : OK for connection and no short to GND drawings MIMEM3 : fault ( SELFW = J2/B4 & SELPHOT = J2/B3 ) inspected MIMEM3 board : correction is present signals at output of FPGA : OK U20 on MIM1 defect ( out 14 does not follow in 6 ) => signal SELFW never given chopper position : excitation should be 320 mV but measured 240 mV 26/08/05 JMG ************ MIM1/U20 new AC541 soldered - signals OK at MIM3 J2 error on MIM2 : +5VREF to U11/U30 connected to Vref chopper position sensor instead ( incomplete correction on PCB ) correction done system reassembled 26/08/05 AM *********** functionnal test with the OBS: - Chopper: the position readout has changed but it is still not the nominal range. - Grating: homing OK, small steps OK. No visible jump in the readout - FW: When moving the FW, the grating moves (and not the FW) although the relays are actuated corrected SPR-0344: When changing the hk diag list, the next diag hk packets do not contain old values anymore at its beginning. Latest version burned in EEPROM. 29/08/05 JMG ************ relay K5 seems not to work drive transistor OK ( voltage on D8 goes to 0 ) BM : using the system for FM grating test after vibration 30/08/05 JMG BM *************** BM : using the system for FM grating test after vibration problem with limit switch in HK ( when testing redundant circuit ) 31/8/05 JMG *********** corrected errors on MIM3 ( wrong resistors values ) connec to EM mechanisms start OBSW from EEprom manual commande - power on grating - enable grating controller - home grating - disable grating controller - power on spec FW - move to position and move 0x300 steps : wheel moves ( relays OK ) problem with initial coil current calculated from Hall sensors ( starts with oscillations and stops at wrong place ) - power on photo FW - move to position and move 0x300 steps : wheel moves ( relays OK ) problem with initial coil current calculated from Hall sensors ( starts with oscillations and stops at wrong place ) - switch off grating controller check HK of FW Hall position sensors found bridges L12-13-15-15 missing on MIM3 - power on spec FW - move steps : OK - move to position 0 : OK - move to position 180 : not found - voltage on U24/14 = -15V ( FW on connector J13 ) - power on spec FW - move steps : OK - move to position 0 : OK - move to position 180 : not found but voltages on U8/8 and 14 comparable ( 0.8V when in position ) ( FW on connector J12 ) 01/09/05 JMG ************ checked input protection diodes on MIM3 - diodes are connected to +/-15V instead of +5V/GND - schematics ( and OIP version ) are OK - removed D27 D57 - power on spec FW - move steps : OK - move to position 0 : OK - move to position 180 : not found but voltages on U8/8 and 14 comparable ( 0.8V when in position ) ( FW on connector J13 ) setup HK diag FW1 pos0 = 555 = 0x22b FW1 pos 180 = 559 = 0x22f 6-50-13-3-22b-22f-ffff 2-76-1-5 2-64-4-1-600 2-77-0 "view hk diag 2x16bits.bat" max values recorded : pos0 : 2065 ; pos180 : 1797 ( threshold is 2000 => no detection ) threshold should be half maximum => 1000 question : are the levels different with the QM and FM filter wheels ? test connector : chopper position amplifier OK check chopper drive operation ( write to DAC register using EZ-ICE ) DAC output OK ( voltage proportional to data ) chopper not moving test amplifier on bench - remove DAC - inject 10 KHz sine 10V pk CHOPON signal => +5V latching relays in "bypass" position connect 100 ohm load between 14&15 ( R1-R2 ) result is about 14V pk sinus across load = about 140mA full scale consistent with design reinstall MIM2 in EM2 with test connector ( same conditions ) using EZ-ICE DAC 0x1000 1x2000 0x4000 0x7fff load 1.75V 3.5V 7V 14V 05/09/05 BM *********** cold test grating 06/09/05 BM EC AM ***************** finish cold test grating then move to upstairs office quick check : hom egrating OK 07/09/05 BM JMG AM EC ********************* no operation ( pacs workshop ) P Delvaux prepares test board for MIM FPGA QM validation 08/09/05 NM , PD **************** 09/09/05 EC , NM , PD ********************** Software started with the QM fpga => OK DEC BASE inserted in the rack ( position : XXX ) in order to synchronise the mechanisms. NM + PD , check all signals on fpga after power up. NM +PD + EC func_grat script started => ok func_chop script started => ok ( but it seems that parameters or analog circuitry are not adapted ) dismount chopper for MPIA open day a_mim1fpga =>OK ( obt counting ok 1 DEC ( which one ??? ) synchro and Bolc synchro) a_FW => not OK , impossible to move the wheel 12/9/05 13/9/05 14/09/05 NM JMG *************** ( using EZ-ICE ) enter test program ( procedure § 5.4.3. ) check software IRQ signals OK disconnect cables to mechanisms but keep MIM boards IN check ramp on MIM3 DAC outputs OK perform 5.4.4. (level 2 tests) minor problem with shifted sync from sync source 1 note : system powered ON permanently to increase FPGA operating hours 15/9/05 16/9/05 19/9/05 JMG NM ************** perform level 3 tests using EZ-ICE OK ( see report and screen copies ) FPGA kept in BOLC lock mode for several hours jitter between BOLC sync and Sample = 1 DSP clock, phase stable 15:00 dismount and condition FPGA for shipping restart with MIM1 EM board missed sync pulses in spectro mode are also present problem is repeatable and understood the pulse following the destructive sync will be missed if the shift value is larger than some constant depending on the destructive sync pulse length criticality : low ( this function is not intended to be used in flight, shifted sync is specified for photo mode ) XXX AM EC ========= perform looping test of changing DEC timing parameters without master reset, checking the red and blue synchronisation pulses do not drift apart ( several days at 1 command every few seconds ) 17/10/05 JMG ============ dismount QM DSP board for mechanical fit check at OIP 21/10/05 ******** Reinstall QM DSP Mounting of MIM1 mezzanine. note: A jumper must connect J24:2-3 ( check phase between CSADCCLK and IRQEXEC for correct jumper choice ) Test calibration sources card ( new PCB, never tested before ) ADC is functional, CALSRC HK is received solder gain setting resistors on the instrumentation amplifiers note : FW connectors on MIM3 EM and QM not at the same place ( but QM is consistent with ICD ) In the OBS, in chopgrat.s, removed the transmission of CS->SignalType with the SEMA. It was not working anymore for CS2 !?! Tested both CS in open loop and in measure only in spectro mode: OK same test in photo mode (with BOLC sync connected): OK Debug of Temperature sensors: 2004004e : command PWM 20040087 : ctrl reg 0x01000000 : inversion du signe (1 = pos) 0x00400000 : enable 0x00800000 : diviseur (1 = pas de division) TODO : check chopper cabling retry chopper control in closed loop repeat tests with OBS; check all possible HK; perform integration period measurement PROCEDURE POUR TESTER LES COMMUNICATIONS: **************************************** Prérequis: Il faut utiliser 2 programmes pour tester les communications: - un programme à lancer sur le PC EGSE: testSpaceWire.exe - un programme à lancer sur la carte CRISA : testSMCS.exe Procédure à suivre: - allumer le PC EGSE et le PC EZ-ICE - mettre sous tension la carte crisa - connecter la sonde EZ-ICE - lancer wice020.exe (raccourcis sur le bureau du PC EZ-ICE) - ouvrir la fenêtre Memory-Program - ouvrir la fenêtre Core-Other-Memory Control - faire un emulator reset - dans la fenêtre Memory Control, mettre PMBANK1 à 00000000 - aller dans File-Load et choisir C:\pacs\test3\testsmcs.exe - aller dans la fenêtre Program-Memory, faire Ctrl-G (Goto Location) et aller à l'address 0x24c - y mettre un breakpoint (F9) - lancer l'exécution (F4) - lorsque le breakpoint est atteint, cliquer OK - sur le PC EGSE, lancer testSpaceWire.exe (raccourcis en bas à gauche sur le bureau), choisir l'option 8 - sur le PC EZ-ICE, effacer le breakpoint (F9) et relancer l'exécution (F4). - sur le PC EGSE, chaque fois que 1000 paquest ont été transmis sur un lien, le numéro du lien s'affiche. - on constate que la communication est perdue, soit: -- quand un numéro de lien ne s'affiche plus -- quand "Error on link X" s'affiche - lorsque la communication est perdue, sur le PC EGSE, quitter testSpaceWire.exe - sur le PC EZ-ICE, faire un emulateur reset et recommencer à "mettre PMBANK1 à 00000000) TODO : **** - New parameters for Grating Lock and Unlock. To be documented in SUM and release notes - procedure de patching pour SCOS2000 (voir soft Nabil) - pour la PLL, il reste à introduire la formule DELTATEC (à faire plus tard) - tester le code de StopFWMove et StopGratMove - looks like CRDCCP is a packet counter (in PHOTO header when in simulation mode only (actually, counting the synchro received (done in chopgrat.s (READOUT_COUNT))). - mettre le signe de l'inductosyn dans du hk pour pouvoir verifier que la commande est passee (a faire en fait pour toutes les commandes) - faire la 'correction table' pour les senseurs de temperature FPU - remettre setpoint et target a zero chaque fois qu'on enable ou switch-on - there is a small error on the temperature measurement to be investigated later - il faut valider les hk des dec uniquement lorsqu'on a la connexion (et pas à la réception de la commande switch on) - remettre le time-out dans le sequencer - remettre les paramètres du chopper - flag the grating following error also in SW global status - vérifier SET_TIME and Write_TIME - Script command 'system' - ajouter une option pour tenir compte ou non de la synchro lors d'un mouvement des mechanismes (à mettre dans DMC_SELECT_MECH_CTRL_MODE) SUM: - écrire ECR pour l'ICD DMC-DPU pour dire qu'on a rajouté 0hAA comme code d'erreur qui peut être renvoyé par les DUMP, CHECK, LOAD, WRITE - documenter procédure de passage spectro/photo (d'abord Write FPGA param (quels params utiliser (voir code) + set_sync_source) - procédure d'upload du soft - documenter DMC_MIM_ST + Timing ctrl/status reg dans le SUM - full scale FW_GRAT DAC = 555mA - SPU T° + PSU T° : 7V = 10mV - nouveaux paramètres Grating + chopper - inclure les checksums - JMG/EC 24/8/05 quand on modifie les bits de commande des CRE s'assurer que les timings sont nominaux ( 32 CRE clock/readout ) Finalisation du code (recherche dans *.c; *.h; *.s) Date TODO (logbook) TODO(code) $$ TBC TBD TOTAL DEBUG 04-05-04-midi 14 29 57 12 4 116 64 04-05-04-fin 17 26 56 12 4 115 62 05-05-04-midi 18 21 51 12 3 105 62 06-05-04-midi 16 20 37 12 2 87 62 07-05-04-fin 13 20 32 0 2 67 62 11-05-04-midi 14 20 48 1 2 85 62 13-05-04-midi 15 20 36 1 2 74 62 13-05-04-fin 21 13 18 1 2 55 62 14-05-04-fin 21 6 11 1 1 40 58 17-05-04-fin 15 5 5 1 1 27 57 18-05-04-fin 15 5 4 1 0 25 56 19-05-04-fin 13 5 4 1 0 23 56 21-05-04 10 5 3 1 0 19 56 28-05-04-mat 15 2 11 1 0 29 47 28-05-04-fin 10 2 11 1 0 24 47 02-06-04 11 2 13 1 0 27 47 09-06-04 12 2 9 1 0 24 47 29-07-04 24 3 7 1 0 35 46 06-08-04 18 2 7 1 0 28 46 16-08-04 19 2 10 1 0 32 49 18-11-04 19 1 8 0 0 28 50 Start-up procedure : ------------------ Dans l'émulateur : 1. Chip Reset 2. F4 to start Wait 10 sec Sur Pacs Egse : 3. Lancer ComExe 4. Lancer SimDPU for SUSW 5. Entrée de menu 6.31 (ignorer les messages d'erreur du SimDPU) 6. Entrée de menu 6.30, adresse = 8032 Ensuite : 15 secondes pour arriver au point 9 : quitter SimDpu et ComExe (le deuxième plante dès qu'on quitte le premier) 7. Lancer ComExe 8. Lancer SimDPU for ASW 9. Attendre que le menu apparaisse (après les 15sec). Si il y a des problèmes de connexions, quitter ComExe et SimDPU et les relancer. Diagnostic HK commands: CAL SRC1: -v445 -v446 -v522 -v523 -v524 -v525 -v526 -v527 -v528 -v529 1bd 1be 20a 20b 20c 20d 20e 20f 210 211 CAL SRC2: -v447 -v448 -v538 -v539 -v540 -v541 -v542 -v543 -v544 -v545 1bf 1c0 21a 21b 21c 21d 21e 21f 220 221 DEC R oscillation: -v637 -v638 -v639 -v640 -v641 -v642 -v643 -v651 289 27d 27e 27f 280 281 282 283