2025-01-20 15:29:15,025 - wsc_run_gdm_sar.py - INFO - workflow invoque with /usr/contrib/cycle/gdm-sar-uga/ws/wsc_run_gdm_sar.py -c isdeform -t df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -v 4 -e source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; 2025-01-20 15:29:15,025 - wsc_run_gdm_sar.py - INFO - -------------> start workflow 2025-01-20 15:29:15,026 - wsc_run_gdm_sar.py - DEBUG - swath final is iw1_iw2_iw3 2025-01-20 15:29:15,026 - wsc_run_gdm_sar.py - INFO - -------------> start workflow 2025-01-20 15:29:15,027 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start DOWNLOAD', 'status': 'running', 'step': 'download', 'progress': 9} 2025-01-20 15:29:15,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 15:29:15,315 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 15:29:15,317 - wsc_run_gdm_sar.py - INFO - asf download then peps in case of 2025-01-20 15:29:15,318 - wsc_run_gdm_sar.py - INFO - asf+peps download cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadAsf.py -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -j 7 -v 4 || source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadPepsData.py -wd /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -c /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/configs/nsbas_config.json -j 3 -v 4 <<< 2025-01-20 15:29:15,318 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 15:29:15,319 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 15:29:15,319 - wsc_run_gdm_sar.py - INFO - get dem cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_getDemFile.py -k -s NASADEM_HGT.001 -v 4 -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/DEM --safe /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -p vv<< 2025-01-20 15:29:15,319 - wsc_run_gdm_sar.py - INFO - -------------- starting step 1: download ---------- 2025-01-20 15:29:15,319 - wf_scheduler - INFO - Sequential scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadAsf.py -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -j 7 -v 4 || source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadPepsData.py -wd /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -c /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/configs/nsbas_config.json -j 3 -v 4 , workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_getDemFile.py -k -s NASADEM_HGT.001 -v 4 -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/DEM --safe /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -p vv, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 15:29:15,319 - wf_scheduler - INFO - launching cmd 0/2: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadAsf.py -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -j 7 -v 4 || source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadPepsData.py -wd /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -c /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/configs/nsbas_config.json -j 3 -v 4 , workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 15:29:15,319 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=2,walltime=12:50:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadAsf.py -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -j 7 -v 4 || source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadPepsData.py -wd /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -c /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/configs/nsbas_config.json -j 3 -v 4 ", "name": "1-0_download_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-0_download.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-0_download.txt"}' 2025-01-20 15:29:15,320 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=2,walltime=12:50:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadAsf.py -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -j 7 -v 4 || source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadPepsData.py -wd /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -c /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/configs/nsbas_config.json -j 3 -v 4 ", "name": "1-0_download_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-0_download.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-0_download.txt"}' ***** 2025-01-20 15:29:15,648 - wf_task - INFO - run with pid 1651809 2025-01-20 15:30:15,648 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651809 2025-01-20 15:31:15,912 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651809 2025-01-20 15:32:16,001 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651809 2025-01-20 15:33:16,078 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651809 2025-01-20 15:34:16,215 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651809 2025-01-20 15:35:16,285 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651809 2025-01-20 15:35:16,351 - wf_task - INFO - ret_code for source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadAsf.py -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -j 7 -v 4 || source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadPepsData.py -wd /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -c /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/configs/nsbas_config.json -j 3 -v 4 is 0 2025-01-20 15:35:16,351 - wf_scheduler - INFO - done with cmd 0/2: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadAsf.py -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -j 7 -v 4 || source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wsc_downloadPepsData.py -wd /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -c /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/configs/nsbas_config.json -j 3 -v 4 , workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=1651809 2025-01-20 15:35:16,351 - wf_scheduler - INFO - launching cmd 1/2: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_getDemFile.py -k -s NASADEM_HGT.001 -v 4 -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/DEM --safe /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -p vv, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 15:35:16,351 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=2,walltime=02:09:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_getDemFile.py -k -s NASADEM_HGT.001 -v 4 -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/DEM --safe /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -p vv", "name": "1-1_dem_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-1_get_dem.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-1_get_dem.txt"}' 2025-01-20 15:35:16,352 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=2,walltime=02:09:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_getDemFile.py -k -s NASADEM_HGT.001 -v 4 -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/DEM --safe /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -p vv", "name": "1-1_dem_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-1_get_dem.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-1_get_dem.txt"}' ***** 2025-01-20 15:35:16,628 - wf_task - INFO - run with pid 1651810 2025-01-20 15:36:16,629 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651810 2025-01-20 15:36:16,712 - wf_task - INFO - ret_code for source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_getDemFile.py -k -s NASADEM_HGT.001 -v 4 -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/DEM --safe /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -p vv is 0 2025-01-20 15:36:16,713 - wf_scheduler - INFO - done with cmd 1/2: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_getDemFile.py -k -s NASADEM_HGT.001 -v 4 -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/DEM --safe /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/SAFE -p vv, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=1651810 2025-01-20 15:36:16,715 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-0_download.txt 2025-01-20 15:36:16,725 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-1_get_dem.txt 2025-01-20 15:36:16,729 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 15:36:16,729 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 15:36:16,731 - wsc_run_gdm_sar.py - INFO - -------------> start mkworkdir 2025-01-20 15:36:16,731 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 15:36:16,732 - wsc_run_gdm_sar.py - INFO - %%%%% running task workdir polarisation=vv on swath iw1 2025-01-20 15:36:16,732 - wsc_run_gdm_sar.py - INFO - get process task: nsbas cmd = >>nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 workdir polarisation=vv<< 2025-01-20 15:36:16,732 - wsc_run_gdm_sar.py - INFO - %%%%% running task workdir polarisation=vv on swath iw2 2025-01-20 15:36:16,732 - wsc_run_gdm_sar.py - INFO - get process task: nsbas cmd = >>nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 workdir polarisation=vv<< 2025-01-20 15:36:16,732 - wsc_run_gdm_sar.py - INFO - %%%%% running task workdir polarisation=vv on swath iw3 2025-01-20 15:36:16,732 - wsc_run_gdm_sar.py - INFO - get process task: nsbas cmd = >>nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 workdir polarisation=vv<< 2025-01-20 15:36:16,732 - wsc_run_gdm_sar.py - INFO - -------------- starting step 1: download ---------- 2025-01-20 15:36:16,732 - wf_scheduler - INFO - Async scheduler: running command=nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 workdir polarisation=vv, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 workdir polarisation=vv, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 workdir polarisation=vv, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 15:36:16,732 - wf_scheduler - INFO - launching cmd 1/3: nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 workdir polarisation=vv 2025-01-20 15:36:16,732 - wf_task - INFO - running nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 workdir polarisation=vv 2025-01-20 15:36:16,732 - wf_task - DEBUG - process-task: running command nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 workdir polarisation=vv (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 15:36:16,733 - wf_scheduler - INFO - launching cmd 2/3: nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 workdir polarisation=vv 2025-01-20 15:36:16,733 - wf_task - INFO - running nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 workdir polarisation=vv 2025-01-20 15:36:16,733 - wf_task - DEBUG - process-task: running command nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 workdir polarisation=vv (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 15:36:16,734 - wf_scheduler - INFO - launching cmd 3/3: nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 workdir polarisation=vv 2025-01-20 15:36:16,734 - wf_task - INFO - running nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 workdir polarisation=vv 2025-01-20 15:36:16,734 - wf_task - DEBUG - process-task: running command nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 workdir polarisation=vv (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 15:36:16,736 - wf_scheduler - DEBUG - status=['Running', 'Running', 'Running'] 2025-01-20 15:36:26,736 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/3 2025-01-20 15:36:26,736 - wf_scheduler - DEBUG - status=['Terminated', 'Terminated', 'Terminated'] 2025-01-20 15:36:26,737 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 15:36:26,737 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 15:36:26,740 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-0_download.txt 2025-01-20 15:36:26,750 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/1-1_get_dem.txt 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - updating procfile /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1/nsbas.proc 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - changing profile keys: atmo -> no, unw_method -> mpd, polarisation -> vv 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - changing lat bounds to 28.205833610842713, 29.275059417637532 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - changing polarisation bounds to vv 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - changing atmo paramater to no 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - changing rint and runw by 2 and 4 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - No constrain on Skip value, as Rlooks_int != Rlooks_unw 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - changing unw_method by mpd 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - Seed point not given by user 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - setting CheckUnwFrac to yes 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - Individual configuration for marie-pierre.doin@univ-grenoble-alpes.fr done 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - DEBUG - expert_param_list are ['filterstyle=SWg', 'PowerMode'] 2025-01-20 15:36:26,756 - wsc_run_gdm_sar.py - INFO - expert mode filterstyle as SWg 2025-01-20 15:36:26,759 - wsc_run_gdm_sar.py - INFO - updating procfile /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2/nsbas.proc 2025-01-20 15:36:26,759 - wsc_run_gdm_sar.py - INFO - changing profile keys: atmo -> no, unw_method -> mpd, polarisation -> vv 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - INFO - changing lat bounds to 28.205833610842713, 29.275059417637532 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - INFO - changing polarisation bounds to vv 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - INFO - changing atmo paramater to no 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - INFO - changing rint and runw by 2 and 4 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - INFO - No constrain on Skip value, as Rlooks_int != Rlooks_unw 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - INFO - changing unw_method by mpd 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - INFO - Seed point not given by user 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - INFO - setting CheckUnwFrac to yes 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - INFO - Individual configuration for marie-pierre.doin@univ-grenoble-alpes.fr done 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - DEBUG - expert_param_list are ['filterstyle=SWg', 'PowerMode'] 2025-01-20 15:36:26,760 - wsc_run_gdm_sar.py - INFO - expert mode filterstyle as SWg 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - updating procfile /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3/nsbas.proc 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - changing profile keys: atmo -> no, unw_method -> mpd, polarisation -> vv 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - changing lat bounds to 28.205833610842713, 29.275059417637532 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - changing polarisation bounds to vv 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - changing atmo paramater to no 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - changing rint and runw by 2 and 4 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - No constrain on Skip value, as Rlooks_int != Rlooks_unw 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - changing unw_method by mpd 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - Seed point not given by user 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - setting CheckUnwFrac to yes 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - Individual configuration for marie-pierre.doin@univ-grenoble-alpes.fr done 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - DEBUG - expert_param_list are ['filterstyle=SWg', 'PowerMode'] 2025-01-20 15:36:26,763 - wsc_run_gdm_sar.py - INFO - expert mode filterstyle as SWg 2025-01-20 15:36:26,766 - wsc_run_gdm_sar.py - INFO - ---------------> start burst_select 2025-01-20 15:36:26,766 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start make_slc', 'status': 'running', 'step': 'burst_select', 'progress': 18} 2025-01-20 15:36:26,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 15:36:27,017 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 15:36:27,018 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 15:36:27,018 - wsc_run_gdm_sar.py - INFO - %%%%% running task burst_select on swath iw1 2025-01-20 15:36:27,019 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 2 burst_select<< 2025-01-20 15:36:27,019 - wsc_run_gdm_sar.py - INFO - %%%%% running task burst_select on swath iw2 2025-01-20 15:36:27,019 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 2 burst_select<< 2025-01-20 15:36:27,019 - wsc_run_gdm_sar.py - INFO - %%%%% running task burst_select on swath iw3 2025-01-20 15:36:27,019 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 2 burst_select<< 2025-01-20 15:36:27,019 - wsc_run_gdm_sar.py - INFO - -------------- starting step 2: burst_select ---------- 2025-01-20 15:36:27,019 - wf_scheduler - INFO - Async scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 2 burst_select, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 2 burst_select, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 2 burst_select, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 15:36:27,019 - wf_scheduler - INFO - launching cmd 1/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 2 burst_select 2025-01-20 15:36:27,019 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=2,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 2 burst_select", "name": "2-0_burst_select_iw1_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw1.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw1.txt"}' 2025-01-20 15:36:27,019 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=2,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 2 burst_select", "name": "2-0_burst_select_iw1_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw1.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw1.txt"}' ***** 2025-01-20 15:36:27,337 - wf_task - INFO - run with pid 1651811 2025-01-20 15:36:27,337 - wf_scheduler - INFO - launching cmd 2/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 2 burst_select 2025-01-20 15:36:27,337 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=2,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 2 burst_select", "name": "2-0_burst_select_iw2_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw2.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw2.txt"}' 2025-01-20 15:36:27,337 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=2,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 2 burst_select", "name": "2-0_burst_select_iw2_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw2.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw2.txt"}' ***** 2025-01-20 15:36:27,656 - wf_task - INFO - run with pid 1651812 2025-01-20 15:36:27,656 - wf_scheduler - INFO - launching cmd 3/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 2 burst_select 2025-01-20 15:36:27,656 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=2,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 2 burst_select", "name": "2-0_burst_select_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw3.txt"}' 2025-01-20 15:36:27,656 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=2,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 2 burst_select", "name": "2-0_burst_select_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw3.txt"}' ***** 2025-01-20 15:36:27,974 - wf_task - INFO - run with pid 1651813 2025-01-20 15:36:27,974 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651811 2025-01-20 15:36:28,082 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651812 2025-01-20 15:36:28,131 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651813 2025-01-20 15:36:28,179 - wf_scheduler - DEBUG - status=['toLaunch', 'toLaunch', 'Waiting'] 2025-01-20 15:36:38,179 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/3 2025-01-20 15:36:38,179 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651811 2025-01-20 15:36:38,257 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651812 2025-01-20 15:36:38,311 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651813 2025-01-20 15:36:38,362 - wf_scheduler - DEBUG - status=['Running', 'Running', 'Waiting'] 2025-01-20 15:36:48,362 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/3 2025-01-20 15:36:48,362 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651811 2025-01-20 15:36:48,437 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651812 2025-01-20 15:36:48,487 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651813 2025-01-20 15:36:48,537 - wf_scheduler - DEBUG - status=['Running', 'Running', 'Running'] 2025-01-20 15:36:58,538 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/3 2025-01-20 15:36:58,538 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651811 2025-01-20 15:36:58,627 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651812 2025-01-20 15:36:58,682 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651813 2025-01-20 15:36:58,736 - wf_scheduler - DEBUG - status=['Terminated', 'Terminated', 'Running'] 2025-01-20 15:37:08,737 - wf_scheduler - DEBUG - nb_done/nb_todo = 2/3 2025-01-20 15:37:08,737 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651813 2025-01-20 15:37:08,820 - wf_scheduler - DEBUG - status=['Terminated', 'Terminated', 'Terminated'] 2025-01-20 15:37:08,820 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651811 2025-01-20 15:37:08,863 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651812 2025-01-20 15:37:08,909 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651813 2025-01-20 15:37:08,967 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 15:37:08,967 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 15:37:08,972 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw1.txt 2025-01-20 15:37:08,975 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw2.txt 2025-01-20 15:37:08,977 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw3.txt 2025-01-20 15:37:08,979 - wsc_run_gdm_sar.py - INFO - get cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wf_check_commons_date.py -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -m slice123 -v 4<< 2025-01-20 15:37:08,979 - wsc_run_gdm_sar.py - INFO - -------------- starting step 2: burst_select ---------- 2025-01-20 15:37:08,980 - wf_scheduler - INFO - Async scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wf_check_commons_date.py -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -m slice123 -v 4, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 15:37:08,980 - wf_scheduler - INFO - launching cmd 1/1: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wf_check_commons_date.py -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -m slice123 -v 4 2025-01-20 15:37:08,980 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=2,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wf_check_commons_date.py -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -m slice123 -v 4", "name": "2-1_burst_select_check_iw1_iw2_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-1_burst_select_check_iw1_iw2_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-1_burst_select_check_iw1_iw2_iw3.txt"}' 2025-01-20 15:37:08,980 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=2,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; wf_check_commons_date.py -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -m slice123 -v 4", "name": "2-1_burst_select_check_iw1_iw2_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-1_burst_select_check_iw1_iw2_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-1_burst_select_check_iw1_iw2_iw3.txt"}' ***** 2025-01-20 15:37:09,320 - wf_task - INFO - run with pid 1651814 2025-01-20 15:37:09,320 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651814 2025-01-20 15:37:09,410 - wf_scheduler - DEBUG - status=['Waiting'] 2025-01-20 15:37:19,411 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/1 2025-01-20 15:37:19,411 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651814 2025-01-20 15:37:19,493 - wf_scheduler - DEBUG - status=['Waiting'] 2025-01-20 15:37:29,494 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/1 2025-01-20 15:37:29,494 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651814 2025-01-20 15:37:29,572 - wf_scheduler - DEBUG - status=['Terminated'] 2025-01-20 15:37:29,572 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651814 2025-01-20 15:37:29,657 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 15:37:29,657 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 15:37:29,661 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw1.txt 2025-01-20 15:37:29,664 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw2.txt 2025-01-20 15:37:29,665 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-0_burst_select_iw3.txt 2025-01-20 15:37:29,666 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/2-1_burst_select_check_iw1_iw2_iw3.txt 2025-01-20 15:37:29,669 - wsc_run_gdm_sar.py - INFO - ---------------> start slice 2025-01-20 15:37:29,669 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start slice', 'status': 'running', 'step': 'make_slc', 'progress': 27} 2025-01-20 15:37:29,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 15:37:29,925 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 15:37:29,926 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 15:37:29,926 - wsc_run_gdm_sar.py - INFO - %%%%% running task slice123 on swath iw1 2025-01-20 15:37:29,926 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 slice123<< 2025-01-20 15:37:29,926 - wsc_run_gdm_sar.py - INFO - %%%%% running task slice123 on swath iw2 2025-01-20 15:37:29,926 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 slice123<< 2025-01-20 15:37:29,926 - wsc_run_gdm_sar.py - INFO - %%%%% running task slice123 on swath iw3 2025-01-20 15:37:29,926 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 slice123<< 2025-01-20 15:37:29,927 - wsc_run_gdm_sar.py - INFO - -------------- starting step 3: make_slc ---------- 2025-01-20 15:37:29,927 - wf_scheduler - INFO - Async scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 slice123, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 slice123, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 slice123, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 15:37:29,927 - wf_scheduler - INFO - launching cmd 1/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 slice123 2025-01-20 15:37:29,927 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 slice123", "name": "3_slice123_iw1_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw1.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw1.txt"}' 2025-01-20 15:37:29,927 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 slice123", "name": "3_slice123_iw1_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw1.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw1.txt"}' ***** 2025-01-20 15:37:30,236 - wf_task - INFO - run with pid 1651815 2025-01-20 15:37:30,237 - wf_scheduler - INFO - launching cmd 2/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 slice123 2025-01-20 15:37:30,237 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 slice123", "name": "3_slice123_iw2_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw2.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw2.txt"}' 2025-01-20 15:37:30,237 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 slice123", "name": "3_slice123_iw2_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw2.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw2.txt"}' ***** 2025-01-20 15:37:30,571 - wf_task - INFO - run with pid 1651816 2025-01-20 15:37:30,571 - wf_scheduler - INFO - launching cmd 3/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 slice123 2025-01-20 15:37:30,571 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 slice123", "name": "3_slice123_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw3.txt"}' 2025-01-20 15:37:30,571 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=01:26:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 slice123", "name": "3_slice123_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw3.txt"}' ***** 2025-01-20 15:37:30,912 - wf_task - INFO - run with pid 1651817 2025-01-20 15:37:30,913 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651815 2025-01-20 15:37:31,025 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651816 2025-01-20 15:37:31,074 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651817 2025-01-20 15:37:31,122 - wf_scheduler - DEBUG - status=['Waiting', 'Waiting', 'Waiting'] 2025-01-20 15:47:31,123 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/3 2025-01-20 15:47:31,123 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651815 2025-01-20 15:47:31,208 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651816 2025-01-20 15:47:31,298 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651817 2025-01-20 15:47:31,354 - wf_scheduler - DEBUG - status=['Terminated', 'Terminated', 'Terminated'] 2025-01-20 15:47:31,354 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651815 2025-01-20 15:47:31,410 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651816 2025-01-20 15:47:31,466 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651817 2025-01-20 15:47:31,520 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 15:47:31,521 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 15:47:31,526 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw1.txt 2025-01-20 15:47:31,536 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw2.txt 2025-01-20 15:47:31,541 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/3_slice123_iw3.txt 2025-01-20 15:47:31,547 - wsc_run_gdm_sar.py - INFO - ---------------> start simulation 2025-01-20 15:47:31,549 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start simulation', 'status': 'running', 'step': 'simulation', 'progress': 36} 2025-01-20 15:47:31,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 15:47:31,827 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 15:47:31,828 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 15:47:31,828 - wsc_run_gdm_sar.py - INFO - %%%%% running task simulation123 on swath iw1 2025-01-20 15:47:31,829 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 simulation123<< 2025-01-20 15:47:31,829 - wsc_run_gdm_sar.py - INFO - %%%%% running task simulation123 on swath iw2 2025-01-20 15:47:31,829 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 simulation123<< 2025-01-20 15:47:31,829 - wsc_run_gdm_sar.py - INFO - %%%%% running task simulation123 on swath iw3 2025-01-20 15:47:31,829 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 simulation123<< 2025-01-20 15:47:31,829 - wsc_run_gdm_sar.py - INFO - -------------- starting step 4: simulation ---------- 2025-01-20 15:47:31,829 - wf_scheduler - INFO - Async scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 simulation123, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 simulation123, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 simulation123, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 15:47:31,829 - wf_scheduler - INFO - launching cmd 1/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 simulation123 2025-01-20 15:47:31,829 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=02:52:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 simulation123", "name": "4_simulation123_iw1_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw1.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw1.txt"}' 2025-01-20 15:47:31,830 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=02:52:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 simulation123", "name": "4_simulation123_iw1_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw1.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw1.txt"}' ***** 2025-01-20 15:47:32,133 - wf_task - INFO - run with pid 1651818 2025-01-20 15:47:32,133 - wf_scheduler - INFO - launching cmd 2/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 simulation123 2025-01-20 15:47:32,133 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=02:52:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 simulation123", "name": "4_simulation123_iw2_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw2.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw2.txt"}' 2025-01-20 15:47:32,133 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=02:52:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 simulation123", "name": "4_simulation123_iw2_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw2.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw2.txt"}' ***** 2025-01-20 15:47:32,450 - wf_task - INFO - run with pid 1651819 2025-01-20 15:47:32,450 - wf_scheduler - INFO - launching cmd 3/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 simulation123 2025-01-20 15:47:32,450 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=02:52:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 simulation123", "name": "4_simulation123_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw3.txt"}' 2025-01-20 15:47:32,450 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=02:52:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 simulation123", "name": "4_simulation123_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw3.txt"}' ***** 2025-01-20 15:47:32,789 - wf_task - INFO - run with pid 1651820 2025-01-20 15:47:32,789 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651818 2025-01-20 15:47:32,875 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651819 2025-01-20 15:47:32,924 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651820 2025-01-20 15:47:32,970 - wf_scheduler - DEBUG - status=['Launching', 'Launching', 'Waiting'] 2025-01-20 15:57:32,971 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/3 2025-01-20 15:57:32,971 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651818 2025-01-20 15:57:33,071 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651819 2025-01-20 15:57:33,124 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651820 2025-01-20 15:57:33,175 - wf_scheduler - DEBUG - status=['Running', 'Running', 'Running'] 2025-01-20 16:07:33,176 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/3 2025-01-20 16:07:33,176 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651818 2025-01-20 16:07:33,269 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651819 2025-01-20 16:07:33,327 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651820 2025-01-20 16:07:33,383 - wf_scheduler - DEBUG - status=['Terminated', 'Terminated', 'Terminated'] 2025-01-20 16:07:33,384 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651818 2025-01-20 16:07:33,442 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651819 2025-01-20 16:07:33,498 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651820 2025-01-20 16:07:33,557 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 16:07:33,557 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 16:07:33,562 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw1.txt 2025-01-20 16:07:33,589 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw2.txt 2025-01-20 16:07:33,615 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/4_simulation123_iw3.txt 2025-01-20 16:07:33,641 - wsc_run_gdm_sar.py - INFO - ---------------> start interfero_spectraldiversity 2025-01-20 16:07:33,642 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start interfero_spectraldiversity', 'status': 'running', 'step': 'interfero_spectraldiversity', 'progress': 45} 2025-01-20 16:07:33,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 16:07:33,972 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 16:07:33,973 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 16:07:33,974 - wsc_run_gdm_sar.py - INFO - %%%%% running task interfero_spectraldiversity on swath iw1 2025-01-20 16:07:33,974 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 interfero_spectraldiversity<< 2025-01-20 16:07:33,974 - wsc_run_gdm_sar.py - INFO - %%%%% running task interfero_spectraldiversity on swath iw2 2025-01-20 16:07:33,974 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 interfero_spectraldiversity<< 2025-01-20 16:07:33,974 - wsc_run_gdm_sar.py - INFO - %%%%% running task interfero_spectraldiversity on swath iw3 2025-01-20 16:07:33,974 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 interfero_spectraldiversity<< 2025-01-20 16:07:33,974 - wsc_run_gdm_sar.py - INFO - -------------- starting step 5: interfero_spectraldiversity ---------- 2025-01-20 16:07:33,974 - wf_scheduler - INFO - Async scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 interfero_spectraldiversity, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 interfero_spectraldiversity, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 interfero_spectraldiversity, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 16:07:33,974 - wf_scheduler - INFO - launching cmd 1/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 interfero_spectraldiversity 2025-01-20 16:07:33,975 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=06:25:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 interfero_spectraldiversity", "name": "5_interfero_spectraldiversity_iw1_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw1.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw1.txt"}' 2025-01-20 16:07:33,975 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=06:25:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1 12 interfero_spectraldiversity", "name": "5_interfero_spectraldiversity_iw1_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw1.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw1.txt"}' ***** 2025-01-20 16:07:34,309 - wf_task - INFO - run with pid 1651825 2025-01-20 16:07:34,309 - wf_scheduler - INFO - launching cmd 2/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 interfero_spectraldiversity 2025-01-20 16:07:34,309 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=06:25:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 interfero_spectraldiversity", "name": "5_interfero_spectraldiversity_iw2_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw2.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw2.txt"}' 2025-01-20 16:07:34,309 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=06:25:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2 12 interfero_spectraldiversity", "name": "5_interfero_spectraldiversity_iw2_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw2.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw2.txt"}' ***** 2025-01-20 16:07:34,651 - wf_task - INFO - run with pid 1651826 2025-01-20 16:07:34,651 - wf_scheduler - INFO - launching cmd 3/3: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 interfero_spectraldiversity 2025-01-20 16:07:34,651 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=06:25:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 interfero_spectraldiversity", "name": "5_interfero_spectraldiversity_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw3.txt"}' 2025-01-20 16:07:34,652 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=06:25:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3 12 interfero_spectraldiversity", "name": "5_interfero_spectraldiversity_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw3.txt"}' ***** 2025-01-20 16:07:34,995 - wf_task - INFO - run with pid 1651827 2025-01-20 16:07:34,996 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651825 2025-01-20 16:07:35,103 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651826 2025-01-20 16:07:35,151 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651827 2025-01-20 16:07:35,233 - wf_scheduler - DEBUG - status=['toLaunch', 'toLaunch', 'Waiting'] 2025-01-20 16:17:35,233 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/3 2025-01-20 16:17:35,233 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651825 2025-01-20 16:17:35,334 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651826 2025-01-20 16:17:35,390 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651827 2025-01-20 16:17:35,445 - wf_scheduler - DEBUG - status=['Running', 'Running', 'Running'] 2025-01-20 16:27:35,445 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/3 2025-01-20 16:27:35,445 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651825 2025-01-20 16:27:35,522 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651826 2025-01-20 16:27:35,570 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651827 2025-01-20 16:27:35,618 - wf_scheduler - DEBUG - status=['Terminated', 'Terminated', 'Terminated'] 2025-01-20 16:27:35,618 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651825 2025-01-20 16:27:35,669 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651826 2025-01-20 16:27:35,717 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651827 2025-01-20 16:27:35,768 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 16:27:35,768 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 16:27:35,773 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw1.txt 2025-01-20 16:27:35,826 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw2.txt 2025-01-20 16:27:35,878 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/5_interfero_spectraldiversity_iw3.txt 2025-01-20 16:27:35,929 - wsc_run_gdm_sar.py - INFO - -----------> start merge if needed 2025-01-20 16:27:35,932 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start merge', 'status': 'running', 'step': 'merge', 'progress': 54} 2025-01-20 16:27:35,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 16:27:36,248 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 16:27:36,250 - wsc_run_gdm_sar.py - INFO - -----------> merge is needed 2025-01-20 16:27:36,250 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 16:27:36,251 - wsc_run_gdm_sar.py - INFO - %%%%% running task merge on swath iw123 2025-01-20 16:27:36,251 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw123 12 merge<< 2025-01-20 16:27:36,251 - wsc_run_gdm_sar.py - INFO - -------------- starting step 6: merge ---------- 2025-01-20 16:27:36,251 - wf_scheduler - INFO - Async scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw123 12 merge, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 16:27:36,251 - wf_scheduler - INFO - launching cmd 1/1: source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw123 12 merge 2025-01-20 16:27:36,251 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=02:09:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw123 12 merge", "name": "6_merge_iw123_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/6_merge_iw123.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/6_merge_iw123.txt"}' 2025-01-20 16:27:36,252 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=02:09:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw123 12 merge", "name": "6_merge_iw123_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/6_merge_iw123.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/6_merge_iw123.txt"}' ***** 2025-01-20 16:27:36,558 - wf_task - INFO - run with pid 1651829 2025-01-20 16:27:36,558 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651829 2025-01-20 16:27:36,633 - wf_scheduler - DEBUG - status=['Waiting'] 2025-01-20 16:37:36,633 - wf_scheduler - DEBUG - nb_done/nb_todo = 0/1 2025-01-20 16:37:36,633 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651829 2025-01-20 16:37:36,719 - wf_scheduler - DEBUG - status=['Terminated'] 2025-01-20 16:37:36,719 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651829 2025-01-20 16:37:36,769 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 16:37:36,769 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 16:37:36,774 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/6_merge_iw123.txt 2025-01-20 16:37:36,792 - wsc_run_gdm_sar.py - INFO - -------------> start atmo filter unwrap 2025-01-20 16:37:36,793 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start filter_unwrap', 'status': 'running', 'step': 'filter_unwrap', 'progress': 63} 2025-01-20 16:37:36,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 16:37:37,036 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 16:37:37,038 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 16:37:37,038 - wsc_run_gdm_sar.py - INFO - %%%%% running task caui on swath iw1_iw2_iw3 2025-01-20 16:37:37,038 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 caui<< 2025-01-20 16:37:37,038 - wsc_run_gdm_sar.py - INFO - -------------- starting step 7: filter_unwrap ---------- 2025-01-20 16:37:37,038 - wf_scheduler - INFO - Sequential scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 caui, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 16:37:37,038 - wf_scheduler - INFO - launching cmd 0/1: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 caui, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 16:37:37,039 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=48:07:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 caui", "name": "7_caui_iw1_iw2_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/7_caui_iw1_iw2_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/7_caui_iw1_iw2_iw3.txt"}' 2025-01-20 16:37:37,039 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=48:07:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 caui", "name": "7_caui_iw1_iw2_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/7_caui_iw1_iw2_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/7_caui_iw1_iw2_iw3.txt"}' ***** 2025-01-20 16:37:37,354 - wf_task - INFO - run with pid 1651830 2025-01-20 16:38:37,354 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:39:37,438 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:40:37,525 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:41:37,604 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:42:37,688 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:43:37,762 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:44:37,846 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:45:37,936 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:46:38,025 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:47:38,122 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:48:38,231 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:49:38,318 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:50:38,404 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:51:38,478 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:52:38,561 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:53:38,655 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:54:38,748 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651830 2025-01-20 16:54:38,837 - wf_task - INFO - ret_code for source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 caui is 0 2025-01-20 16:54:38,837 - wf_scheduler - INFO - done with cmd 0/1: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsbas.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 caui, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=1651830 2025-01-20 16:54:38,837 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 16:54:38,837 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 16:54:38,843 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/7_caui_iw1_iw2_iw3.txt 2025-01-20 16:54:38,869 - wsc_run_gdm_sar.py - DEBUG - number of interfero 6 2025-01-20 16:54:38,869 - wsc_run_gdm_sar.py - INFO - -------------> start cleaning if requested 2025-01-20 16:54:38,870 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start cleaning_before_ts', 'status': 'running', 'step': 'timeseries', 'progress': 72} 2025-01-20 16:54:38,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 16:54:39,164 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 16:54:39,165 - wsc_run_gdm_sar.py - INFO - -------------> cleaning coreg, INTERFERO_sd and interferograms before merging 2025-01-20 16:54:39,165 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 16:54:39,166 - wsc_run_gdm_sar.py - INFO - %%%%% running task cleaning_before_ts on swath iw1 2025-01-20 16:54:39,166 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1/nsbas.proc -v<< 2025-01-20 16:54:39,166 - wsc_run_gdm_sar.py - INFO - %%%%% running task cleaning_before_ts on swath iw2 2025-01-20 16:54:39,166 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2/nsbas.proc -v<< 2025-01-20 16:54:39,166 - wsc_run_gdm_sar.py - INFO - %%%%% running task cleaning_before_ts on swath iw3 2025-01-20 16:54:39,166 - wsc_run_gdm_sar.py - INFO - get oartask: nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3/nsbas.proc -v<< 2025-01-20 16:54:39,166 - wsc_run_gdm_sar.py - INFO - -------------- starting step 8: timeseries ---------- 2025-01-20 16:54:39,166 - wf_scheduler - INFO - Sequential scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1/nsbas.proc -v, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2/nsbas.proc -v, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3/nsbas.proc -v, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 16:54:39,166 - wf_scheduler - INFO - launching cmd 0/3: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1/nsbas.proc -v, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 16:54:39,167 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=12:02:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1/nsbas.proc -v", "name": "8_cleaning_before_ts_iw1_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw1.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw1.txt"}' 2025-01-20 16:54:39,167 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=12:02:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1/nsbas.proc -v", "name": "8_cleaning_before_ts_iw1_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw1.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw1.txt"}' ***** 2025-01-20 16:54:39,480 - wf_task - INFO - run with pid 1651831 2025-01-20 16:55:39,480 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651831 2025-01-20 16:55:39,566 - wf_task - INFO - ret_code for source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1/nsbas.proc -v is 0 2025-01-20 16:55:39,567 - wf_scheduler - INFO - done with cmd 0/3: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1/nsbas.proc -v, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=1651831 2025-01-20 16:55:39,567 - wf_scheduler - INFO - launching cmd 1/3: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2/nsbas.proc -v, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 16:55:39,567 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=12:02:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2/nsbas.proc -v", "name": "8_cleaning_before_ts_iw2_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw2.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw2.txt"}' 2025-01-20 16:55:39,567 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=12:02:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2/nsbas.proc -v", "name": "8_cleaning_before_ts_iw2_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw2.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw2.txt"}' ***** 2025-01-20 16:55:41,059 - wf_task - INFO - run with pid 1651833 2025-01-20 16:56:41,059 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651833 2025-01-20 16:56:41,142 - wf_task - INFO - ret_code for source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2/nsbas.proc -v is 0 2025-01-20 16:56:41,142 - wf_scheduler - INFO - done with cmd 1/3: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2/nsbas.proc -v, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=1651833 2025-01-20 16:56:41,142 - wf_scheduler - INFO - launching cmd 2/3: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3/nsbas.proc -v, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 16:56:41,142 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12,walltime=12:02:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3/nsbas.proc -v", "name": "8_cleaning_before_ts_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw3.txt"}' 2025-01-20 16:56:41,142 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12,walltime=12:02:00", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3/nsbas.proc -v", "name": "8_cleaning_before_ts_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw3.txt"}' ***** 2025-01-20 16:56:41,416 - wf_task - INFO - run with pid 1651834 2025-01-20 16:57:41,417 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651834 2025-01-20 16:57:41,518 - wf_task - INFO - ret_code for source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3/nsbas.proc -v is 0 2025-01-20 16:57:41,518 - wf_scheduler - INFO - done with cmd 2/3: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_clean_coreg_int.py -c -s -i -t /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3/nsbas.proc -v, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=1651834 2025-01-20 16:57:41,518 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 16:57:41,518 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 16:57:41,524 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw1.txt 2025-01-20 16:57:41,526 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw2.txt 2025-01-20 16:57:41,528 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/8_cleaning_before_ts_iw3.txt 2025-01-20 16:57:41,530 - wsc_run_gdm_sar.py - INFO - -------------> start time series if needed 2025-01-20 16:57:41,530 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start timeseries', 'status': 'running', 'step': 'timeseries', 'progress': 72} 2025-01-20 16:57:41,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 16:57:41,782 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 16:57:41,783 - wsc_run_gdm_sar.py - INFO - -------------> skipping timeseries 2025-01-20 16:57:41,784 - wsc_run_gdm_sar.py - INFO - -------------> start cleaning if requested 2025-01-20 16:57:41,784 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start time_serie', 'status': 'running', 'step': 'timeseries', 'progress': 72} 2025-01-20 16:57:41,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 16:57:42,037 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 16:57:42,038 - wsc_run_gdm_sar.py - INFO - -------------> skipping cleaning 2025-01-20 16:57:42,038 - wsc_run_gdm_sar.py - INFO - -------------> start populate and make products 2025-01-20 16:57:42,039 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'Start publish', 'status': 'running', 'step': 'make_data', 'progress': 81} 2025-01-20 16:57:42,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 16:57:42,282 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 16:57:42,283 - wsc_run_gdm_sar.py - DEBUG - populate ctip calcall and ltip InW_radar_runw,InW_geo_runw,APS_radar_runw,APS_geo_runw,InU_geo_runw,InWF_radar_runw,InWF_geo_runw,Coh_radar_runw,Coh_geo_runw,CosENU_radar_runw,CosENU_geo_runw,CosNEU_radar_runw,CosNEU_geo_runw,DEM_radar_runw,DEM_geo_runw,Lut_geo_runw,Lut_radar_runw,aux_files,InU_radar_runw,TCoh_radar_runw,TCoh_geo_runw 2025-01-20 16:57:42,283 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 16:57:42,284 - wsc_run_gdm_sar.py - DEBUG - get nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_populate_product.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 calcall epos -v ln -id GDMSARUGA<< 2025-01-20 16:57:42,284 - wsc_run_gdm_sar.py - INFO - -------------- starting step 9: make_data ---------- 2025-01-20 16:57:42,284 - wf_scheduler - INFO - Sequential scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_populate_product.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 calcall epos -v ln -id GDMSARUGA, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 16:57:42,284 - wf_scheduler - INFO - launching cmd 0/1: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_populate_product.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 calcall epos -v ln -id GDMSARUGA, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 16:57:42,284 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_populate_product.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 calcall epos -v ln -id GDMSARUGA", "name": "9-0_populate_iw1_iw2_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-0_populate_iw1_iw2_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-0_populate_iw1_iw2_iw3.txt"}' 2025-01-20 16:57:42,284 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_populate_product.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 calcall epos -v ln -id GDMSARUGA", "name": "9-0_populate_iw1_iw2_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-0_populate_iw1_iw2_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-0_populate_iw1_iw2_iw3.txt"}' ***** 2025-01-20 16:57:42,624 - wf_task - INFO - run with pid 1651835 2025-01-20 16:58:42,625 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 16:59:42,728 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 17:00:42,817 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 17:01:42,912 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 17:02:43,022 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 17:03:43,108 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 17:04:43,212 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 17:05:43,284 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 17:06:43,357 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 17:07:43,434 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 17:08:43,512 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651835 2025-01-20 17:08:43,605 - wf_task - INFO - ret_code for source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_populate_product.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 calcall epos -v ln -id GDMSARUGA is 0 2025-01-20 17:08:43,606 - wf_scheduler - INFO - done with cmd 0/1: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_populate_product.sh /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 12 calcall epos -v ln -id GDMSARUGA, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=1651835 2025-01-20 17:08:43,606 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 17:08:43,606 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 17:08:43,611 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-0_populate_iw1_iw2_iw3.txt 2025-01-20 17:08:43,623 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'select products', 'status': 'running', 'step': 'make_data', 'progress': 81} 2025-01-20 17:08:43,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 17:08:43,932 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 17:08:43,934 - wsc_run_gdm_sar.py - INFO - Computing on ist-oar, walltime ceiling of 480 hours 2025-01-20 17:08:43,934 - wsc_run_gdm_sar.py - INFO - get nsbas cmd = >>source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_data.py -s /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -w iw1_iw2_iw3 -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product -c 12 -t InW_radar_runw,InW_geo_runw,APS_radar_runw,APS_geo_runw,InU_geo_runw,InWF_radar_runw,InWF_geo_runw,Coh_radar_runw,Coh_geo_runw,CosENU_radar_runw,CosENU_geo_runw,CosNEU_radar_runw,CosNEU_geo_runw,DEM_radar_runw,DEM_geo_runw,Lut_geo_runw,Lut_radar_runw,aux_files,InU_radar_runw,TCoh_radar_runw,TCoh_geo_runw -j epos -m -p -g -v 4 -b backup_product.json -plot -mask<< 2025-01-20 17:08:43,934 - wsc_run_gdm_sar.py - INFO - -------------- starting step 9: make_data ---------- 2025-01-20 17:08:43,935 - wf_scheduler - INFO - Sequential scheduler: running command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_data.py -s /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -w iw1_iw2_iw3 -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product -c 12 -t InW_radar_runw,InW_geo_runw,APS_radar_runw,APS_geo_runw,InU_geo_runw,InWF_radar_runw,InWF_geo_runw,Coh_radar_runw,Coh_geo_runw,CosENU_radar_runw,CosENU_geo_runw,CosNEU_radar_runw,CosNEU_geo_runw,DEM_radar_runw,DEM_geo_runw,Lut_geo_runw,Lut_radar_runw,aux_files,InU_radar_runw,TCoh_radar_runw,TCoh_geo_runw -j epos -m -p -g -v 4 -b backup_product.json -plot -mask, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 17:08:43,935 - wf_scheduler - INFO - launching cmd 0/1: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_data.py -s /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -w iw1_iw2_iw3 -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product -c 12 -t InW_radar_runw,InW_geo_runw,APS_radar_runw,APS_geo_runw,InU_geo_runw,InWF_radar_runw,InWF_geo_runw,Coh_radar_runw,Coh_geo_runw,CosENU_radar_runw,CosENU_geo_runw,CosNEU_radar_runw,CosNEU_geo_runw,DEM_radar_runw,DEM_geo_runw,Lut_geo_runw,Lut_radar_runw,aux_files,InU_radar_runw,TCoh_radar_runw,TCoh_geo_runw -j epos -m -p -g -v 4 -b backup_product.json -plot -mask, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 17:08:43,935 - wf_task - DEBUG - oar request: http://localhost:6668/oarapi/jobs data=b'{"resource": "/nodes=1/core=12", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_data.py -s /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -w iw1_iw2_iw3 -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product -c 12 -t InW_radar_runw,InW_geo_runw,APS_radar_runw,APS_geo_runw,InU_geo_runw,InWF_radar_runw,InWF_geo_runw,Coh_radar_runw,Coh_geo_runw,CosENU_radar_runw,CosENU_geo_runw,CosNEU_radar_runw,CosNEU_geo_runw,DEM_radar_runw,DEM_geo_runw,Lut_geo_runw,Lut_radar_runw,aux_files,InU_radar_runw,TCoh_radar_runw,TCoh_geo_runw -j epos -m -p -g -v 4 -b backup_product.json -plot -mask", "name": "9-1_make_iw1_iw2_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-1_make_iw1_iw2_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-1_make_iw1_iw2_iw3.txt"}' 2025-01-20 17:08:43,935 - wf_task - INFO - ***** PARAMS : b'{"resource": "/nodes=1/core=12", "command": "source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_data.py -s /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -w iw1_iw2_iw3 -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product -c 12 -t InW_radar_runw,InW_geo_runw,APS_radar_runw,APS_geo_runw,InU_geo_runw,InWF_radar_runw,InWF_geo_runw,Coh_radar_runw,Coh_geo_runw,CosENU_radar_runw,CosENU_geo_runw,CosNEU_radar_runw,CosNEU_geo_runw,DEM_radar_runw,DEM_geo_runw,Lut_geo_runw,Lut_radar_runw,aux_files,InU_radar_runw,TCoh_radar_runw,TCoh_geo_runw -j epos -m -p -g -v 4 -b backup_product.json -plot -mask", "name": "9-1_make_iw1_iw2_iw3_df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "workdir": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f", "project": "f-isdeform", "property": "network_address=\'ist-calcul17.u-ga.fr\'", "stdout": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-1_make_iw1_iw2_iw3.txt", "stderr": "/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-1_make_iw1_iw2_iw3.txt"}' ***** 2025-01-20 17:08:44,255 - wf_task - INFO - run with pid 1651836 2025-01-20 17:09:44,255 - wf_task - DEBUG - requesting : http://localhost:6668/oarapi/jobs/1651836 2025-01-20 17:09:44,340 - wf_task - INFO - ret_code for source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_data.py -s /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -w iw1_iw2_iw3 -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product -c 12 -t InW_radar_runw,InW_geo_runw,APS_radar_runw,APS_geo_runw,InU_geo_runw,InWF_radar_runw,InWF_geo_runw,Coh_radar_runw,Coh_geo_runw,CosENU_radar_runw,CosENU_geo_runw,CosNEU_radar_runw,CosNEU_geo_runw,DEM_radar_runw,DEM_geo_runw,Lut_geo_runw,Lut_radar_runw,aux_files,InU_radar_runw,TCoh_radar_runw,TCoh_geo_runw -j epos -m -p -g -v 4 -b backup_product.json -plot -mask is 0 2025-01-20 17:09:44,340 - wf_scheduler - INFO - done with cmd 0/1: command=source /usr/contrib/cycle/nsbas_env/env_isdeform.sh; nsb_make_data.py -s /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f -w iw1_iw2_iw3 -d /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product -c 12 -t InW_radar_runw,InW_geo_runw,APS_radar_runw,APS_geo_runw,InU_geo_runw,InWF_radar_runw,InWF_geo_runw,Coh_radar_runw,Coh_geo_runw,CosENU_radar_runw,CosENU_geo_runw,CosNEU_radar_runw,CosNEU_geo_runw,DEM_radar_runw,DEM_geo_runw,Lut_geo_runw,Lut_radar_runw,aux_files,InU_radar_runw,TCoh_radar_runw,TCoh_geo_runw -j epos -m -p -g -v 4 -b backup_product.json -plot -mask, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=1651836 2025-01-20 17:09:44,340 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 17:09:44,340 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 17:09:44,346 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-0_populate_iw1_iw2_iw3.txt 2025-01-20 17:09:44,359 - wsc_run_gdm_sar.py - INFO - checking /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs/9-1_make_iw1_iw2_iw3.txt 2025-01-20 17:09:44,383 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'format products', 'status': 'running', 'step': 'make_data', 'progress': 81} 2025-01-20 17:09:44,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 17:09:44,626 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 17:09:44,627 - wsc_run_gdm_sar.py - INFO - Starting building job summary... 2025-01-20 17:09:48,774 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_2rlks.trans is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1/20241224/geomap_quad_2rlks.trans'), ignoring it's size. 2025-01-20 17:09:48,806 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_2rlks.trans is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw3/20241224/geomap_quad_2rlks.trans'), ignoring it's size. 2025-01-20 17:09:48,851 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_2rlks.trans is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw2/20241224/geomap_quad_2rlks.trans'), ignoring it's size. 2025-01-20 17:09:48,893 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241212_20241224/geomap_quad_4rlks.trans'), ignoring it's size. 2025-01-20 17:09:48,893 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241212_20241224/geomap_quad_4rlks.trans.rsc'), ignoring it's size. 2025-01-20 17:09:48,893 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc.hst is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241212_20241224/geomap_quad_4rlks.trans.rsc.hst'), ignoring it's size. 2025-01-20 17:09:48,895 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241212_20250105/geomap_quad_4rlks.trans'), ignoring it's size. 2025-01-20 17:09:48,895 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241212_20250105/geomap_quad_4rlks.trans.rsc'), ignoring it's size. 2025-01-20 17:09:48,895 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc.hst is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241212_20250105/geomap_quad_4rlks.trans.rsc.hst'), ignoring it's size. 2025-01-20 17:09:48,897 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241212_20250117/geomap_quad_4rlks.trans'), ignoring it's size. 2025-01-20 17:09:48,897 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241212_20250117/geomap_quad_4rlks.trans.rsc'), ignoring it's size. 2025-01-20 17:09:48,897 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc.hst is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241212_20250117/geomap_quad_4rlks.trans.rsc.hst'), ignoring it's size. 2025-01-20 17:09:48,900 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241224_20250105/geomap_quad_4rlks.trans'), ignoring it's size. 2025-01-20 17:09:48,900 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241224_20250105/geomap_quad_4rlks.trans.rsc'), ignoring it's size. 2025-01-20 17:09:48,900 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc.hst is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241224_20250105/geomap_quad_4rlks.trans.rsc.hst'), ignoring it's size. 2025-01-20 17:09:48,903 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241224_20250117/geomap_quad_4rlks.trans'), ignoring it's size. 2025-01-20 17:09:48,903 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241224_20250117/geomap_quad_4rlks.trans.rsc'), ignoring it's size. 2025-01-20 17:09:48,903 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc.hst is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20241224_20250117/geomap_quad_4rlks.trans.rsc.hst'), ignoring it's size. 2025-01-20 17:09:48,906 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20250105_20250117/geomap_quad_4rlks.trans'), ignoring it's size. 2025-01-20 17:09:48,906 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20250105_20250117/geomap_quad_4rlks.trans.rsc'), ignoring it's size. 2025-01-20 17:09:48,906 - wsc_run_gdm_sar.py - WARNING - File /home/isterrem/geomap_quad_4rlks.trans.rsc.hst is not accessible ([Errno 2] No such file or directory: '/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/INTERFERO/int_20250105_20250117/geomap_quad_4rlks.trans.rsc.hst'), ignoring it's size. 2025-01-20 17:09:48,963 - wsc_run_gdm_sar.py - INFO - Building job summary done! 2025-01-20 17:09:48,970 - nsb_build_synthesis - INFO - writing to /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/0_synthesis.html 2025-01-20 17:09:48,970 - nsb_build_synthesis - INFO - launched as /mnt/contrib/cycle/gdm-sar-uga/ws/nsb_build_synthesis.py -o /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/0_synthesis.html -c /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/0_summary.json -w /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product -v 4 2025-01-20 17:09:48,970 - nsb_build_synthesis - DEBUG - getting orbit types 2025-01-20 17:09:48,972 - nsb_build_synthesis - DEBUG - getting info from summary 2025-01-20 17:09:48,974 - nsb_build_synthesis - DEBUG - getting info from summary --> Done 2025-01-20 17:09:48,984 - nsb_build_synthesis - DEBUG - building step_time_plot 2025-01-20 17:09:48,985 - nsb_build_synthesis - DEBUG - processing token: df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f 2025-01-20 17:09:48,986 - nsb_build_synthesis - DEBUG - data collection done, preparing plot 2025-01-20 17:09:49,120 - nsb_build_synthesis - DEBUG - after subplot 2025-01-20 17:09:49,120 - nsb_build_synthesis - DEBUG - step_stats keys ['1-0', '1-1', '2-0', '2-1', '3', '4', '5', '6', '7', '8', '9-0', '9-1'] 2025-01-20 17:09:49,120 - nsb_build_synthesis - DEBUG - after sort 2025-01-20 17:09:49,120 - nsb_build_synthesis - DEBUG - 1: before loop 2025-01-20 17:09:49,120 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,120 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=download, minutes=5 2025-01-20 17:09:49,182 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,182 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=dem, minutes=1 2025-01-20 17:09:49,185 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,185 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=burst_select_iw1, minutes=1 2025-01-20 17:09:49,187 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=burst_select_iw2, minutes=1 2025-01-20 17:09:49,188 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=burst_select_iw3, minutes=1 2025-01-20 17:09:49,189 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,189 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=burst_select_check_iw1_iw2_iw3, minutes=1 2025-01-20 17:09:49,190 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,190 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=slice123_iw1, minutes=2 2025-01-20 17:09:49,191 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=slice123_iw2, minutes=1 2025-01-20 17:09:49,192 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=slice123_iw3, minutes=2 2025-01-20 17:09:49,193 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,193 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=simulation123_iw1, minutes=15 2025-01-20 17:09:49,194 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=simulation123_iw2, minutes=16 2025-01-20 17:09:49,194 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=simulation123_iw3, minutes=15 2025-01-20 17:09:49,195 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,195 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=interfero_spectraldiversity_iw1, minutes=10 2025-01-20 17:09:49,196 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=interfero_spectraldiversity_iw2, minutes=13 2025-01-20 17:09:49,197 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=interfero_spectraldiversity_iw3, minutes=12 2025-01-20 17:09:49,197 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,197 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=merge_iw123, minutes=2 2025-01-20 17:09:49,198 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,198 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=caui_iw1_iw2_iw3, minutes=16 2025-01-20 17:09:49,199 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,199 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=cleaning_before_ts_iw1, minutes=1 2025-01-20 17:09:49,200 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=cleaning_before_ts_iw2, minutes=1 2025-01-20 17:09:49,200 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=cleaning_before_ts_iw3, minutes=1 2025-01-20 17:09:49,201 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,201 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=populate_iw1_iw2_iw3, minutes=10 2025-01-20 17:09:49,202 - nsb_build_synthesis - DEBUG - 1.1: in loop step_num={step_num} 2025-01-20 17:09:49,202 - nsb_build_synthesis - DEBUG - 1.2: in loop 2 step_name=make_iw1_iw2_iw3, minutes=1 2025-01-20 17:09:49,203 - nsb_build_synthesis - DEBUG - 2: after loop 2025-01-20 17:09:49,213 - nsb_build_synthesis - DEBUG - 3: computes nb minutes max_minutes={max_minutes} 2025-01-20 17:09:49,275 - nsb_build_synthesis - DEBUG - 4: after max_minutes 2025-01-20 17:09:49,329 - nsb_build_synthesis - DEBUG - 5: before save fig 1 2025-01-20 17:09:49,506 - nsb_build_synthesis - DEBUG - 6: before save fig 2 2025-01-20 17:09:49,615 - nsb_build_synthesis - DEBUG - 6: build_step_time_plot DONE 2025-01-20 17:09:49,615 - nsb_build_synthesis - DEBUG - building step_time_plot --> DONE 2025-01-20 17:09:49,618 - nsb_build_synthesis - INFO - DONE with ret_code 0 2025-01-20 17:09:49,618 - wsc_run_gdm_sar.py - INFO - build_synthesis done: going to publish step 2025-01-20 17:09:49,618 - wsc_run_gdm_sar.py - INFO - task created [, ] 2025-01-20 17:09:49,618 - wsc_run_gdm_sar.py - INFO - scheduled created 2025-01-20 17:09:49,618 - wsc_run_gdm_sar.py - INFO - notifying 2025-01-20 17:09:49,619 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'publish products', 'status': 'running', 'step': 'publish', 'progress': 90} 2025-01-20 17:09:49,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443 2025-01-20 17:09:49,887 - urllib3.connectionpool - DEBUG - https://gdm.poleterresolide.fr:443 "POST /scripts/setStatus HTTP/11" 200 None 2025-01-20 17:09:49,888 - wsc_run_gdm_sar.py - INFO - running publish 2025-01-20 17:09:49,888 - wsc_run_gdm_sar.py - INFO - -------------- starting step 10: publish ---------- 2025-01-20 17:09:49,888 - wf_scheduler - INFO - Sequential scheduler: running command=rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None command=rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/ formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 ; rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/backup_product.json formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 17:09:49,889 - wf_scheduler - INFO - launching cmd 0/2: command=rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 17:09:49,889 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 17:09:49,889 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 17:09:50,089 - wf_task - INFO - ret_code for rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ is 0 2025-01-20 17:09:50,089 - wf_scheduler - INFO - done with cmd 0/2: command=rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=1885551 2025-01-20 17:09:50,089 - wf_scheduler - INFO - launching cmd 1/2: command=rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/ formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 ; rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/backup_product.json formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=None 2025-01-20 17:09:50,089 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/ formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 ; rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/backup_product.json formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 17:09:50,089 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/ formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 ; rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/backup_product.json formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 17:13:12,934 - wf_task - INFO - ret_code for rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/ formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 ; rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/backup_product.json formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ is 0 2025-01-20 17:13:12,935 - wf_scheduler - INFO - done with cmd 1/2: command=rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/ formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3 ; rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/iw1_iw2_iw3/Product/backup_product.json formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/, workdir=/data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f id=1885563 2025-01-20 17:13:12,935 - wf_task - INFO - running rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ 2025-01-20 17:13:12,935 - wf_task - DEBUG - process-task: running command rsync -rLz --exclude={'*.int','*.unw','*.cor','*.rsc'} /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/logs formater.osug:/data/gdm-sar-uga/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f/ (from directory /data/ist-calcul17/local/gdm-sar/df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f) 2025-01-20 17:13:12,942 - wsc_run_gdm_sar.py - DEBUG - notify: sending notification to https://gdm.poleterresolide.fr/scripts/setStatus with data={'process_token': 'df6d97d8-4b7f-4ee0-becb-8b2a3fa0dd9f', 'log': 'workflow done', 'status': 'terminated', 'step': 'publish', 'progress': 90} 2025-01-20 17:13:12,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): gdm.poleterresolide.fr:443