Tasks crash with 'Invalid input format'

Message boards : Number crunching : Tasks crash with 'Invalid input format'
Message board moderation

To post messages, you must log in.

AuthorMessage
gemini8

Send message
Joined: 1 Dec 17
Posts: 5
Credit: 847,172
RAC: 1,775
Message 3629 - Posted: 2 Jan 2023, 5:56:24 UTC
Last modified: 2 Jan 2023, 6:10:06 UTC

Good morning.
I have several tasks crashing with this error (full stderr out):
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
Invalid input format

</stderr_txt>
]]>

This is happening on two machines, and I'll add a third for testing.

*edit*
It seems to me these are all odlkmax tasks dating from 2023-01-01 or 2023-01-02.
*end edit*
- - - - - - - - - -
Greetings, Jens
ID: 3629 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dr Who Fan

Send message
Joined: 10 Nov 17
Posts: 8
Credit: 194,049
RAC: 4
Message 3630 - Posted: 2 Jan 2023, 8:22:52 UTC - in response to Message 3629.  

I am seeing similar CRASH / ERROR in last 3 days I have accrued 13 INVALID tasks with exit code = 1 and the same "Invalid input format" message in stderr ONLY on odlkmax type tasks:
Stderr output
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<message>
Incorrect function.
(0x1) - exit code 1 (0x1)</message>
<stderr_txt>
Invalid input format

</stderr_txt>
]]>
ID: 3630 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
gemini8

Send message
Joined: 1 Dec 17
Posts: 5
Credit: 847,172
RAC: 1,775
Message 3631 - Posted: 2 Jan 2023, 8:42:34 UTC
Last modified: 2 Jan 2023, 8:55:00 UTC

The odlkmax tasks on the Ryzen 7 2700 running Ubuntu 20.4 that I added are failing as well. Will set it to NNW again.

My Ryzen 7 5700x running Ubuntu 22.4 LTS did finish some fresh odlkmax tasks which validated, my Ryzen 7 3700x running Ubuntu 20.4 LTS only did so with tasks sent to it before 31 Dec 2022, 19:48:00 UTC.
The 5700x had my first failed tasks which was sent out 31 Dec 2022, 21:48:03 UTC.

I have pending tasks, so there might be more that don't have issues.

My computers are visible.
- - - - - - - - - -
Greetings, Jens
ID: 3631 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dr Who Fan

Send message
Joined: 10 Nov 17
Posts: 8
Credit: 194,049
RAC: 4
Message 3632 - Posted: 2 Jan 2023, 17:50:40 UTC - in response to Message 3630.  
Last modified: 2 Jan 2023, 17:57:37 UTC

PROJECT AMINS >> YOU HAVE A PROBLEM THAT NEEDS FIXING: Now up to 20 ERRORED "oldmax" task....
BAD DATA FILES BEING SENT OUT POSSIBLY?

I have disabled request for the max task on my PC's to see if the other app/data is flakey.
---------
edit to add Sent PM to Project Admin Natalia to check into the errors

ID: 3632 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile PecosRiverM

Send message
Joined: 8 Jun 18
Posts: 1
Credit: 46,403,632
RAC: 0
Message 3633 - Posted: 2 Jan 2023, 21:39:08 UTC

I have 5700 of these. Not all work units are affected..
ID: 3633 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dr Who Fan

Send message
Joined: 10 Nov 17
Posts: 8
Credit: 194,049
RAC: 4
Message 3634 - Posted: 2 Jan 2023, 23:04:43 UTC

So far it's only the oldkmax tasks with input error message.
ID: 3634 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
k

Send message
Joined: 28 Jan 18
Posts: 3
Credit: 60,935
RAC: 0
Message 3635 - Posted: 3 Jan 2023, 14:09:58 UTC

same situation.
ID: 3635 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Natalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 3636 - Posted: 3 Jan 2023, 14:30:38 UTC
Last modified: 3 Jan 2023, 14:45:27 UTC

edit to add Sent PM to Project Admin Natalia to check into the errors

I'm not a project administrator, I don't have access to the server, and I can't check for errors.

Wait for the response of the ice00 project administrator.

You can send a PM to the project administrator - ice00.
ID: 3636 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Natalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 3637 - Posted: 3 Jan 2023, 14:38:31 UTC

ice00
If you have added new tasks to the odlkmax Application, please check their format.
ID: 3637 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Natalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 3638 - Posted: 3 Jan 2023, 15:09:48 UTC
Last modified: 3 Jan 2023, 15:11:18 UTC

I see task information

Задание 434274288
Имя odlkmax_7731_1672492482.285039_0
Задача 374903308
Создан 31 Dec 2022, 13:14:46 UTC
Отправлен 1 Jan 2023, 10:15:49 UTC
Крайний срок отчёта 8 Jan 2023, 10:15:49 UTC
Получен 1 Jan 2023, 10:16:59 UTC
Состояние сервера Завершено
Результат выполнения Ошибка вычислений
Состояние клиента Ошибка расчётов
Статус выхода 1 (0x00000001) Unknown error code

But I don't see the task!
ID: 3638 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dingo
Avatar

Send message
Joined: 6 Dec 17
Posts: 2
Credit: 3,018,383
RAC: 0
Message 3639 - Posted: 5 Jan 2023, 1:40:08 UTC
Last modified: 5 Jan 2023, 1:40:38 UTC

I just noticed this error. I have over 700 work units that are invalid. This is a recent one https://boinc.multi-pool.info/latinsquares/result.php?resultid=435701916

Proud Founder and member of



Have a look at my WebCam
ID: 3639 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 189
Credit: 57,756
RAC: 5
Message 3640 - Posted: 5 Jan 2023, 9:10:16 UTC - in response to Message 3639.  

hi,

it was added more odlmax rule 27 workunits recently as done one year ago:

New WUs added to the project
On January 4 of this year, 112118 WUs from rule 26 were added to the project.
Added 94372 WUs from rule 27 today.


At that time only 94372 units were added of the more than 300.000 generated as there were reached a good amount to process overall.

Now that the WU were reduced, news were added to restore the good amount.

WU were generated a year ago an stay ready to be processed when needed, so they were of the same type that was being processed during this year.

Being investigated why there are problems on that.
ID: 3640 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Natalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 3641 - Posted: 5 Jan 2023, 9:29:27 UTC
Last modified: 5 Jan 2023, 9:33:57 UTC

At that time only 94372 units were added of the more than 300.000 generated as there were reached a good amount to process overall.

ice00
Last February 1, I sent you the WU_from_rule27.txt file, which contains 94372 WUs.
If all of these were then added to the Application, then there can be no other WUs in rule 27.

What have you added as new WUs in rule 27?
Obviously these are the wrong WUs.
All added WUs in rule 27 must be removed.

To add new WUs, take the following rule.
ID: 3641 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 189
Credit: 57,756
RAC: 5
Message 3642 - Posted: 5 Jan 2023, 10:31:41 UTC - in response to Message 3641.  

hi,

fortunately the error is catch (and source generator fixed).

The procedure that generate WU from your txt file did not breaks correctly when finishing the input file, so it still generate WU with that "null" invalid data that makes the WU to be rejected.

That WU stay in server ready to be added and apparently they seems corrected.
To avoid other possible file with different kind of errors (not contemplate with the above problem) that could pass the generation, I will add a file dimension check before WU generation to skip such file
ID: 3642 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Natalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3083
Credit: 0
RAC: 0
Message 3643 - Posted: 5 Jan 2023, 11:09:55 UTC - in response to Message 3642.  
Last modified: 5 Jan 2023, 11:32:06 UTC

The procedure that generate WU from your txt file did not breaks correctly when finishing the input file, so it still generate WU with that "null" invalid data that makes the WU to be rejected.

I don't understand what you are talking about.
The WUs in my file have been used for almost a year and everything has been correct so far.
Why is the generation routine failing just now?
And during the year this procedure was completed correctly?

You wrote that you added new WUs for rule 27.
At that time only 94372 units were added of the more than 300.000 generated as there were reached a good amount to process overall.

What new WUs for rule 27 have you added?
My file had exactly 94372 WUs.
Which "of the more than 300.000 generated" are you talking about?

Please show me the WU from rule 27 which is not correctly generated.

PS. The task is given by two squares and does not depend on the end of the input file.
If you correctly set the 94372nd WU, where does the end of the input file come from?

This is WU 94372

0 9 8 4 7 6 3 2 5 1
2 1 4 6 5 3 8 9 0 7
6 0 2 9 8 4 1 3 7 5
1 7 6 3 9 8 4 5 2 0
9 2 3 0 4 7 5 8 1 6
8 3 0 1 2 5 7 6 9 4
7 4 5 8 0 9 6 1 3 2
3 5 9 2 6 1 0 7 4 8
4 6 7 5 1 2 9 0 8 3
5 8 1 7 3 0 2 4 6 9

0 9 8 4 7 6 3 2 5 1
8 1 7 5 3 2 9 4 0 6
4 5 2 9 6 0 1 3 7 8
9 7 6 3 1 8 4 0 2 5
6 2 0 1 4 7 5 8 9 3
3 8 1 6 2 5 0 9 4 7
7 4 5 8 0 9 6 1 3 2
2 3 9 0 5 1 8 7 6 4
1 6 3 7 9 4 2 5 8 0
5 0 4 2 8 3 7 6 1 9
ID: 3643 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 189
Credit: 57,756
RAC: 5
Message 3644 - Posted: 5 Jan 2023, 13:05:43 UTC

Wrong file deleted.

Now Boinc itself will take up up to 7 days to clean up tasks without a file to download.


Unfortunately BOINC workunit table is not the best way to store data for automatic process as the filename is inside an xml tag, that means that deleting a WU using boinc itself needs around 1 minute for each WU for getting the WU ID from file name and then call the boinc script to delete it and all this should be done with server stopped!
ID: 3644 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 189
Credit: 57,756
RAC: 5
Message 3645 - Posted: 8 Jan 2023, 18:42:51 UTC

hi

now all WU without an file to download are marked as not to be processed (after some days of our hidden working and 1,3h of Boinc being stopped: this was for accelerate the 7 days timeout of WU) , so Boinc already remove them from the list of available tasks.

Now you can reactivate the oldkmax download

Tnaks
ID: 3645 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
gemini8

Send message
Joined: 1 Dec 17
Posts: 5
Credit: 847,172
RAC: 1,775
Message 3646 - Posted: 9 Jan 2023, 20:19:04 UTC

Working fine again, thx!
- - - - - - - - - -
Greetings, Jens
ID: 3646 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Tasks crash with 'Invalid input format'


©2024 Progger & Stefano Tognon (ice00)