Thread 'The processing of results'

Message boards : Number crunching : The processing of results
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · Next

AuthorMessage
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 766 - Posted: 15 Jun 2019, 16:37:06 UTC

Updated:





the script need some fixed as global DB divided by types is not as planner, so it is probably that after I will restart the process again, so new improved result will comes after a week.
ID: 766 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 768 - Posted: 19 Jun 2019, 18:43:21 UTC

Updated:



ID: 768 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 770 - Posted: 20 Jun 2019, 17:02:34 UTC

Updated:



ID: 770 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 771 - Posted: 22 Jun 2019, 8:02:04 UTC

Updated:



ID: 771 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 772 - Posted: 23 Jun 2019, 9:56:50 UTC

Updated:



ID: 772 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 775 - Posted: 27 Jun 2019, 18:15:55 UTC - in response to Message 772.  

Updated:



ID: 775 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 780 - Posted: 1 Jul 2019, 18:15:50 UTC

Updated:





ID: 780 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 785 - Posted: 14 Jul 2019, 7:10:11 UTC

Calculation finished:





Now I have to check the total DB generated to be sure all is fine and then total result could be published.
ID: 785 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3086
Credit: 0
RAC: 0
Message 786 - Posted: 14 Jul 2019, 10:58:53 UTC

ice00
Can you show the solution Type3 for June 2019?
ID: 786 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 788 - Posted: 21 Jul 2019, 18:10:52 UTC

hi,

this is the solution of type 3 of June

0 9 8 7 6 4 5 2 3 1
9 1 5 6 3 2 8 4 0 7
6 8 2 4 0 1 7 3 9 5
7 2 1 3 9 0 4 5 6 8
1 5 7 9 4 6 0 8 2 3
3 6 4 2 8 5 9 1 7 0
8 3 0 5 2 7 6 9 1 4
5 0 9 8 1 3 2 7 4 6
4 7 3 0 5 9 1 6 8 2
2 4 6 1 7 8 3 0 5 9

p.s
I still controlling the global DB generated automatically before public all data.
ID: 788 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3086
Credit: 0
RAC: 0
Message 789 - Posted: 21 Jul 2019, 23:22:55 UTC
Last modified: 22 Jul 2019, 2:55:34 UTC

You are not mistaken?

This solution was found in the ODLK1 project in January 2019.
See
https://boinc.progger.info/odlk/forum_thread.php?id=42&postid=3021#3021

0 1 2 3 4 5 6 7 8 9
1 2 0 4 3 7 9 8 6 5
9 0 4 6 1 8 3 2 5 7
8 3 9 7 2 6 5 4 0 1
6 8 7 9 5 1 2 0 4 3
4 9 6 1 8 3 0 5 7 2
5 7 8 0 6 2 1 9 3 4
2 5 3 8 9 4 7 6 1 0
3 4 5 2 7 0 8 1 9 6
7 6 1 5 0 9 4 3 2 8

This solution is in format 1.

You make a check for unique solutions.
How did you get the same solution?
ID: 789 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 790 - Posted: 22 Jul 2019, 18:32:30 UTC - in response to Message 789.  

Yes, error in copy the file, here the correct one:

more output_kf_3_2019-06.txt 
0 7 3 9 6 8 5 2 4 1
2 1 4 7 5 9 8 6 0 3
6 5 2 4 1 7 0 3 9 8
1 0 7 3 8 2 4 9 5 6
8 9 1 2 4 6 3 5 7 0
9 4 0 6 7 5 1 8 3 2
7 3 5 8 9 1 6 0 2 4
4 8 9 1 3 0 2 7 6 5
3 2 6 5 0 4 9 1 8 7
5 6 8 0 2 3 7 4 1 9
ID: 790 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3086
Credit: 0
RAC: 0
Message 791 - Posted: 23 Jul 2019, 3:19:53 UTC - in response to Message 790.  

Yes, this is a unique solution.
This is the 99th type 3 solution in our common database.

See
https://boinc.progger.info/odlk/forum_thread.php?id=42

Thanks.
ID: 791 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 793 - Posted: 28 Jul 2019, 15:20:13 UTC

With 60% of result verification done, just a statistic on result of ODLK1 up to 06/2019:

DB of Ortogon squares: 14.307.219

DB of Mariage squares: 6.673.441

I hope to have the complete result in some days.
ID: 793 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3086
Credit: 0
RAC: 0
Message 797 - Posted: 10 Aug 2019, 16:19:55 UTC - in response to Message 793.  

I hope to have the complete result in some days.

You wanted to say - in a few weeks? :)

Have you processed the results for July? Are there type 3 or type 4 solutions?
ID: 797 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 798 - Posted: 12 Aug 2019, 10:11:45 UTC

The control pops up that Ortogon result were collected wrong in global DB (sos_operate did not works correctly with them), so all process is restarted after correctly the script.
Now it is at arrived to process 2018-07.

Else the way sos_operate works means it load all DB in memory (and for Ortogon it is about 3GB) so it is over the memory of old server, so it cannot create the global DB of Ortogon that must to be created in the PC with the months result (and so the script is not able to eliminate the duplicated that can happen from month to global DB. Global DB created into the PC will have duplicate eliminated).

We are studying to have global DB in compact form to eliminate this problem.

2019-07 is not processed as the start of the processing again of result was before having those result in it, so it is appended in the queue (it will be processed in 14 days at the current rate of going on server).

Unprocessed result are on Gdrive: https://drive.google.com/open?id=1wIf97_bLDvnb1WalOsjU9vODZqRZ_WNw
ID: 798 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ProfileNatalia Makarova
Project scientist
Avatar

Send message
Joined: 22 Oct 17
Posts: 3086
Credit: 0
RAC: 0
Message 799 - Posted: 12 Aug 2019, 12:11:35 UTC

Is the global database problem unsolvable?
We will not see the general database of the ODLK1 project?

11 days of August have passed. July results still not processed?
ID: 799 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 801 - Posted: 12 Aug 2019, 12:58:03 UTC

Server CPU require about 1.5 days to process a month (using the automatic script that eliminate duplicated and generate statistics and global DB).

-rw-rw-r--  1 ice ice 132943872 Aug 12 03:08 DB_kf_1_processed.txt
-rw-rw-r--  1 ice ice   1002972 Aug 12 03:08 DB_kf_2_processed.txt
-rw-rw-r--  1 ice ice      1696 Aug 12 03:08 DB_kf_3_processed.txt
-rw-rw-r--  1 ice ice       636 Aug 12 03:08 DB_kf_4_processed.txt
-rw-rw-r--  1 ice ice 498043968 Aug 12 02:51 DB_kfall_processed.txt
-rw-rw-r--  1 ice ice 498043968 Aug 12 02:54 DB_kfmate_processed.txt
-rw-rw-r--  1 ice ice 498043968 Aug 12 02:49 DB_klpmd_processed.txt
-rw-rw-r--  1 ice ice    110592 Aug 12 03:03 DB_ortogon_processed.txt


DB_ortogon global DB get reset when it reached 2GB, so I can get it by processing in PC (with 8GB or RAM) by hand from the months DB:

ice@pine64:/opt/odlk1/processed$ ls  output_ortogon*
output_ortogon_2017-11.txt  output_ortogon_2017-12.txt  output_ortogon_2018-01.txt  output_ortogon_2018-02.txt  output_ortogon_2018-03.txt  output_ortogon_2018-04.txt  output_ortogon_2018-05.txt  output_ortogon_2018-06.txt


DB of Mariage squares up to 2019-06 is about 1.3GB
ID: 801 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 813 - Posted: 27 Aug 2019, 19:16:47 UTC

During the holiday there were some blackout of current and so processing on old server stop (it is a manual starting process to be sure it go correctly). Fortunately battery pack on ODLK1 go perfectly and server were always up.

With that stop that was happened the processing probably now finish in 5 days.

I will start to check if the processing during holiday were done correctly on Saturday.
ID: 813 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
ice00
Project administrator
Project developer

Send message
Joined: 28 Oct 17
Posts: 221
Credit: 59,304
RAC: 11
Message 818 - Posted: 31 Aug 2019, 17:31:14 UTC

Result where finished in processing.

Here the result up to 31/07/2019 on ODLK1:

Unprocessed squares find with BOINC: 3377513
Mariage squares: 6977782
divided into:
KF1: 6964000
KF2: 13749
KF3: 24
KF4: 9

Ortogon square: 13897285

Tomorrow I can process August, so then as soon as it finish (maybe i 2 days) I can publish the complete DB of ODLK1





ID: 818 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · Next

Message boards : Number crunching : The processing of results

©2024 ©2024 Progger & Stefano Tognon (ice00) & Reese