First step achieved: having only one file from ODLK1 with unique squares by automate the process of adding new month result of the server.

For the periods of 2018-05 .. 2019-03 ODLK1 has: 2086518 unique CFs ODLS.

Are these unprocessed results?

Are these unprocessed results?

This are partial processed result: only unique CFs.

This are partial processed result: only unique CFs.

My results (quotes)

Results of the ODLK1 BOINC-project for May 2018

447612 unique CF ODLS

©2018 Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/LeB8/8BVqTzBfj

Created on June 24, 2018

Results of the ODLK1 BOINC-project for June 2018

486061 unique CF ODLS

©2018 Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/ENBK/Ljqnqji79

Created on July 2, 2018

Results of the ODLK1 BOINC-project for July 2018

407150 unique CF ODLS

©2018 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/56UQ/sNubiDF71

Created on August 2, 2018

Results of the ODLK1 BOINC-project for August 2018

400553 unique CF ODLS

©2018 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/92Zh/MwrNAcdW9

Created on September 3, 2018

Results of the ODLK1 BOINC-project for September 2018

407995 unique CFs ODLS

©2019 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/LcCy/rERVzo8qv


Results of the ODLK1 BOINC-project for October 2018

404478 unique CFs ODLS

©2019 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/oPwi/fv9FetwmS

The results are processed by XAVER in the autonomous subproject.

Results of the ODLK1 BOINC-project for November 2018

355634 unique CFs ODLS

©2019 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/G1Q2/Qy343sBQg


Results of the ODLK1 BOINC-project for December 2018

343240 unique CFs ODLS

©2019 (C) Progger & Stefano Tognon (ice00)

Link
https://cloud.mail.ru/public/GTHq/qv6ZXcn4R

The results are processed by XAVER in the autonomous subproject.

See topics
https://boinc.multi-pool.info/latinsquares/forum_thread.php?id=35
https://boinc.multi-pool.info/latinsquares/forum_thread.php?id=46

For January - March 2019 I did not process the results.

Your result is processed, so more solution should comes out from that procedure.

I forget to use the data from 2017 year, so I had to lunch the script again with all data before starting to use the true processing script.

Partial processed result from 11/2017..03/2019 on ODLK1: 2762444 unique CFs ODLS

ice00
your diagram is good, but need processed results.

What do you have with the zamyk.bat script?
Tomas Brada used this script in Linux and it worked.

By the way, let's see the raw results for January - March 2019.

For January

odlk3 – 148199 CF ODLK
odlkmax – 39307 CF ODLK

Total: 187506 CF ODLK.

Your data: 183067 unique CF ODLK.
Not unique: 4439 CF ODLK.

For February

odlk3 – 140372 CF ODLK
odlkmax – 36938 CF ODLK

Total: 177310 CF ODLK.

Your data: 170863 unique CF ODLK.
Not unique: 6447 CF ODLK.

For March

odlk3 – 145843 CF ODLK
odlkmax – 33344 CF ODLK

Total: 179187 CF ODLK.

Your data: 172525 unique CF ODLK.
Not unique: 6662 CF ODLK.

A lot of not unique CFs ODLK!
I assume that not unique CFs ODLK gives the odlkmax Application.
This is required to check.
If my assumption is correct, the odlkmax application needs to be adjusted or stopped.

ice00
your diagram is good, but need processed results.

What do you have with the zamyk.bat script?
Tomas Brada used this script in Linux and it worked.

Yes, zamyk.bat is ready to be lanched for processing the data, but I'm testing the staff.

A lot of not unique CFs ODLK!
I assume that not unique CFs ODLK gives the odlkmax Application.
This is required to check.
If my assumption is correct, the odlkmax application needs to be adjusted or stopped.

The new check I'm doing is to remove all duplicated that are present into the set of data being processed (so one month), to all the data previous collected in ODLK1.


So if you eliminate duplicates from ODLK3 and ODLKMAX on one month, you probanly find no duplicate, but if you check with all previous data, then you can find some duplicated that are removed.

The data that come out from this pre-processing will then go to process by zamyk.bat

The new check I'm doing is to remove all duplicated that are present into the set of data being processed (so one month), to all the data previous collected in ODLK1.

I understand it.

So if you eliminate duplicates from ODLK3 and ODLKMAX on one month, you probanly find no duplicate, but if you check with all previous data, then you can find some duplicated that are removed.

I understand that too.
However, the algorithm of the odlk3 and odlkmax Applications is such that there should not be so many duplicates.
This is a wrong work.

4 months (January - April) did you perform pretreatment?

If you process the results of each month with the zamyk.bat script, and then check the uniqueness of the obtained decisions regarding the entire database, there will be no duplicates.

We have a complete database as of January 1, 2019.
Why handle everything from the beginning?

The data that come out from this pre-processing will then go to process by zamyk.bat

When will it be? What year?

You need to process the monthly results with the zamyk.bat script (from January 2019) and compare the results with the existing database to remove duplicates.
All that you have done (pre-processing) is not necessary!

You do not want to understand what to do.
I will no longer explain to you.

I think that we will not see the processed results for January - April 2019.
We also will not see the general database after January 1, 2019.

Tomas Brada processed the results for January 2019 and received a general database.
At this all stopped.

If you cannot configure the zamyk.bat script, you can contact Tomas Brada or the script author Belyshev for help.

Quote

Результаты проекта ODLK1 за январь с. г. обработал доброволец tomas (предполагаю, что это Tomas Brada)
https://boinc.progger.info/odlk/forum_thread.php?id=102&postid=3034#3034

Ну, tomas теперь держатель нашей тотальной БД.
Если что, обращайтесь к нему :)
У меня нет всей БД с обоих проектов в одном файле, только отдельными частями, да и это по 1 января 2019 г.

See
https://boinc.progger.info/odlk/forum_thread.php?id=103&postid=3036#3036

Quote

Processing ODLK1_RESULT/unprocessed_result.2019.01.zip

0 local duplicates in file
4522 raw squares already in database (duplicate)
182984 new squares from raw file
0 non-odls in raw file
7 fancy cf dls of link length 4
373292 cf odls
4 cf odls already in database (duplicate)
373288 new cf odls
6088213 cf odlk total in database
369898 co-squares

New CF ODLS:
count[1]: 372573
count[2]: 714
count[3]: 1

See
https://boinc.progger.info/odlk/forum_thread.php?id=102&postid=3034#3034

ice00
do you have "6088213 cf odlk total in database" (as of February 1, 2019)?
Tomas Brada has it.

Now process the results for February 2019 with the zamyk.bat script and attach the solutions obtained to the common database "6088213 cf odlk total in database", removing duplicates.
Then do this for March 2019 results. And so on.

Is it hard to do?

The process of even old months is for some reasons:

1) be sure that the Linux scripts and programs works correctly by compare previous obtained result (so I have to process all the result)
2) have a script that manage a month and are able to integrate to DB obtained previously
3) generate easy to read statistic of result
4) generate log of operation than can catch eventual problems (this as the script run as Cron scheduled Job in the old server that has no battery, so power off can kill the process in a intermediate processing status)

The process of even old months is for some reasons:
. . .

Your reasons are very complex and numerous.

Hopefully by next Christmas we will see the processed results :)

The automatic script to get unique result works very well (I even simulate a shutdown in middle of work).

It processes all data, removing duplicate from previous months (look at pink color in graphic) and prepare clean moths file for being processed by zamyk.bat equivalent script.
The next month it will use already collected result and will process only that new data.

It takes 20 minute to reprocess all data.

Now I start to process with zamyk.bat, but processing time will be about 12h for a month (if my calculation are right) and the script will be to be calibrate a little.

UNIQUE SQUARES FOUND BY BOINC ODLK1 (not jet processed)

2017-11: initial number to preprocess= 8700 
2017-11: after preprocess= 8700 
2017-11: delta= 0 
2017-11: total DB= 8700 

2017-12: initial number to preprocess= 29824 
2017-12: after preprocess= 29824 
2017-12: delta= 0 
2017-12: total DB= 38524 

2018-01: initial number to preprocess= 107543 
2018-01: after preprocess= 107543 
2018-01: delta= 0 
2018-01: total DB= 146067 

2018-02: initial number to preprocess= 158849 
2018-02: after preprocess= 158849 
2018-02: delta= 0 
2018-02: total DB= 304916 

2018-03: initial number to preprocess= 201431 
2018-03: after preprocess= 201333 
2018-03: delta= 98 
2018-03: total DB= 506249 

2018-04: initial number to preprocess= 191592 
2018-04: after preprocess= 191498 
2018-04: delta= 94 
2018-04: total DB= 697747 

2018-05: initial number to preprocess= 203081 
2018-05: after preprocess= 203032 
2018-05: delta= 49 
2018-05: total DB= 900779 

2018-06: initial number to preprocess= 223258 
2018-06: after preprocess= 223228 
2018-06: delta= 30 
2018-06: total DB= 1124007 

2018-07: initial number to preprocess= 188762 
2018-07: after preprocess= 188748 
2018-07: delta= 14 
2018-07: total DB= 1312755 

2018-08: initial number to preprocess= 190148 
2018-08: after preprocess= 190146 
2018-08: delta= 2 
2018-08: total DB= 1502901 

2018-09: initial number to preprocess= 197960 
2018-09: after preprocess= 196883 
2018-09: delta= 1077 
2018-09: total DB= 1699784 

2018-10: initial number to preprocess= 196216 
2018-10: after preprocess= 195477 
2018-10: delta= 739 
2018-10: total DB= 1895261 

2018-11: initial number to preprocess= 173738 
2018-11: after preprocess= 172640 
2018-11: delta= 1098 
2018-11: total DB= 2067901 

2018-12: initial number to preprocess= 172245 
2018-12: after preprocess= 168088 
2018-12: delta= 4157 
2018-12: total DB= 2235989 

2019-01: initial number to preprocess= 187506 
2019-01: after preprocess= 183067 
2019-01: delta= 4439 
2019-01: total DB= 2419056 

2019-02: initial number to preprocess= 177310 
2019-02: after preprocess= 170863 
2019-02: delta= 6447 
2019-02: total DB= 2589919 

2019-03: initial number to preprocess= 179187 
2019-03: after preprocess= 172525 
2019-03: delta= 6662 
2019-03: total DB= 2762444 

2019-04: initial number to preprocess= 169818 
2019-04: after preprocess= 166229 
2019-04: delta= 3589 
2019-04: total DB= 2928673 

From tomorrow, the processing script will start to process all data (simulation seems to go fine)

Thanks

After some correction of the scripts, now it is running 24h/24h to process all the data of ODLK1

Here the partial result achieved:

Preprocessed= unique squares find each month by ODLK1 cpus
Mariage QF= marriage (ODLS) find by klpmd's zamyk eleboration
Ortogon = marriage (ODLS) find by ortogon's zamyk eleboration
Delta = number of squares that are present in previous months



Here the details about Mariage QF divided by each type (1, 2 ...)

Updated:

Updated:

Updated:




QF in logaritmic scale